An image classification method, device, equipment and computer readable storage medium

Abstract

The present application relates to the technical field of image processing, in particular to an image classification method, device, equipment and computer readable storage medium, each graphics processor utilizes the initial width sub-image classification model matched with the sub-image data set sent by the central processing unit to independently and parallelly train each width sub-image classification model to obtain the trained each width sub-image classification model; according to the merging data set, the similarity between the feature images of the node output of each layer of each width sub-image classification model under the merging data set is analyzed to determine the node with the maximum similarity between each width sub-image classification model. According to the merging weight parameters transmitted by the central processing unit, the feature images corresponding to the node with the maximum similarity are analyzed layer by layer to obtain the image classification model after width merging, and parallel training is carried out, the acquired image to be identified is analyzed by using the trained image classification model to determine the image category, and the communication overhead of image classification is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image classification method, apparatus, device and computer-readable storage medium. Background Art

[0002] In recent years, deep neural network models have been widely used due to their outstanding performance. However, this improvement in model accuracy has also led to an explosive growth in data and computational complexity, resulting in reduced model training efficiency. Currently, efforts to improve training efficiency focus on algorithmic models and hardware acceleration. Hardware acceleration solutions based on distributed computing are a key factor influencing model performance.

[0003] Model training is divided into full parameter training and partial parameter training. Partial parameter training targets pre-trained general models and is suitable for model deployment applications in different scenarios. Full parameter training targets new model architectures or general model variants without pre-training and is suitable for model structure improvement and development.

[0004] In image classification applications, full parameter training can be used to obtain an image classification model, which can then be used to identify image categories. However, full parameter training requires a large amount of computation and is inefficient. Current distributed computing solutions commonly use data parallelism and model parallelism, but these parallel computations still require data exchange, resulting in significant communication overhead.

[0005] It can be seen that how to reduce the communication overhead of image classification is a problem that those skilled in the art need to solve. Summary of the Invention

[0006] The purpose of the embodiments of the present invention is to provide an image classification method, apparatus, device, and computer-readable storage medium, which can solve the problem of high communication overhead in image classification.

[0007] To solve the above technical problems, an embodiment of the present invention provides an image classification method applicable to heterogeneous platforms, comprising:

[0008] Each graphics processor independently and in parallel trains its matched initial width sub-image classification model using the sub-image dataset sent by the central processor to obtain trained width sub-image classification models; the initial width sub-image classification model is obtained by the central processor splitting the initial image classification model according to the width dimension;

[0009] Each of the graphics processors analyzes, based on the merged data set sent by the central processing unit, the similarities between the feature images output by the nodes of each layer of the width sub-image classification model under the merged data set, so as to determine the node of each layer in each width sub-image classification model that has the greatest similarity with the node in the corresponding layer of each other width sub-image classification model;

[0010] The central processor adds a merging weight parameter between the node of each layer in each width sub-image classification model and the node with the greatest similarity in the corresponding layer in other width sub-image classification models; and transmits the merging weight parameter to each of the graphics processors;

[0011] Each of the graphics processors analyzes, layer by layer, the feature images corresponding to the nodes of each layer in each width sub-image classification model and the nodes with the greatest similarity in the corresponding layers of other width sub-image classification models according to the merging weight parameters, to obtain a width-merged image classification model;

[0012] Each of the graphics processors performs parallel training on the width-merged image classification models deployed therein, and uses the trained image classification models to analyze the acquired image to be identified to determine the image category to which the image to be identified belongs.

[0013] The present invention also provides an image classification device suitable for heterogeneous platforms, the device comprising a graphics processor and a central processing unit; the graphics processor comprising a parallel training unit, a node determination unit, an analysis unit and a recognition unit; the central processing unit comprising an addition unit and a transmission unit;

[0014] The parallel training unit is configured to independently and parallelly train the initial width sub-image classification model matched to the sub-image dataset sent by the central processing unit to obtain trained classification models for each width sub-image; the initial width sub-image classification model is obtained by the central processing unit splitting the initial image classification model according to the width dimension;

[0015] The node determination unit is configured to analyze, based on the merged data set sent by the central processing unit, the similarities between the feature images output by the nodes of each layer of the width sub-image classification model under the merged data set, so as to determine the node of each layer in each width sub-image classification model that has the greatest similarity with the node in the corresponding layer of each other width sub-image classification model;

[0016] The adding unit is used to add a merging weight parameter between the nodes of each layer in each width sub-image classification model and the nodes with the greatest similarity in the corresponding layers in other width sub-image classification models;

[0017] The transmission unit is configured to transmit the combined weight parameter to each of the graphics processors;

[0018] The analysis unit is configured to analyze, in accordance with the merging weight parameter, the feature images corresponding to the nodes of each layer in each width sub-image classification model and the nodes with the greatest similarity in the corresponding layers of the other width sub-image classification models, layer by layer, in each of the graphics processors, to obtain a width-merged image classification model;

[0019] The recognition unit is used for each of the graphics processors to perform parallel training on the width-merged image classification models deployed respectively, and use the trained image classification models to analyze the acquired image to be recognized to determine the image category to which the image to be recognized belongs.

[0020] An embodiment of the present invention further provides an image classification device, comprising:

[0021] memory for storing computer programs;

[0022] A processor is used to execute the computer program to implement the steps of the above-mentioned image classification method.

[0023] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned image classification method are implemented.

[0024] As can be seen from the above technical solution, each graphics processor independently and in parallel trains its matching initial width sub-image classification model using the sub-image dataset sent by the central processor to obtain trained width sub-image classification models. The initial width sub-image classification model is obtained by the central processor splitting the initial image classification model according to the width dimension. Based on the merged dataset sent by the central processor, each graphics processor analyzes the similarity between the feature images output by the nodes of each width sub-image classification model at each layer of the merged dataset to determine the nodes with the greatest similarity in each layer of each width sub-image classification model and the nodes with the greatest similarity in the corresponding layer of each other width sub-image classification model. The central processor adds a merging weight parameter between the nodes in each layer of each width sub-image classification model and the nodes with the greatest similarity in the corresponding layer of each other width sub-image classification model; the merging weight parameter is transmitted to each graphics processor. Each graphics processor analyzes the feature images corresponding to the nodes in each layer of each width sub-image classification model and the nodes with the greatest similarity in the corresponding layer of each other width sub-image classification model according to the merging weight parameter, layer by layer, to obtain the width-merged image classification model. Each graphics processor trains its own deployed width-merged image classification model in parallel, using the trained image classification model to analyze the acquired image to be identified to determine the image category to which the image belongs. The present invention has the beneficial effect of employing a staged model training approach. In the first stage, the initial image classification model is split into multiple width-sub-image classification models, enabling parallel training of multiple width-sub-image classification models. The training process of each width-sub-image classification model does not require data exchange, significantly reducing the communication overhead incurred during the training process. After each width-sub-image classification model is trained, a merging weight parameter is added based on the node with the greatest similarity corresponding to each node. Based on the merging weight parameter, the feature images between the nodes with the greatest similarity are analyzed layer by layer to merge the multiple width-sub-image classification models, thereby obtaining a width-merged image classification model. In the second stage, the same width-merged image classification model is deployed on different graphics processors using a data-parallel approach for parallel training. This second stage of training significantly improves the accuracy of the image classification model. Through the coordinated training of these two stages, a balance between training accuracy and efficiency is achieved. By using the trained image classification model to analyze the acquired image to be identified, the image category to which the image to be identified belongs can be accurately determined. While ensuring the accuracy of image category recognition, the communication overhead of image classification is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0026] Figure 1 A flowchart of an image classification method provided by an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of data set splitting and initial image classification model splitting provided by an embodiment of the present invention;

[0028] Figure 3 A schematic diagram of parallel training of multiple initial width sub-image classification models provided by an embodiment of the present invention;

[0029] Figure 4 A flowchart of a method for merging classification models of sub-images of different widths provided by an embodiment of the present invention;

[0030] Figure 5 A schematic diagram of merging all nodes of each layer of a classification model of multiple width sub-images provided by an embodiment of the present invention;

[0031] Figure 6 A schematic diagram of a width-merged image classification model provided by an embodiment of the present invention;

[0032] Figure 7 A schematic structural diagram of an image classification device provided by an embodiment of the present invention;

[0033] Figure 8 A structural diagram of an image classification device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0035] The terms "including" and "having," as used in the present description and accompanying drawings, and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements and may include steps or elements that are not listed.

[0036] In order to enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0037] Next, an image classification method provided by an embodiment of the present invention is described in detail. Figure 1 This is an architecture diagram of a heterogeneous platform for implementing an image classification method provided by an embodiment of the present invention. Figure 1 The example above takes N graphics processors and one central processing unit as an example.

[0038] In practical applications, the central processing unit can split the image dataset into multiple sub-image datasets; split the initial image classification model into multiple initial sub-image classification models according to the width dimension, which is the same as the number of sub-image datasets; and transmit each sub-image dataset and its matching initial width sub-image classification model to their corresponding graphics processors.

[0039] Each graphics processor uses the sub-image data set sent by the central processing unit to independently and parallelly train its matched initial width sub-image classification model to obtain trained width sub-image classification models.

[0040] For ease of distinction, the multiple sub-image datasets obtained by splitting can be referred to as sub-image dataset 1 to sub-image dataset N. The multiple initial sub-image classification models obtained by splitting can be referred to as initial width sub-image classification model 1 to initial width sub-image classification model N. The trained width sub-image classification models can be referred to as width sub-image classification model 1 to width sub-image classification model N.

[0041] To merge multiple width-sub image classification models to form a complete image classification model, the central processing unit (CPU) can select a merged dataset from the image dataset according to a set ratio and transmit the merged dataset to each graphics processing unit (GPU). Based on the merged dataset sent by the CPU, each GPU can analyze the similarity between the feature images output by the nodes at each layer of each width-sub image classification model under the merged dataset to determine the nodes at each layer of each width-sub image classification model that are most similar to the nodes at the corresponding layers of other width-sub image classification models.

[0042] The central processing unit adds a merging weight parameter between the node of each layer in each width sub-image classification model and the node with the greatest similarity in the corresponding layer in other width sub-image classification models; and transmits the merging weight parameter to each graphics processor.

[0043] Each graphics processor analyzes the feature images corresponding to the nodes in each layer of each width sub-image classification model and the nodes with the greatest similarity in the corresponding layers of other width sub-image classification models layer by layer according to the merging weight parameters to obtain the image classification model after width merging.

[0044] In order to improve the recognition accuracy of the width-merged image classification model, the width-merged image classification model can be deployed on multiple graphics processors at the same time for parallel training.

[0045] Figure 1 The example above takes the image classification model after width merging deployed on M GPUs as an example. In actual applications, there is no limit on the number of parallel training for the image classification model after width merging. Figure 1 The deployment of the width-merged image classification model on M GPUs is only an example. According to actual training requirements, the width-merged image classification model can be deployed on more or fewer GPUs.

[0046] Each graphics processor trains its own deployed width-merged image classification model in parallel, and uses the trained image classification model to analyze the acquired image to be identified to determine the image category to which the image to be identified belongs.

[0047] In an embodiment of the present invention, an image classification model can be obtained by parallel training based on a new structural model. The image classification model can adopt a convolutional neural network (CNN) structural model in which neurons in the front and back layers are fully connected and different neurons are connected in the same way. The model structure is mainly composed of convolutional layers and fully connected layers. The image classification model contains multiple layers, each layer has its corresponding neurons, and a neuron can be regarded as a node. Each node can implement image analysis and processing.

[0048] Commonly used large-scale image datasets include ImageNet, ImageNet-21k, and the JFT-300M Large-Scale Image Classification Dataset, which correspond to medium-sized, large-scale, and very large-scale datasets, respectively. ImageNet contains 1.28M images across 1,000 categories, ImageNet-21k contains 14.2M images across 21,000 categories, and JFT-300 contains 300M images across 18,291 categories.

[0049] In an embodiment of the present invention, the central processing unit may obtain an initial image dataset from a public dataset based on the model scale and accuracy requirements of the image classification model. For example, when the model scale is large and the accuracy requirements are high, an image dataset may be selected from JFT-300.

[0050] Due to the diversity of image dataset sources, to improve the quality of the image dataset, the central processing unit may preprocess each image frame contained in the image dataset. Preprocessing methods may include brightness normalization, noise filtering, scale normalization, cropping, etc. In practical applications, the central processing unit may perform brightness normalization, noise filtering, scale normalization, and / or cropping on the initial image dataset to obtain the image dataset.

[0051] After acquiring the image dataset, the central processing unit can split the image dataset into multiple sub-image datasets with balanced categories according to training requirements.

[0052] To improve model training efficiency, image classification model training can be performed on heterogeneous computing platforms. Considering that the number of width sub-image classification models is the same as the number of sub-image datasets, when determining the number of sub-image datasets, training requirements should include the number of parallel units on the heterogeneous computing platform, the model width of the image classification model, and the accuracy requirements.

[0053] In an embodiment of the present invention, the central processing unit may determine the number of sub-image data sets according to the number of parallel units of the heterogeneous computing platform, the model width of the image classification model, and the accuracy requirement.

[0054] In a specific implementation, the central processing unit can determine the upper limit of the splitting based on the number of parallel units of the heterogeneous computing platform and the model width of the image classification model; determine the lower limit of the splitting based on the accuracy requirements of the image classification model; and select a value between the lower limit and the upper limit as the number of sub-image data sets.

[0055] In practical applications, any integer value between the lower split limit and the upper split limit can be selected as the number of sub-image datasets. In addition, the average of the lower split limit and the upper split limit can also be directly used as the number of sub-image datasets.

[0056] After the number of sub-image data sets is determined, the image data set may be split into sub-image data sets whose number is the same as the number of sub-image data sets according to a class balance principle.

[0057] Taking the image dataset D as an example, it can be split into s sub-image datasets {D1, ..., D s The principle of balanced splitting means that each sub-image dataset should contain images corresponding to all image categories, and the number of images corresponding to each image category is the same or similar. In other words, all samples of each image category in dataset D are evenly distributed to each sub-image dataset.

[0058] In the embodiment of the present invention, the central processing unit can split the initial image classification model into multiple initial width sub-image classification models of the same number according to the number of sub-image data sets. In the above example, the number of sub-image data sets is s, and the initial image classification model is also split into s initial width sub-image classification models {subM1…subM s}.

[0059] When splitting the model, the initial image classification model can be split into multiple initial width sub-image classification models with the same number of sub-image datasets, keeping the model depth unchanged and the model width evenly divided. The depth of the initial image classification model is the number of network layers L, and the width is the number of nodes in each layer n. Then, the number of layers in the initial width sub-image classification model after splitting is also L, and the number of nodes in each layer sn = n / s.

[0060] Taking into account that there may be connection relationships between the nodes of the different initial width sub-image classification models after splitting, in order to ensure the independent training of each initial width sub-image classification model, the connection relationships between the nodes in each initial width sub-image classification model can be retained, and the connection relationships between the nodes of different initial width sub-image classification models can be deleted to obtain the final multiple initial width sub-image classification models.

[0061] Figure 2 A schematic diagram of data set splitting and initial image classification model splitting provided by an embodiment of the present invention, Figure 2 It contains two parts. The leftmost column corresponds to the sub-dataset after splitting, and the remaining part corresponds to the initial width sub-image classification model after splitting. The sub-datasets after splitting are data1, data2, ..., dataN. Each data set corresponds to an initial width sub-image classification model. Figure 2 In the figure, the dotted line is used as the dividing line to divide the sub-image classification models with different initial widths. Figure 2 As can be seen from the figure, the initial classification model is split into N initial width sub-image classification models. The depth of each initial width sub-image classification model is the same. The width of all initial width sub-image classification models is spliced ​​together to obtain a complete initial image classification model. Each initial width sub-image classification model contains multiple layers. Figure 2 Each column in represents a layer. Figure 2 The dots in each layer represent a node, and the connecting lines between nodes represent the connection relationship between nodes.

[0062] from Figure 2It can be seen that by splitting in a way that the depth remains unchanged and the width is evenly divided, multiple initial width sub-image classification models can be obtained. There is still a connection relationship between the nodes of these multiple initial width sub-image classification models. In order to ensure the independent training of each initial width sub-image classification model, the connection relationship between the nodes in each initial width sub-image classification model can be retained, and the connection relationship between the nodes of different initial width sub-image classification models can be deleted to obtain the final multiple initial width sub-image classification models.

[0063] The parallel training method proposed in this paper is primarily applicable to newly developed CNN models in which both front and back nodes are fully connected, using the same connectivity scheme. Taking the novel lightweight CNN model VanillaNet as an example, the basic structure of the VanillaNet network primarily consists of convolutional layers and fully connected layers. The model structure is streamlined, with both front and back nodes fully connected, and the initial model parameters are typically random numbers.

[0064] Combined with the above examples, different sub-image data sets {D1…D s} Randomly assigned to different initial width sub-image classification models {subM1…subM s} constitutes a sub-training task, so that each initial width sub-image classification model learns different model parameters. The training of each initial width sub-image classification model is completely independent. Therefore, deploying it on a parallel computing platform, such as a multi-core, multi-computing device and other parallel computing system, can have a greater acceleration effect.

[0065] In the embodiment of the present invention, the initial width sub-image classification model is deployed on a graphics processor as an example for description.

[0066] Each initial width sub-image classification model is trained in a similar manner. For example, the GPU can input a sub-dataset into the initial width sub-image classification model. The initial width sub-image classification model consists of multiple layers, each of which outputs a corresponding feature image. The final layer of the initial width sub-image classification model outputs a classification probability prediction vector. This classification probability prediction vector contains the predicted classification probability for each image class in the sub-image dataset.

[0067] Figure 3 A schematic diagram of parallel training of multiple initial width sub-image classification models provided by an embodiment of the present invention, Figure 3 The leftmost column in the represents the sub-image datasets, namely data1, data2, ..., dataN. Each dataset corresponds to an initial width sub-image classification model. Figure 3The right part of the first arrow represents the initial width sub-image classification model after splitting. The number of initial width sub-image classification models is the same as the number of sub-image datasets, which are subM1, subM2, ..., subM N , a virtual image frame corresponds to an initial width sub-image classification model. The last layer of each initial width sub-image classification model will output the classification probability prediction vector, which are y1, y2, ..., y N .

[0068] In an embodiment of the present invention, in order to achieve parallel training of multiple initial width sub-image classification models, multiple initial width sub-image classification models can be deployed on different graphics processors, and one initial width sub-image classification model can be deployed on each graphics processor, and each initial width sub-image classification model corresponds to a sub-dataset.

[0069] In practical applications, each initial width sub-image classification model deployed on a different graphics processor can be used to analyze its corresponding sub-image data set to obtain a classification probability prediction vector; based on each classification probability prediction vector and the actual category label vector corresponding to each sub-image data set, the model parameters of each initial width sub-image classification model are updated to obtain a trained classification model for each width sub-image.

[0070] For ease of description, in this embodiment of the present invention, the initial width sub-image classification model may be referred to as a sub-model. Different sub-models can be deployed on different GPUs or other heterogeneous devices, without limitation here, as long as the sub-models can be trained independently and in parallel.

[0071] The forward propagation calculation of the sub-model is the model inference operation from the input image to the output classification probability prediction value; the back-propagation calculation of the model includes the classification loss calculation and the loss function optimization solution. The classification loss function measures the deviation between the classification prediction value and the label value. The loss type can be selected from cross entropy or mean squared error. The optimization solution process of the loss function uses the gradient descent method to back-propagate from the output to the input layer by layer to calculate the gradient and update the model parameters. The model parameters can include the first model parameter and the second model parameter. For ease of description, w can be used to represent the first model parameter and b to represent the second model parameter. For each width sub-image classification model, each layer has a corresponding model parameter.

[0072] Each initial width sub-image classification model is trained in a similar manner. Taking any one of the initial width sub-image classification models, namely the first initial width sub-image classification model, as an example, the graphics processor can determine a first loss function based on the classification probability prediction vector and the actual category label vector corresponding to the first initial width sub-image classification model. A first gradient of the first loss function is determined using a gradient descent method, and the model parameters of the first initial width sub-image classification model are updated using the first gradient to obtain a trained first width sub-image classification model.

[0073] After completing the training of each sub-model, each sub-model has learned different features for each sub-image dataset, but there is no connection between the sub-models. It is necessary to use a merging strategy to restore the front and back node connections between different sub-models, and then merge them together to generate the original model structure.

[0074] The merging strategy adopted in the embodiment of the present invention is to merge the nodes of different sub-models based on the similarity of their feature images. The specific merging strategy is: find the node that is most similar to the nodes of each sub-model and the nodes of other sub-models, and then perform weighted summation on all nodes to generate a new node, which is called a merged node. All merged nodes form a new network layer, which is called a merged layer. The first layer of all sub-models plus all new merged layers are combined to obtain the original model structure. The method of merging sub-models based on the similarity of feature images can be found in Figure 4 The introduction of , will not be repeated here.

[0075] In an embodiment of the present invention, a parallel training strategy based on model splitting and merging is adopted to initialize model parameters, which can initially achieve a low-precision model parameter initialization, sacrificing accuracy to obtain improved training efficiency. First, the model training task is split into several independent sub-model training tasks to ensure that the training tasks of each sub-model are completely parallel. Then, a merging strategy based on the similarity between feature images is adopted to merge the several sub-models to restore the complete image classification model. In order to facilitate the distinction from the image classification model obtained in the subsequent training stage, the merged image classification model can be referred to as the image classification model after width merging.

[0076] The image classification model after width merging is a low-precision model. To ensure the recognition accuracy of the image classification model, the image classification model after width merging can be deployed on different graphics processors in a parallel training manner to achieve parallel training of multiple image classification models with the same width merging.

[0077] Current parallel training methods include data parallelism and model parallelism.

[0078] Data parallelism is to divide the samples of each batch data set into several sub-batch data sets on average, and also copy the same model to each computing device. The sub-batch data sets are input to different computing devices for forward propagation and backpropagation calculations respectively. Finally, the gradients calculated by each computing device are averaged, and the gradient mean is used to update the model parameters.

[0079] Model parallelism divides the model's computational process from front to back into several subprocesses. Each subprocess is assigned to a different computing device, with sample data passing through the forward and backward propagation of the subprocess on each device. Model parallelism achieves pipeline parallelism by overlapping the computation of different subprocesses for different sample data. Model parallelism creates dependencies between nodes and makes control more complex. Data parallelism, on the other hand, simplifies node control.

[0080] In an embodiment of the present invention, for the training of the image classification model after width merging, the model parameters can be trained using a data parallel method with relatively simple control, provided that the memory of the computing device meets the memory occupancy of the model.

[0081] In practical applications, the central processing unit (CPU) can select a batch dataset from the image dataset based on the principle of class balance; split the batch dataset into multiple sub-batches; and transmit each sub-batch dataset to its corresponding graphics processing unit (GPU). The wide-merged image classification model on each GPU analyzes its corresponding sub-batch dataset to obtain a merged classification probability prediction vector. Based on each merged classification probability prediction vector and the actual class label vector corresponding to each sub-batch dataset, each GPU updates the model parameters of the wide-merged image classification model to obtain the final trained image classification model.

[0082] Common AI (Artificial Intelligence) algorithm development frameworks such as TensorFlow (an end-to-end open source machine learning platform) and Pytorch (a deep learning framework library) both provide a variety of distributed parallel computing interfaces. For example, the distributed data parallel (Pytorch DDP) interface enables batch data set partitioning, data parallel computing on various graphics processors, and data communication.

[0083] When splitting a batch dataset into multiple sub-batch datasets, the batch dataset may be preprocessed first, wherein the preprocessing may include brightness normalization, noise filtering, scale normalization and / or cropping. The preprocessed batch dataset is split into multiple sub-batch datasets. For example, each batch dataset D batch Split into s sub-batch datasets

[0084] Each graphics processor performs training calculations on a portion of the batch dataset, and the training of each sub-batch dataset is parallel to each other. The training process includes forward propagation and back-propagation calculations, and the forward propagation and back-propagation calculations are the same as the model parameter initialization training. Each graphics processor can determine the loss function corresponding to the image classification model after each width is merged based on the merged classification probability prediction vector corresponding to the image classification model after each width is merged and the actual category label vector of the corresponding sub-batch dataset. Each graphics processor determines the gradient of each loss function based on the gradient descent method and transmits each gradient to the central processor, so that the central processor can perform an average operation on all gradients to obtain the gradient mean, and then send the gradient mean to each graphics processor.

[0085] Each graphics processor uses the gradient mean to update the model parameters of the width-merged image classification model deployed by it to obtain the final trained image classification model.

[0086] In practice, different sub-batch datasets are fed into the image classification model after each width is merged for forward propagation, performing model inference from input to output. Backward propagation then involves calculating the classification loss and optimizing the loss function, with gradients obtained during the loss function optimization process. Finally, the gradients calculated on each GPU are synchronized and transmitted to the master GPU, which calculates the mean of all gradients and updates the model parameters. The updated model parameters are then synchronized to each GPU. Similarly, distributed data-parallel training is performed on all batch datasets until all training iterations are completed.

[0087] The trained image classification model can be deployed on a graphics processor. After obtaining the image to be identified, the graphics processor can input the image to be identified into the trained image classification model and output the classification probability prediction value of each image category of the image to be identified. The image category corresponding to the highest classification probability prediction value is the image category to which the image to be identified belongs.

[0088] As can be seen from the above technical solution, each graphics processor independently and in parallel trains its matching initial width sub-image classification model using the sub-image dataset sent by the central processor to obtain trained width sub-image classification models. The initial width sub-image classification model is obtained by the central processor splitting the initial image classification model according to the width dimension. Based on the merged dataset sent by the central processor, each graphics processor analyzes the similarity between the feature images output by the nodes of each width sub-image classification model at each layer of the merged dataset to determine the nodes with the greatest similarity in each layer of each width sub-image classification model and the nodes with the greatest similarity in the corresponding layer of each other width sub-image classification model. The central processor adds a merging weight parameter between the nodes in each layer of each width sub-image classification model and the nodes with the greatest similarity in the corresponding layer of each other width sub-image classification model; the merging weight parameter is transmitted to each graphics processor. Each graphics processor analyzes the feature images corresponding to the nodes in each layer of each width sub-image classification model and the nodes with the greatest similarity in the corresponding layer of each other width sub-image classification model according to the merging weight parameter, layer by layer, to obtain the width-merged image classification model. Each graphics processor trains its own deployed width-merged image classification model in parallel. The trained image classification model is used to analyze the acquired image to be identified to determine the image category to which the image belongs. The present invention has the beneficial effect of employing a staged model training approach. In the first stage, the initial image classification model is split into multiple width-sub-image classification models, enabling parallel training of multiple width-sub-image classification models. The training process of each width-sub-image classification model does not require data exchange, significantly reducing the communication overhead incurred during the training process. After each width-sub-image classification model is trained, a merging weight parameter is added based on the node with the greatest similarity corresponding to each node. Based on the merging weight parameter, the feature images between the nodes with the greatest similarity are analyzed layer by layer, merging the multiple width-sub-image classification models to obtain a width-merged image classification model. In the second stage, the same width-merged image classification model is deployed on different graphics processors using a data-parallel approach for parallel training. This second stage of training significantly improves the accuracy of the image classification model. The coordinated training of these two stages achieves a balance between training accuracy and efficiency. By using the trained image classification model to analyze the acquired image to be identified, the image category to which the image to be identified belongs can be accurately determined. While ensuring the accuracy of image category recognition, the communication overhead of image classification is reduced.

[0089] Figure 4 A flowchart of a method for merging classification models of sub-images of different widths provided in an embodiment of the present invention, the method comprising:

[0090] S401: The central processing unit selects a merged data set from the image data set according to a set ratio.

[0091] The width sub-image classification model and process require collaborative training between each width sub-image classification model, which involves communication interaction and reduces efficiency. To this end, a small amount of sample data can be selected from the image dataset D to participate in the calculation, and a certain proportion of samples can be randomly selected from all samples of each category to form a dataset D. comb , which is called the merged dataset.

[0092] In practical applications, the central processing unit can determine the proportion of the merged data set in the image data set based on the efficiency and accuracy requirements of model training; select image data from the data set corresponding to each image category of the image data set according to the proportion; and merge the image data selected from each image category as the merged data set.

[0093] When focusing on model training efficiency, the ratio can be set smaller; when focusing on model training accuracy, the ratio can be set larger. In practical applications, the specific value of the ratio can be determined based on the balance between efficiency and accuracy.

[0094] Assuming the ratio is set to 10%, 10% of the image data in the image dataset can be selected as the merged dataset. To ensure the balance of image categories, the same number of images can be selected for each image category. Assuming that the merged dataset includes a total of 10,000 images and there are 100 image categories, 10,000 / 100 = 100 images can be selected from each image category.

[0095] S402: Each graphics processor inputs the same training sample in the merged data set into each width sub-image classification model to obtain a feature image output by a node of each layer in each width sub-image classification model.

[0096] During the model training process, image data from a single training session is often selected from the merged dataset as training samples, and the merged weight parameters are updated through continuous iterative training.

[0097] S403: Each graphics processor determines the nodes in each layer of each width sub-image classification model that have the greatest similarity with the nodes in the corresponding layers of other width sub-image classification models based on the similarity between the feature images output by the nodes in each layer of the width sub-image classification model it deploys and the feature images output by the nodes in each layer of other width sub-image classification models.

[0098] In the embodiments of the present application, the process of finding the node with high similarity in other width sub-image classification models based on the similarity of the feature images output by the nodes of different width sub-image classification models can be referred to as node routing. The process of generating a new node by weighting and merging the nodes of different width sub-image classification models based on different merging weight parameters can be referred to as node weighting. The merging weight parameter is a learnable parameter, and each node in each layer of the width sub-image classification model has a corresponding merging weight parameter. In the initial state, the merging weight parameter can be set to 1.

[0099] Taking the lth layer as an example, C l represents the front layer network layer before merging, C l+1 is the back layer network layer before merging, is the back layer network layer after merging, by performing feature similarity analysis on the nodes on C l+1 , the node with the maximum feature similarity in each width sub-image classification model is obtained, all nodes are routed to a new node, and each node is weighted and summed, so that the new node is obtained, and the network layer after merging is

[0100] In the embodiments of the present application, each graphics processor can construct a corresponding similarity matrix of each layer based on the cosine similarity between the feature images output by the nodes of each layer in the width sub-image classification model deployed by the graphics processor and the feature images output by the nodes of each layer in other width sub-image classification models. The similarity matrix is a three-dimensional matrix, the first dimension is the total number of nodes, the second dimension is the total number of all width sub-image classification models minus one, and the third dimension is the number of nodes contained in a single width sub-image classification model. According to the subscript corresponding to the maximum similarity on the third dimension of the corresponding similarity matrix of each layer, the node of each layer in each width sub-image classification model and the node with the maximum similarity in the corresponding layer of other width sub-image classification models are determined.

[0101] For the process of node routing, first, the same sample in the merging data set D comb can be input into each width sub-image classification model, and the feature images output by each layer are calculated through forward propagation. Taking the lth layer of the width sub-image classification model subM i as an example, the feature images output by the lth layer are subjected to convolutional transformation with the weight w l+1 of the l+1th layer to obtain the feature images of the l+1th layer:

[0102]

[0103] wherein, represents the feature images of the l+1th layer of the i th width sub-image classification model, represents the first model parameter of the l+1th layer of the i-th width sub-image classification model, represents the second model parameter of the l+1th layer of the i-th width sub-image classification model, Represents the feature image of the lth layer of the i-th width sub-image classification model.

[0104] The feature similarity calculation is performed on the feature images corresponding to each node in the l+1 layer one by one. Taking the first node of the l-th width sub-image classification model as an example, its corresponding feature image is Calculate the cosine similarity of all feature images with each other width sub-image classification model respectively. Taking the jth node of the second sub-model as an example, the cosine similarity calculation formula can be called to analyze the feature images output by the nodes of each layer in each width sub-image classification model of each frame image in the merged data set with the feature images output by the nodes of each layer in other width sub-image classification models to obtain the cosine similarity of the feature images output by the nodes of each layer in each width sub-image classification model with the feature images output by the nodes of each layer in other width sub-image classification models. The cosine similarity calculation formula is:

[0105]

[0106] in, represents the cosine similarity between the feature image output by node 1 of the l+1th layer of the first width sub-image classification model and the feature image output by node j of the l+1th layer of the second width sub-image classification model, Represents the feature image output by node 1 of the l+1th layer of the 1st width sub-image classification model, Represents the feature image output by node j of the l+1th layer of the second width sub-image classification model.

[0107] Considering that each frame image in the merged dataset is input into each width sub-image classification model respectively, a corresponding cosine similarity matrix will be obtained. Therefore, in practical applications, the cosine similarity matrices corresponding to each frame image in the merged dataset in the target layer can be accumulated and averaged to obtain the final similarity matrix corresponding to the target layer; wherein the target layer is any layer among all the layers in the width sub-image classification model.

[0108] For the convenience of description, the cosine similarity between the feature images of node 1 on width sub-image classification model 1 and all nodes on width sub-image classification model 2 can be calculated as Similarly, the feature similarity between each node of each width sub-image classification model of the l+1 layer and all nodes of other width sub-image classification models is calculated to obtain a similarity matrix simlMatl+ 1 .

[0109] simlMat l+1 is a three-dimensional matrix. The dimensions are [s l+1 × sn l+1 , s l+1 -1, sn l+1 ]. The first dimension represents the total number of nodes of all width sub-image classification models on the l+1 layer, i.e., the total number of nodes, the second dimension represents the total number of all width sub-image classification models minus one, i.e., the number of other width sub-image classification models, and the third dimension represents the number of nodes of each width sub-image classification model. The node feature similarity matrix between each width sub-image classification model on each layer is calculated layer by layer until all samples in D comb complete node feature similarity calculation, and the feature node similarity matrix of all images is accumulated.

[0110] Then, the maximum node similarity between the sub-classification models of each layer is calculated. The feature similarity matrix simlMat l+1 between each node of the width sub-image classification models of the l+1 layer and all nodes of the width sub-image classification models of other layers is taken as an example. The maximum value of the third dimension of the matrix simlMat l+1 is calculated, and the index corresponding to the maximum value is obtained, which is denoted as simlMaxId l+1 .

[0111] simlMaxId l+1 is a two-dimensional matrix, and the dimensions are [s l+1 × sn l+1 , s l+1 -1]. The matrix element represents the index of the most similar node of a certain node and other width sub-image classification models. Similarly, the most similar node of each node on each layer and other width sub-image classification models is calculated layer by layer until the last layer (output layer) of the model. The output layer does not need to perform node routing because the merging of the output layer is to directly accumulate the output of each width sub-image classification model to complete node routing. In addition, unlike the convolution layer, the node feature similarity calculation method of the fully connected layer is the mean square error distance, and the feature similarity of the node with the smallest mean square error distance on the fully connected layer is the largest. Thus, the node routing is completed.

[0112] S404: The graphics processor performs weighted summation on the feature images corresponding to the nodes with the largest similarity in the corresponding layers of the other width sub-image classification models, according to the set merging weight parameters, to obtain the classification probability prediction vector corresponding to the new nodes after merging of all nodes in the last layer.

[0113] The GPU can combine each node in each layer of the wide sub-image classification model and all the most similar nodes in other wide sub-image classification models, performing a weighted summation based on a merging weight parameter to generate a new node. The merging weight parameter is learned through training and all initial values ​​are 1.0. All model parameters w and b of the wide sub-image classification model are fixed, and the merging weight parameter is learned through classification loss optimization. For ease of description, the merging weight parameter can be represented by α.

[0114] For the update process of the merge weight parameters, first, merge the dataset D comb The same sample is input into each width sub-image classification model, and the forward propagation calculates the feature image output by each layer. Taking the lth layer as an example, the weight of the feature image output by the lth layer after the l+1th layer is w l+1 The convolution transformation obtains the feature image of the l+1 layer, and each feature image corresponds to a node. The feature images output by the i-th width sub-image classification model of the l layer and the l+1 layer are respectively recorded as The feature image output by the jth node of the i-th width sub-image classification model of the l+1th layer is as follows:

[0115]

[0116] in, represents the feature image output by the jth node of the i-th width sub-image classification model of the l+1th layer, represents the first model parameter of the jth node of the i-th width sub-image classification model of the l+1th layer, represents the feature image output by the jth node of the i-th width sub-image classification model of the l-th layer, Represents the second model parameter of the j-th node of the i-th width sub-image classification model of the l+1-th layer.

[0117] Then, the most similar feature images of each width sub-image classification model are weighted summed, and the matrix simlMaxId obtained from the node routing in the previous step is obtained. l+1 , taking the jth node of the i-th sub-model of the l+1th layer as an example, find the feature image output by the node in the k-th width sub-image classification model that is most similar to its feature and record it as Perform weighted summation on all feature images.

[0118] In an embodiment of the present invention, the feature merging formula can be called in a layer-by-layer calculation manner to analyze the feature images corresponding to the nodes of the target layer in each width sub-image classification model and the nodes with the greatest similarity in the target layer in other width sub-image classification models, so as to obtain the feature images corresponding to the new nodes after all the nodes in the target layer are merged; wherein, the feature merging formula is:

[0119]

[0120] in, The feature image corresponding to the new node is the result of merging the j-th node of the i-th width sub-image classification model in the l+1-th layer with the node that has the greatest similarity with the j-th node of the i-th width sub-image classification model in other width sub-image classification models. represents the merge weight parameter corresponding to the jth node of the i-th width sub-image classification model of the l+1th layer, represents the feature image output by the jth node of the i-th width sub-image classification model of the l+1th layer, Represents the feature image output by the node with the greatest similarity between the k-th width sub-image classification model of the l+1-th layer and the j-th node of the i-th width sub-image classification model, represents the merge weight parameter corresponding to the node with the greatest similarity between the k-th width sub-image classification model of the l+1th layer and the j-th node of the i-th width sub-image classification model, s l+1 Represents the total number of sub-image classification models of all widths.

[0121] S405: The graphics processor accumulates the classification probability prediction vectors corresponding to the new nodes formed by merging all nodes in the last layer of the classification models of the sub-images of each width, to obtain a merged classification probability prediction vector.

[0122] According to the implementation method of S404, the feature image output by the new node after merging all nodes on each layer of each width sub-image classification model is calculated layer by layer until the last layer, i.e., the output layer, is reached. When the target layer is the last layer, the feature image output by the new node after merging all nodes in the target layer is the classification probability prediction vector.

[0123] The classification probability prediction vectors corresponding to the new nodes after all nodes in the last layer are merged are accumulated to obtain the merged classification probability prediction vector. At this point, the forward propagation calculation of the model merge weighted training is completed.

[0124] S406: The graphics processor updates the merging weight parameter based on the merged classification probability prediction vector and the actual category label vector corresponding to the merged data set.

[0125] Calculate the classification loss Loss and use the gradient descent optimization method for back propagation calculation. Different from the previous width sub-image classification model training, all model parameters of the width sub-image classification model are fixed here, and the gradient of the merged weight parameter is calculated and updated. Repeat the above steps S402 to S406 until the merged dataset D comb Complete the merge weight training.

[0126] S407: Using the updated merging weight parameters, the model parameters of all width sub-image classification models are updated to obtain a width-merged image classification model.

[0127] After completing the update of the node merging weight parameters, in order to restore the structure of the original image classification model, it is necessary to remove the node routing process between the front and back layers and restore the node connections between sub-image classification models of different widths. That is, the nodes with the most similar features between the sub-image classification models of each width are directly connected to the merging node, and the model parameters and merging weight parameters of each width sub-image classification model are merged together as the model parameters of the new node.

[0128] Taking the first neuron node of the i-th sub-model of the l+1th layer as an example, the first parameter update formula can be called to analyze the updated merge weight parameters and the first model parameters of all width sub-image classification models to obtain the updated first model parameters; wherein the first parameter update formula is:

[0129]

[0130] in, represents the first model parameter after updating the first node of the i-th width sub-image classification model of the l+1-th layer, represents the first model parameter of the first node of the i-th width sub-image classification model of the l+1-th layer before updating, represents the updated merge weight parameter corresponding to the first node of the i-th width sub-image classification model of the l+1-th layer, represents the sth layer of the l+1th layer l+1 The first model parameter of the first node of the width sub-image classification model before updating, represents the sth layer of the l+1th layer l+1 The updated merge weight parameter corresponding to the first node of the width sub-image classification model, i={1,2,…,s l+1};

[0131] The second parameter update formula is called to analyze the updated merging weight parameter and the second model parameters of all width sub-image classification models to obtain the updated second model parameters; wherein the second parameter update formula is:

[0132]

[0133] in, represents the updated second model parameter of the first node of the i-th width sub-image classification model of the l+1-th layer, represents the second model parameter of the first node of the k-th width sub-image classification model of the l+1-th layer before updating, represents the updated merge weight parameter corresponding to the first node of the k-th width sub-image classification model of the l+1th layer, i={1,2,…,s l+1};

[0134] All width sub-image classification models after updating the first model parameters and the second model parameters are merged as a width-merged image classification model.

[0135] Represents the first model parameter corresponding to the first node of the first width sub-image classification model of the l+1th layer, Represents the second model parameter corresponding to the first node of the first width sub-image classification model of the l+1th layer, Contains sn l+1 convolution kernel parameters and 1 bias parameter, so the weight of the new node The model parameters of each width sub-image classification model are weighted and combined using the merging weight parameters. Similarly, the model is merged layer by layer and node by node, generating the weight parameters of each merged node until the last layer of the model. At this point, the merging of the width sub-image classification models is completed, and the original model is generated.

[0136] Figure 5 A schematic diagram of merging all nodes of each layer of a multiple width sub-image classification model provided by an embodiment of the present invention is provided. Figure 5 The following is an example of N width sub-image classification model, which are subM1, subM2, ..., subM N Taking the lth layer of each width sub-image classification model as an example, C l represents the previous network layer before merging, C l+1 is the back layer before merging, Represents the network layer after merging. Each layer contains multiple nodes. Figure 5 The lines between different nodes in C represent the connection relationship. l and C l+1 They all belong to the front network layer before merging, so C l and C l+1 Only nodes of the sub-image classification model with the same width have a connection relationship. It belongs to the back-layer network layer after the merger. The back-layer network layer after the merger takes into account the connection relationship between the nodes of the sub-image classification models of different widths, so there will be connections between the nodes of the sub-image classification models of different widths.

[0137] Figure 5 The middle is C l+1 Merge the midpoints to get For example, by analogy, the corresponding back-layer network layer of each front-layer network layer can be determined layer by layer. By directly replacing the corresponding front-layer network layer with the back-layer network layer, the merged image classification model can be obtained.

[0138] Figure 6 A schematic diagram of a width-merged image classification model provided by an embodiment of the present invention, taking the first layer of each width sub-image classification model as an example, C l Represents the previous network layer before merging, Represents the network layer after merging. After completing the update of the node's merged weight parameters, the model parameters and the updated merged weight parameters can be directly merged together as the model parameters of the new node. Figure 6 The first node of the i-th width sub-image classification model of the l+1 layer is taken as an example. The first model parameter of the new node is

[0139] In an embodiment of the present invention, a phased model training approach is employed. Taking into account the varying precision requirements of different training stages, different distributed parallel methods are employed for each training stage to achieve a balance between training accuracy and efficiency. For the first stage, parameter initialization, a parallel training method based on a model splitting and merging strategy is proposed. This splits the model training task into completely independent, parallel sub-training tasks, which are then merged and restored using a feature similarity-based merging strategy to yield an image classification model, sacrificing parameter accuracy for high computational efficiency.

[0140] Figure 7 A schematic diagram of the structure of an image classification device provided by an embodiment of the present invention, wherein the device includes a graphics processor and a central processing unit; the graphics processor includes a parallel training unit 71, a node determination unit 72, an analysis unit 75, and a recognition unit 76; the central processing unit includes an adding unit 73 and a transmission unit 74;

[0141] The parallel training unit 71 is configured to independently and parallelly train the initial width sub-image classification model matched thereto using the sub-image dataset sent by the central processing unit to obtain trained classification models for each width sub-image; the initial width sub-image classification model is obtained by the central processing unit splitting the initial image classification model according to the width dimension;

[0142] A node determination unit 72 is configured to analyze, based on the merged data set delivered by the central processing unit, the similarities between the feature images output by the nodes of each layer of the merged data set by each width sub-image classification model, so as to determine the node of each layer in each width sub-image classification model that has the greatest similarity to the node in the corresponding layer of each other width sub-image classification model;

[0143] An adding unit 73 is used to add a merging weight parameter between the nodes of each layer in each width sub-image classification model and the nodes with the greatest similarity in the corresponding layers of other width sub-image classification models;

[0144] a transmission unit 74, configured to transmit the combined weight parameters to each graphics processor;

[0145] An analysis unit 75 is configured for each graphics processor to analyze, layer by layer, the feature images corresponding to the nodes in each layer of each width sub-image classification model and the nodes in the corresponding layers of other width sub-image classification models with the greatest similarity according to the merging weight parameter, to obtain a width-merged image classification model;

[0146] The recognition unit 76 is used for each graphics processor to perform parallel training on the width-merged image classification model deployed by each graphics processor, and use the trained image classification model to analyze the acquired image to be recognized to determine the image category to which the image to be recognized belongs.

[0147] In some embodiments, the central processing unit further includes a split unit;

[0148] A splitting unit is used to split the image data set into multiple sub-image data sets; split the initial image classification model into multiple initial sub-image classification models with the same number as the sub-image data sets according to the width dimension;

[0149] A transmission unit, configured to transmit each sub-image data set and its matching initial width sub-image classification model to a corresponding graphics processor;

[0150] The parallel training unit is used to use each initial width sub-image classification model to analyze its corresponding sub-image data set to obtain a classification probability prediction vector; based on each classification probability prediction vector and the actual category label vector corresponding to each sub-image data set, the model parameters of each initial width sub-image classification model are updated to obtain a trained classification model for each width sub-image.

[0151] In some embodiments, the parallel training unit includes a loss function determination subunit and a model parameter update subunit;

[0152] a loss function determination subunit, configured to determine a first loss function based on a classification probability prediction vector and an actual category label vector corresponding to a first initial width sub-image classification model; wherein the first initial width sub-image classification model is any one of all initial width sub-image classification models;

[0153] The model parameter updating subunit is used to determine the first gradient of the first loss function according to the gradient descent method, and use the first gradient to update the model parameters of the first initial width sub-image classification model to obtain a trained first width sub-image classification model.

[0154] In some embodiments, the central processing unit further includes a selection unit;

[0155] A selection unit is used to select a merged data set from the image data set according to a set ratio; and transmit the merged data set to each graphics processor;

[0156] The node determination unit is used to input the same training sample in the merged data set into each width sub-image classification model to obtain the feature image output by the node of each layer in each width sub-image classification model; based on the similarity between the feature image output by the node of each layer in the deployed width sub-image classification model and the feature image output by the node of each layer in other width sub-image classification models, determine the node of each layer in each width sub-image classification model with the greatest similarity to the node in the corresponding layer in other width sub-image classification models.

[0157] In some embodiments, the analyzing unit includes a weighted summing subunit, an accumulating subunit, a first updating subunit, and a second updating subunit;

[0158] The weighted summation subunit is used to perform weighted summation of the feature images corresponding to the nodes of each layer in the deployed width sub-image classification model and the nodes with the greatest similarity in the corresponding layers of other width sub-image classification models according to the merging weight parameter, so as to obtain the classification probability prediction vector corresponding to the new node after all nodes in the last layer are merged;

[0159] An accumulation subunit, configured to accumulate the classification probability prediction vectors corresponding to the new nodes after merging all nodes in the last layer of the classification model of each width sub-image, to obtain a merged classification probability prediction vector;

[0160] A first updating subunit, configured to update a merging weight parameter based on the merged classification probability prediction vector and the actual category label vector corresponding to the merged data set;

[0161] The second updating subunit is used to update the model parameters of all sub-image classification models using the updated merging weight parameters to obtain a width-merged image classification model.

[0162] In some embodiments, the selection unit includes a ratio determination subunit, an image data selection subunit, and a merging subunit;

[0163] A ratio determination subunit is used to determine the ratio of the merged dataset to the image dataset based on the efficiency and accuracy requirements of the model training;

[0164] An image data selection subunit, configured to select image data from a data set corresponding to each image category in the image data set according to a proportion;

[0165] The merging subunit is used to merge the image data selected from each image category into a merged data set.

[0166] In some embodiments, the node determination unit is used to construct a similarity matrix corresponding to each layer based on the cosine similarity between the feature images output by the nodes of each layer in the width sub-image classification model deployed by it and the feature images output by the nodes of each layer in other width sub-image classification models; wherein the similarity matrix is ​​a three-dimensional matrix, the first dimension is the total number of nodes, the second dimension is the total number of all width sub-image classification models minus one, and the third dimension is the number of nodes included in a single width sub-image classification model; according to the subscript corresponding to the maximum similarity value on the third dimension of the similarity matrix corresponding to each layer, the node with the greatest similarity between each layer in each width sub-image classification model and the node in the corresponding layer in other width sub-image classification models is determined.

[0167] In some embodiments, the node determination unit includes a first calculation unit and a second calculation unit;

[0168] The first calculation unit is used to call the cosine similarity calculation formula to analyze the feature images output by the nodes of each layer in each width sub-image classification model of each frame image in the merged data set with the feature images output by the nodes of each layer in other width sub-image classification models, so as to obtain the cosine similarity of the feature images output by the nodes of each layer in each width sub-image classification model with the feature images output by the nodes of each layer in other width sub-image classification models; the cosine similarity calculation formula is:

[0169]

[0170] in, represents the cosine similarity between the feature image output by node 1 of the l+1th layer of the first width sub-image classification model and the feature image output by node j of the l+1th layer of the second width sub-image classification model, Represents the feature image output by node 1 of the l+1th layer of the 1st width sub-image classification model, a feature image output by a jth node of an (l+1)th layer of an ith width sub-image classification model;

[0171] a second calculation unit configured to accumulate and average the cosine similarity matrices corresponding to the target layer of each frame image in the merged data set to obtain a final similarity matrix corresponding to the target layer; wherein the target layer is any one of the layers in the width sub-image classification model.

[0172] In some embodiments, the weighted summation sub-unit is configured to call the feature merging formula in a layer-by-layer calculation manner, analyze the feature images corresponding to the nodes of the target layer in each width sub-image classification model and the nodes with the highest similarity in the target layer in each other width sub-image classification model to obtain a feature image corresponding to the new node after merging all the nodes of the target layer; wherein the feature merging formula is:

[0173]

[0174] wherein, a feature image corresponding to a new node after merging the jth node of the ith width sub-image classification model of the (l+1)th layer and the nodes with the highest similarity in each other width sub-image classification model to the jth node of the ith width sub-image classification model, a merging weight parameter corresponding to the jth node of the ith width sub-image classification model of the (l+1)th layer, a feature image output by the jth node of the ith width sub-image classification model of the (l+1)th layer, a feature image output by the node with the highest similarity in the kth width sub-image classification model of the (l+1)th layer to the jth node of the ith width sub-image classification model, a merging weight parameter corresponding to the node with the highest similarity in the kth width sub-image classification model of the (l+1)th layer to the jth node of the ith width sub-image classification model, l+1 a total number of all the width sub-image classification models;

[0175] In the case where the target layer is the last layer, the feature image output by the new node after merging all the nodes of the target layer is a classification probability prediction vector.

[0176] In some embodiments, the second updating sub-unit is configured to call the first parameter updating formula, analyze the updated merging weight parameters and the first model parameters of all the width sub-image classification models to obtain updated first model parameters; wherein the first parameter updating formula is:

[0177]

[0178] wherein, represents the first model parameter after updating the first node of the i-th width sub-image classification model of the l+1-th layer, represents the first model parameter of the first node of the i-th width sub-image classification model of the l+1-th layer before updating, represents the updated merge weight parameter corresponding to the first node of the i-th width sub-image classification model of the l+1-th layer, represents the sth layer of the l+1th layer l+1 The first model parameter of the first node of the width sub-image classification model before updating, represents the sth layer of the l+1th layer l+1 The updated merge weight parameter corresponding to the first node of the width sub-image classification model;

[0179] The second parameter update formula is called to analyze the updated merging weight parameter and the second model parameters of all width sub-image classification models to obtain the updated second model parameters; wherein the second parameter update formula is:

[0180]

[0181] in, represents the updated second model parameter of the first node of the i-th width sub-image classification model of the l+1-th layer, represents the second model parameter of the first node of the k-th width sub-image classification model of the l+1-th layer before updating, Represents the updated merge weight parameter corresponding to the first node of the k-th width sub-image classification model of the l+1-th layer;

[0182] All width sub-image classification models after updating the first model parameters and the second model parameters are merged as a width-merged image classification model.

[0183] In some embodiments, the identification unit includes a receiving subunit, a predicting subunit, and a third updating subunit;

[0184] The receiving sub-unit is used to receive the sub-batch data sets sent by the central processing unit; wherein the sub-batch data sets are the batch data sets selected by the central processing unit from the image data set according to the category balance principle; and the batch data sets are split into multiple sub-batch data sets;

[0185] The prediction subunit is used to analyze the corresponding sub-batch data set using the width-merged image classification model to obtain a merged classification probability prediction vector;

[0186] The third updating subunit is configured to update the model parameters of the width-merged image classification model based on the respective merged classification probability prediction vectors and the actual class label vectors corresponding to the respective sub-batch data sets, so as to obtain a final trained image classification model.

[0187] In some embodiments, the third updating subunit is configured to determine the loss function corresponding to each width-merged image classification model according to the merged classification probability prediction vector corresponding to each width-merged image classification model and the actual class label vector of the sub-batch data set corresponding to the merged classification probability prediction vector.

[0188] Each graphics processor determines the gradient of each loss function according to the gradient descent method, and transmits each gradient to the central processor, so that the central processor performs average value operation on all the gradients to obtain a gradient mean value, and the gradient mean value is issued to each graphics processor.

[0189] Each graphics processor updates the model parameters of the width-merged image classification model deployed by the graphics processor using the gradient mean value, so as to obtain a final trained image classification model.

[0190] In some embodiments, the splitting unit comprises an acquisition subunit, a preprocessing subunit and a splitting subunit.

[0191] The acquisition subunit is configured to acquire an initial image data set from a public data set according to the model size and accuracy requirement of the image classification model.

[0192] The preprocessing subunit is configured to pre-process the initial image data set to obtain an image data set.

[0193] The splitting subunit is configured to split the image data set into a plurality of sub-image data sets according to the training requirement.

[0194] In some embodiments, the splitting subunit is configured to determine the number of sub-image data sets according to the number of parallel units of the heterogeneous computing platform, the model width of the image classification model and the accuracy requirement; and split the image data set into sub-image data sets according to the number of sub-image data sets according to the principle of class balance.

[0195] In some embodiments, the splitting subunit is configured to determine a splitting upper limit value according to the number of parallel units of the heterogeneous computing platform and the model width of the image classification model; determine a splitting lower limit value according to the accuracy requirement of the image classification model; and select a value between the splitting lower limit value and the splitting upper limit value as the number of sub-image data sets.

[0196] In some embodiments, the splitting subunit is configured to take the mean value of the splitting lower limit value and the splitting upper limit value as the number of sub-image data sets.

[0197] In some embodiments, the splitting unit is used to split the initial image classification model into multiple initial width sub-image classification models with the same number as the sub-image data sets in a manner that keeps the model depth unchanged and divides the model width evenly; retain the connection relationship between the nodes in each initial width sub-image classification model, and delete the connection relationship between the nodes of different initial width sub-image classification models to obtain the final multiple initial width sub-image classification models.

[0198] Figure 7 The description of the features in the corresponding embodiment can be found in Figure 1 The relevant descriptions of the corresponding embodiments will not be repeated here one by one.

[0199] From the above technical solution can be seen, each graphics processor utilizes the central processor to send the initial width sub-image classification model matched with the sub-image data set for independent parallel training, to obtain the trained each width sub-image classification model; the initial width sub-image classification model is obtained by the central processor splitting the initial image classification model according to width dimension; each graphics processor analyzes the similarity between the feature images of each layer node output of each width sub-image classification model in the merging data set issued by the central processor, to determine the node of each layer in each width sub-image classification model and the node of the corresponding layer in other width sub-image classification models with the maximum similarity. The central processor adds merging weight parameters between the node of each layer in each width sub-image classification model and the node of the corresponding layer in other width sub-image classification models with the maximum similarity; the merging weight parameters are transmitted to each graphics processor. Each graphics processor analyzes the feature images corresponding to the node of each layer in each width sub-image classification model and the node of the corresponding layer in other width sub-image classification models with the maximum similarity layer by layer according to the merging weight parameters, to obtain the width-merged image classification model. Each graphics processor parallel trains the width-merged image classification model deployed by itself, and analyzes the acquired to-be-identified image by using the trained image classification model, to determine the image category to which the to-be-identified image belongs. The beneficial effects of the present application are that the model training method is adopted in stages, the initial image classification model is split into multiple width sub-image classification models in the first stage, parallel training of the multiple width sub-image classification models can be realized, the training process of each width sub-image classification model does not require data interaction, and the communication overhead generated in the training process is greatly reduced. After the width sub-image classification models are trained, the merging weight parameters are added based on the node corresponding to the node with the maximum similarity, the feature images between the nodes with the maximum similarity are analyzed layer by layer according to the merging weight parameters, the multiple width sub-image classification models are merged, and thus the width-merged image classification model is obtained. In the second stage, the data parallel method is adopted, and the same width-merged image classification model is deployed on different graphics processors for parallel training. Through the training in the second stage, the accuracy of the image classification model can be greatly improved. Through the training in the two stages, the balance between training accuracy and efficiency is realized. Thus, the acquired to-be-identified image is analyzed by using the trained image classification model, and the image category to which the to-be-identified image belongs can be accurately determined. While ensuring the image category recognition accuracy, the communication overhead of image classification is reduced.

[0200] Figure 8 A structure diagram of an image classification device provided for an embodiment of the present application is shown in FIG. 1. Figure 8 As shown in FIG. 1, the image classification device includes a memory 80 for storing a computer program.

[0201] The processor 81 is configured to implement the steps of the image classification method in the above embodiment when executing a computer program.

[0202] The image classification device provided in this embodiment may include but is not limited to a smartphone, a tablet computer, a laptop computer, or a desktop computer.

[0203] In this embodiment, the memory 80 is used to store at least the following computer program 801. When loaded and executed by the processor 81, the computer program can implement the relevant steps of the image classification method disclosed in any of the aforementioned embodiments. Furthermore, the resources stored in the memory 80 may also include an operating system 802 and data 803, which may be stored in a temporary or permanent manner. The operating system 802 may include Windows, Unix, Linux, etc. The data 803 may include, but is not limited to, image datasets and images to be identified.

[0204] In some embodiments, the image classification device may further include a display screen 82 , an input / output interface 83 , a communication interface 84 , a power supply 85 , and a communication bus 86 .

[0205] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the image classification device, and may include more or fewer components than shown in the figure.

[0206] It is understood that if the image classification method in the above-mentioned embodiment is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the current technology, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and performs all or part of the steps of the method of each embodiment of the present invention.

[0207] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned image classification method are implemented.

[0208] The above describes in detail an image classification method, apparatus, device, and computer-readable storage medium provided by embodiments of the present invention. The various embodiments are described in a progressive manner throughout this specification, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between the various embodiments can be referenced for clarity.

[0209] The units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the above description generally describes the components and steps of each example based on their functions. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0210] The above describes in detail the image classification method, apparatus, device, and computer-readable storage medium provided by the present invention. This document uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above examples is intended only to facilitate understanding of the method and core concepts of the present invention. Various improvements and modifications may be made to the present invention without departing from the principles of the present invention, and such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. An image classification method, characterized in that: Applicable to heterogeneous platforms, the method includes: Each graphics processor independently and in parallel trains its matched initial width sub-image classification model using the sub-image dataset sent by the central processor to obtain trained width sub-image classification models; the initial width sub-image classification model is obtained by the central processor splitting the initial image classification model according to the width dimension; Each of the graphics processors analyzes, based on the merged data set sent by the central processing unit, the similarities between the feature images output by the nodes of each layer of the width sub-image classification model under the merged data set, so as to determine the node of each layer in each width sub-image classification model that has the greatest similarity with the node in the corresponding layer of each other width sub-image classification model; The central processor adds a merging weight parameter between the node of each layer in each width sub-image classification model and the node with the greatest similarity in the corresponding layer in other width sub-image classification models; and transmits the merging weight parameter to each of the graphics processors; Each of the graphics processors analyzes, layer by layer, the feature images corresponding to the nodes of each layer in each width sub-image classification model and the nodes with the greatest similarity in the corresponding layers of other width sub-image classification models according to the merging weight parameters, to obtain a width-merged image classification model; Each of the graphics processors performs parallel training on the width-merged image classification models deployed therein, and uses the trained image classification models to analyze the acquired image to be identified to determine the image category to which the image to be identified belongs.

2. The image classification method according to claim 1, characterized in that Each graphics processor uses the sub-image data set sent by the central processing unit to independently and parallelly train its matched initial width sub-image classification model to obtain trained width sub-image classification models, including: The central processing unit splits the image data set into a plurality of sub-image data sets; splits the initial image classification model into a plurality of initial sub-image classification models according to the width dimension, which is the same as the number of the sub-image data sets; and transmits each of the sub-image data sets and the initial width sub-image classification model matched thereto to a corresponding graphics processing unit. Each of the initial width sub-image classification models on each of the graphics processors analyzes its corresponding sub-image data set to obtain a classification probability prediction vector; based on each of the classification probability prediction vectors and the actual category label vector corresponding to each of the sub-image data sets, the model parameters of each of the initial width sub-image classification models are updated to obtain a trained classification model for each of the width sub-images.

3. The image classification method according to claim 2, characterized in that The updating of model parameters of each of the initial width sub-image classification models based on each of the classification probability prediction vectors and the actual category label vectors corresponding to each of the sub-image data sets to obtain the trained width sub-image classification models includes: Determining a first loss function based on a classification probability prediction vector and an actual category label vector corresponding to a first initial width sub-image classification model; wherein the first initial width sub-image classification model is any one of all the initial width sub-image classification models; A first gradient of the first loss function is determined according to a gradient descent method, and the model parameters of the first initial width sub-image classification model are updated using the first gradient to obtain a trained first width sub-image classification model.

4. The image classification method according to claim 1, wherein: Each of the graphics processors analyzes, based on the merged data set sent by the central processing unit, the similarity between the feature images output by the nodes of each layer of the width sub-image classification model under the merged data set, so as to determine that the nodes of each layer in each width sub-image classification model have the greatest similarity with the nodes in the corresponding layers of other width sub-image classification models, including: The central processing unit selects a merged data set from the image data set according to a set ratio; and transmits the merged data set to each of the graphics processing units; Each of the graphics processors inputs the same training sample in the merged data set into each of the width sub-image classification models to obtain a feature image output by a node of each layer in each of the width sub-image classification models; Each of the graphics processors determines the nodes in each layer of each width sub-image classification model that have the greatest similarity with the nodes in the corresponding layers of other width sub-image classification models based on the similarity between the feature images output by the nodes in each layer of the width sub-image classification model deployed by it and the feature images output by the nodes in each layer of other width sub-image classification models.

5. The image classification method according to claim 4, characterized in that Each of the graphics processors analyzes, layer by layer, the feature images corresponding to the nodes of each layer in each width sub-image classification model and the nodes with the greatest similarity in the corresponding layers of other width sub-image classification models according to the merging weight parameters, to obtain a width-merged image classification model, including: According to the merging weight parameter, weighted summation is performed on the feature images corresponding to the nodes of each layer in the deployed width sub-image classification model and the nodes with the greatest similarity in the corresponding layers of other width sub-image classification models, so as to obtain the classification probability prediction vector corresponding to the new node after all nodes in the last layer are merged; Accumulating the classification probability prediction vectors corresponding to the new nodes after merging all the nodes in the last layer of each width sub-image classification model to obtain a merged classification probability prediction vector; Updating the merging weight parameter based on the merged classification probability prediction vector and the actual category label vector corresponding to the merged data set; The model parameters of all the sub-image classification models are updated using the updated merging weight parameters to obtain the width-merged image classification model.

6. The image classification method according to claim 4, characterized in that The central processing unit selects a merged data set from the image data set according to a set ratio, including: Determining the proportion of the combined dataset in the image dataset based on efficiency and accuracy requirements of model training; Selecting image data from a data set corresponding to each image category in the image data set according to the ratio; The image data selected from each image category are merged to form the merged data set.

7. The image classification method according to claim 4, characterized in that Each graphics processor determines, based on the similarity between the feature images output by the nodes of each layer in the deployed width sub-image classification model and the feature images output by the nodes of each layer in other width sub-image classification models, that the nodes of each layer in each width sub-image classification model have the greatest similarity with the nodes in the corresponding layers in other width sub-image classification models, including: Each graphics processor constructs a similarity matrix corresponding to each layer based on the cosine similarity between the feature images output by the nodes of each layer in the width sub-image classification model deployed therein and the feature images output by the nodes of each layer in other width sub-image classification models; wherein the similarity matrix is ​​a three-dimensional matrix, wherein the first dimension is the total number of nodes, the second dimension is the total number of all width sub-image classification models minus one, and the third dimension is the number of nodes included in a single width sub-image classification model; Each graphics processor determines the node with the greatest similarity in each layer of each width sub-image classification model and the node with the greatest similarity in the corresponding layer of other width sub-image classification models based on the subscript corresponding to the maximum similarity on the third dimension of the similarity matrix corresponding to each layer.

8. The image classification method according to claim 7, characterized in that: Each graphics processor constructs a similarity matrix corresponding to each layer based on the cosine similarity between the feature images output by the nodes of each layer in the width sub-image classification model deployed therein and the feature images output by the nodes of each layer in other width sub-image classification models, including: The cosine similarity calculation formula is called to analyze the feature images output by the nodes of each layer in each width sub-image classification model of each frame image in the merged data set with the feature images output by the nodes of each layer in other width sub-image classification models, so as to obtain the cosine similarity of the feature images output by the nodes of each layer in each width sub-image classification model with the feature images output by the nodes of each layer in other width sub-image classification models; the cosine similarity calculation formula is: in, represents the cosine similarity between the feature image output by node 1 of the l+1th layer of the first width sub-image classification model and the feature image output by node j of the l+1th layer of the second width sub-image classification model, Represents the feature image output by node 1 of the l+1th layer of the 1st width sub-image classification model, Represents the feature image output by node j of the l+1th layer of the second width sub-image classification model; The cosine similarity matrices corresponding to the target layer of each frame image in the merged data set are cumulatively added and averaged to obtain the similarity matrix finally corresponding to the target layer; wherein the target layer is any layer among all layers in the width sub-image classification model.

9. The image classification method according to claim 5, characterized in that According to the merging weight parameter, weighted summation is performed on the feature images corresponding to the nodes of each layer in the deployed width sub-image classification model and the nodes with the greatest similarity in the corresponding layers of other width sub-image classification models layer by layer to obtain the classification probability prediction vector corresponding to the new node after all nodes in the last layer are merged, including: The feature merging formula is called in a layer-by-layer calculation manner to analyze the feature images corresponding to the nodes of the target layer in each width sub-image classification model and the nodes with the greatest similarity in the target layer in other width sub-image classification models, so as to obtain the feature images corresponding to the new nodes after all nodes in the target layer are merged; wherein, the feature merging formula is: in, The feature image corresponding to the new node is the result of merging the j-th node of the i-th width sub-image classification model in the l+1-th layer with the node that has the greatest similarity with the j-th node of the i-th width sub-image classification model in other width sub-image classification models. represents the merge weight parameter corresponding to the jth node of the i-th width sub-image classification model of the l+1th layer, represents the feature image output by the jth node of the i-th width sub-image classification model of the l+1th layer, Represents the feature image output by the node with the greatest similarity between the k-th width sub-image classification model of the l+1-th layer and the j-th node of the i-th width sub-image classification model, represents the merge weight parameter corresponding to the node with the greatest similarity between the k-th width sub-image classification model of the l+1th layer and the j-th node of the i-th width sub-image classification model, s l+1 Represents the total number of sub-image classification models of all widths; In the case where the target layer is the last layer, the feature image output by the new node after all nodes of the target layer are merged is the classification probability prediction vector.

10. The image classification method according to claim 9, characterized in that: The updating of the model parameters of all the sub-image classification models using the updated merging weight parameters to obtain the width-merged image classification model includes: The first parameter update formula is called to analyze the updated merging weight parameter and the first model parameters of all the width sub-image classification models to obtain the updated first model parameters; wherein the first parameter update formula is: in, represents the first model parameter after updating the first node of the i-th width sub-image classification model of the l+1-th layer, represents the first model parameter of the first node of the i-th width sub-image classification model of the l+1-th layer before updating, represents the updated merge weight parameter corresponding to the first node of the i-th width sub-image classification model of the l+1-th layer, represents the sth layer of the l+1th layer l+1 The first model parameter of the first node of the width sub-image classification model before updating, represents the sth layer of the l+1th layer l+1 The updated merge weight parameter corresponding to the first node of the width sub-image classification model; The second parameter update formula is called to analyze the updated merging weight parameter and the second model parameters of all the width sub-image classification models to obtain the updated second model parameters; wherein the second parameter update formula is: in, represents the updated second model parameter of the first node of the i-th width sub-image classification model of the l+1-th layer, represents the second model parameter of the first node of the k-th width sub-image classification model of the l+1-th layer before updating, Represents the updated merge weight parameter corresponding to the first node of the k-th width sub-image classification model of the l+1-th layer; All width sub-image classification models after updating the first model parameters and the second model parameters are merged as the width-merged image classification model.

11. The image classification method according to claim 1, wherein: The parallel training of the width-merged image classification models deployed by the graphics processors includes: Each of the graphics processors receives a sub-batch data set sent by the central processor; wherein the sub-batch data set is a batch data set selected by the central processor from the image data set according to a class balance principle; and the batch data set is split into multiple sub-batch data sets; The width-merged image classification model on each graphics processor analyzes its corresponding sub-batch data set to obtain a merged classification probability prediction vector; Each graphics processor updates the model parameters of the width-merged image classification model based on each merged classification probability prediction vector and the actual category label vector corresponding to each sub-batch data set to obtain the finally trained image classification model.

12. The image classification method according to claim 11, characterized in that: Each of the graphics processors updates the model parameters of the width-merged image classification model based on each of the merged classification probability prediction vectors and the actual category label vectors corresponding to each of the sub-batch data sets to obtain the finally trained image classification model, including: Each of the graphics processors determines a loss function corresponding to each of the width-merged image classification models based on a merged classification probability prediction vector corresponding to each of the width-merged image classification models and an actual category label vector of a corresponding sub-batch data set; Each of the graphics processors determines the gradient of each of the loss functions according to a gradient descent method, and transmits each of the gradients to the central processing unit, so that the central processing unit performs an average operation on all of the gradients to obtain a gradient mean, and sends the gradient mean to each of the graphics processors; Each graphics processor uses the gradient mean to update the model parameters of the width-merged image classification model deployed by it to obtain the finally trained image classification model.

13. The image classification method according to claim 2, wherein: The central processing unit splits the image data set into a plurality of sub-image data sets, including: Obtaining an initial image dataset from a public dataset based on the model scale and accuracy requirements of the image classification model; Preprocessing the initial image dataset to obtain the image dataset; According to training requirements, the image dataset is split into multiple sub-image datasets with balanced categories.

14. The image classification method according to claim 13, wherein: The step of splitting the image dataset into a plurality of sub-image datasets with balanced categories according to training requirements includes: Determining the number of sub-image data sets according to the number of parallel units of the heterogeneous computing platform, the model width and accuracy requirements of the image classification model; According to the category balance principle, the image dataset is split into sub-image datasets whose number is the same as the number of the sub-image datasets.

15. The image classification method according to claim 14, characterized in that: Determining the number of sub-image data sets according to the number of parallel units of the heterogeneous computing platform, the model width and accuracy requirements of the image classification model includes: Determining a split upper limit value according to the number of parallel units of the heterogeneous computing platform and the model width of the image classification model; Determine a split lower limit value according to the accuracy requirements of the image classification model; A value between the split lower limit value and the split upper limit value is selected as the number of the sub-image data sets.

16. The image classification method according to claim 15, characterized in that: The selecting of a value between the split lower limit value and the split upper limit value as the number of the sub-image data sets includes: The average of the split lower limit value and the split upper limit value is used as the number of the sub-image data sets.

17. The image classification method according to claim 2, characterized in that: The step of splitting the initial image classification model into a plurality of initial sub-image classification models having the same number as the sub-image data sets according to the width dimension comprises: Splitting the initial image classification model into a number of sub-image classification models of the initial width equal to the number of the sub-image data sets in a manner that the model depth remains unchanged and the model width is evenly divided; The connection relationship between the nodes in each initial width sub-image classification model is retained, and the connection relationship between the nodes of different initial width sub-image classification models is deleted to obtain the final multiple initial width sub-image classification models.

18. An image classification device, characterized in that: Applicable to heterogeneous platforms, the device includes a graphics processor and a central processing unit; the graphics processor includes a parallel training unit, a node determination unit, an analysis unit and an identification unit; the central processing unit includes an addition unit and a transmission unit; The parallel training unit is configured to independently and parallelly train the initial width sub-image classification model matched to the sub-image dataset sent by the central processing unit to obtain trained classification models for each width sub-image; the initial width sub-image classification model is obtained by the central processing unit splitting the initial image classification model according to the width dimension; The node determination unit is configured to analyze, based on the merged data set sent by the central processing unit, the similarities between the feature images output by the nodes of each layer of the width sub-image classification model under the merged data set, so as to determine the node of each layer in each width sub-image classification model that has the greatest similarity with the node in the corresponding layer of each other width sub-image classification model; The adding unit is used to add a merging weight parameter between the nodes of each layer in each width sub-image classification model and the nodes with the greatest similarity in the corresponding layers in other width sub-image classification models; The transmission unit is configured to transmit the combined weight parameter to each of the graphics processors; The analysis unit is configured to analyze, in accordance with the merging weight parameter, the feature images corresponding to the nodes of each layer in each width sub-image classification model and the nodes with the greatest similarity in the corresponding layers of the other width sub-image classification models, layer by layer, in each of the graphics processors, to obtain a width-merged image classification model; The recognition unit is used for each of the graphics processors to perform parallel training on the width-merged image classification models deployed respectively, and use the trained image classification models to analyze the acquired image to be recognized to determine the image category to which the image to be recognized belongs.

19. An image classification device, characterized in that include: memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the image classification method according to any one of claims 1 to 17.

20. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the image classification method according to any one of claims 1 to 17.

Images