Image recognition method and device
By constructing an image recognition integration model and using the characteristics of multiple sub-models to determine the loss function, the problem of poor performance of image recognition models in the prior art when defending against attacks is solved, and higher adversarial robustness and recognition accuracy are achieved.
Patent Information
- Application Number
- CN202411995323.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-06-06
AI Technical Summary
In the prior art, image recognition models perform poorly when defending against adversarial attacks, and it is difficult to effectively defend against different types of adversarial attacks.
By constructing an image recognition ensemble model, the loss function of the ensemble model is determined using the characteristics of multiple image recognition sub-models, including integrated cross-entropy loss, weight correlation and weight concentration, to improve the robustness of the model.
The diversity analysis of the image recognition ensemble model when defending against attacks is achieved is more accurate and has better adversarial robustness, thereby improving the accuracy of image recognition.
Smart Images
Figure CN120107645A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an image recognition method and device. Background Art
[0002] Image recognition models based on deep learning are very vulnerable to adversarial attacks. Adversarial attacks construct adversarial samples by adding perturbations to normal image samples to be recognized, which can cause the image recognition model to output incorrect recognition results without affecting human judgment. Therefore, researchers not only study how to construct adversarial samples to check the vulnerability of image recognition models based on deep learning, but also study related defense methods to enhance the security and robustness of image recognition models based on deep learning.
[0003] Research shows that adversarial training is the most widely used method to defend against adversarial attacks, which sacrifices the generalization of deep learning-based image recognition models to improve robustness. However, since the strategy for constructing adversarial samples in adversarial training is fixed, it is very difficult to defend against different types of adversarial attacks. Summary of the invention
[0004] The present invention provides an image recognition method and device, which are used to solve the defect of poor performance of image recognition model in the prior art in defending against attacks, and achieve the effect of improving the accuracy of image recognition.
[0005] In a first aspect, the present invention provides an image recognition method, comprising: Based on the image sample data and the corresponding image recognition result labels, multiple image recognition sub-models in the image recognition integrated model are trained in parallel until the loss function of the image recognition integrated model converges to obtain a trained image recognition integrated model; Inputting the image to be recognized into the trained image recognition integrated model to obtain an image recognition result output by the trained image recognition integrated model; The loss function is determined as follows: Based on the image sample data and the corresponding image recognition result labels, determining the weight correlation between the image recognition sub-models and the weight information entropy of each layer in each image recognition sub-model; Determining the weight correlation of the image recognition integrated model based on the weight correlation between the various image recognition sub-models; Based on the weight information entropy of each layer in each image recognition sub-model, the weight centralization of each image recognition sub-model is determined; Determining the integrated cross entropy loss of the image recognition integrated model based on the cross entropy losses of each image recognition sub-model; The loss function is determined based on the integrated cross entropy loss, the weight correlation of the image recognition integrated model and the weight centrality of each image recognition sub-model.
[0006] In one embodiment, determining the weight correlation of the image recognition integrated model based on the weight correlation between the image recognition sub-models includes: The weight correlations between the image recognition sub-models are arithmetic averaged to obtain the weight correlation of the image recognition integrated model.
[0007] In one embodiment, the method further comprises: Based on the similarity of each layer of the weight matrix between the image recognition sub-models, the weight correlation between the image recognition sub-models is determined.
[0008] In one embodiment, determining the weight centrality of each image recognition sub-model based on the weight information entropy of each layer in each image recognition sub-model includes: The weight information entropy of each layer in each image recognition sub-model is summed to obtain the weight centralization of each image recognition sub-model.
[0009] In one embodiment, the method further comprises: Based on the weight parameters of each layer in each image recognition sub-model, the weight information entropy of each layer in each image recognition sub-model is determined.
[0010] In one embodiment, the method further comprises: The cross entropy losses of each image recognition sub-model are summed to obtain the integrated cross entropy loss of the image recognition integrated model.
[0011] In a second aspect, the present invention provides an image recognition device, comprising: A training module, used to train multiple image recognition sub-models in the image recognition integrated model in parallel based on the image sample data and the corresponding image recognition result labels, until the loss function of the image recognition integrated model converges to obtain a trained image recognition integrated model; A recognition module, used for inputting the image to be recognized into the trained image recognition integrated model to obtain the image recognition result output by the trained image recognition integrated model; The loss function is determined as follows: Based on the image sample data and the corresponding image recognition result labels, determining the weight correlation between the image recognition sub-models and the weight information entropy of each layer in each image recognition sub-model; Determining the weight correlation of the image recognition integrated model based on the weight correlation between the various image recognition sub-models; Based on the weight information entropy of each layer in each image recognition sub-model, the weight centralization of each image recognition sub-model is determined; Determining the integrated cross entropy loss of the image recognition integrated model based on the cross entropy losses of each image recognition sub-model; The loss function is determined based on the integrated cross entropy loss, the weight correlation of the image recognition integrated model and the weight centrality of each image recognition sub-model.
[0012] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image recognition method as described in the first aspect above when executing the computer program.
[0013] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image recognition method as described in the first aspect above.
[0014] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the image recognition method as described in the first aspect above is implemented.
[0015] The image recognition method and device provided by the present invention construct the loss function of the image recognition integrated model according to the integrated cross entropy loss and weight correlation of the image recognition integrated model and the weight centralization of each image recognition sub-model, so that the image integrated model fully integrates the characteristics of each image recognition sub-model, thereby making the diversity analysis of the image recognition integrated model more accurate and having better adversarial robustness when defending against adversarial attacks, thereby improving the accuracy of image recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0017] Figure 1 It is a flow chart of the image recognition method provided by the present invention.
[0018] Figure 2 It is a schematic diagram of iterative optimization of the image recognition integrated model provided by the present invention.
[0019] Figure 3 It is a schematic diagram of the acquisition process of the image recognition integrated model provided by the present invention.
[0020] Figure 4 It is a structural schematic diagram of the image recognition device provided by the present invention.
[0021] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0023] Figure 1 It is a flow chart of the image recognition method provided by the present invention, such as Figure 1 As shown, the method may include the following steps: Step 110: Based on the image sample data and the corresponding image recognition result labels, multiple image recognition sub-models in the image recognition integrated model are trained in parallel until the loss function of the image recognition integrated model converges to obtain a trained image recognition integrated model; Step 120: input the image to be recognized into the trained image recognition integrated model to obtain the image recognition result output by the trained image recognition integrated model; Among them, the loss function is determined as follows: Based on the image sample data and the corresponding image recognition result labels, determine the weight correlation between each image recognition sub-model and the weight information entropy of each layer in each image recognition sub-model; Determine the weight correlation of the image recognition integrated model based on the weight correlation between each image recognition sub-model; Based on the weight information entropy of each layer in each image recognition sub-model, the weight centralization of each image recognition sub-model is determined; Based on the cross entropy losses of each image recognition sub-model, determine the integrated cross entropy loss of the image recognition integrated model; The loss function is determined based on the integrated cross entropy loss, the weight correlation of the image recognition integrated model and the weight centrality of each image recognition sub-model.
[0024] It should be noted that the execution subject of the above-mentioned image recognition method can be a computer device, such as a mobile phone, a tablet computer, a laptop computer, a PDA, etc.
[0025] In the present invention, the image recognition integrated model is a model that combines multiple individual image recognition sub-models to achieve better prediction performance and generalization ability. By combining the prediction results of multiple image recognition sub-models, the image recognition integrated model can reduce the bias, variance or error of a single image recognition sub-model and improve the overall prediction accuracy.
[0026] The types of the image recognition sub-models in the image recognition integrated model may be different, or they may be of the same type but use different model parameters.
[0027] For example, an image recognition integrated model may include four image recognition sub-models, namely DNN (deep neural network), CNN (convolutional neural network), ResNet (residual network) and GAN (generative adversarial network). Alternatively, an image recognition integrated model may include four DNNs that use different model parameters. Alternatively, an image recognition integrated model may include two CNNs and one DNN that use different model parameters. And so on.
[0028] The above are only examples of the application of the image recognition integrated model and are not intended to limit the specific structure of the image recognition integrated model.
[0029] In the present invention, the image to be identified may be any image including a target object. For example, for an image recording a lakeside scene, the target object may be various objects such as lake water, water plants in the lake, and ducks in the lake. The specific content and type of the image to be identified, as well as the target object of the image to be identified, may be adjusted according to actual needs, and the present invention does not specifically limit this.
[0030] In step 110, multiple rounds of iterative training may be performed in parallel on multiple image recognition sub-models in the image recognition integrated model based on the image sample data and the image recognition result labels corresponding to the image samples.
[0031] In each round of iteration, each image recognition sub-model will adjust its own parameters based on the image sample data and the image recognition result label corresponding to the image sample. The loss function of the image recognition integrated model will be adjusted according to the parameters of each image recognition sub-model until convergence. The image recognition integrated model after the loss function converges is the trained image recognition integrated model. The following is a detailed description: Ensemble models for image recognition In general, when each image recognition sub-model is trained in parallel based on the image sample data and the corresponding image recognition result label, the cross entropy loss of each image recognition sub-model can be obtained. , and can be further based on the cross entropy loss of each image recognition sub-model Get the integrated cross entropy loss of the image recognition integrated model .
[0032] Furthermore, in each round of iteration, the parameters of each layer of each image recognition sub-model will change according to the image sample data and the corresponding image recognition result label. By calculating the similarity between layers, the weight correlation between each image recognition sub-model can be obtained. , and further weight correlation between each image recognition sub-model Get the weight correlation of the image recognition ensemble model .
[0033] The similarity calculation may be performed using various similarity algorithms, such as a cosine similarity algorithm, a Euclidean distance algorithm, a Pearson correlation coefficient algorithm, a Jaccard similarity coefficient algorithm, and the like.
[0034] By calculating the weight information entropy of each layer in each image recognition sub-model , we can get the weight concentration of each image recognition sub-model ,in, Represents the parameters of the image recognition ensemble model.
[0035] Determining weight relevance for image recognition ensemble models and the weight concentration of each image recognition sub-model After that, we use the integrated cross entropy loss , Weight Correlation of Image Recognition Ensemble Model and the weight concentration of each image recognition sub-model Constructing the loss function of the image recognition ensemble model .
[0036] Specifically, the loss function can be calculated according to the following formula: = + + To further improve the accuracy of the loss function, the weight correlation of the image recognition ensemble model can also be determined. The weight parameter , and the weight concentration of each image recognition sub-model The weight parameter , then the loss function can be calculated according to the following formula: = + + Among them, the weight parameter And the weight parameters It is a preset value, and its size can be adjusted according to actual needs, and the present invention does not make any specific limitation on this.
[0037] Based on the image sample data and the corresponding image recognition result labels, multiple rounds of iterative training are performed on each image recognition sub-model in parallel until the loss function Convergence, we can get the parameters of the trained image recognition ensemble model , based on this parameter, the trained image recognition ensemble model can be determined.
[0038] After the image recognition integrated model is trained in step 110 to obtain the trained image recognition integrated model, the image to be recognized can be input into the trained image recognition integrated model in step 120 to obtain the image recognition result output by the trained image recognition integrated model.
[0039] The image recognition method provided by the present invention constructs the loss function of the image recognition integrated model according to the integrated cross entropy loss and weight correlation of the image recognition integrated model and the weight centralization of each image recognition sub-model, so that the image integrated model fully integrates the characteristics of each image recognition sub-model, thereby making the diversity analysis of the image recognition integrated model more accurate and having better adversarial robustness when defending against adversarial attacks, thereby improving the accuracy of image recognition.
[0040] In one embodiment, the image recognition method provided by the present invention may further include: The cross entropy losses of each image recognition sub-model are summed to obtain the integrated cross entropy loss of the image recognition integrated model.
[0041] For example, let the cross entropy loss of each image recognition sub-model be ,in, Indicates The image recognition sub-model is used to input the model The predicted value of Represents input The true value of To calculate the Cross entropy loss for the image recognition sub-model.
[0042] By summing the cross entropy losses of each image recognition sub-model, we can get the integrated cross entropy loss , as follows: in, Indicates the number of image recognition sub-models.
[0043] The image recognition method provided by the present invention obtains the integrated cross entropy loss of the image recognition integrated model by summing the cross entropy losses of each image recognition sub-model, thereby improving the robustness of the image recognition integrated model.
[0044] In one embodiment, determining the weight correlation of the image recognition integrated model based on the weight correlation between the respective image recognition sub-models may include: The weight correlations between the image recognition sub-models are arithmetic averaged to obtain the weight correlation of the image recognition integrated model.
[0045] Specifically, the weight correlation between the various image recognition sub-models can be determined based on the weight matrix of each layer of every two image recognition sub-models.
[0046] For those with For the image recognition integrated model with image recognition sub-models, the weight correlation between each image recognition sub-model is calculated. , which can be specifically expressed as follows: in, Represents the image recognition sub-model No. The layer weight matrix, Represents the image recognition sub-model No. The layer weight matrix, represents the 2-norm; is the number of image recognition sub-models.
[0047] The weight correlation between each image recognition sub-model is taken as the arithmetic average to obtain the weight correlation of the image recognition integrated model. , as follows: in, Indicates the number of image recognition sub-models.
[0048] The image recognition method provided by the present invention improves the robustness of the image recognition integrated model, including the following two aspects: reducing the weight correlation between image recognition sub-models, which is defined as the similarity of the weight matrices between the image recognition sub-models. Intuitively, the loss gradient relative to the input is determined by the model weight; increasing the weight centralization of the image recognition sub-models, so that fewer weights play a more important role in decision-making, and will not affect the diversity of the image recognition integrated model.
[0049] In one embodiment, determining the weight centrality of each image recognition sub-model based on the weight information entropy of each layer in each image recognition sub-model may include: The weight information entropy of each layer in each image recognition sub-model is summed to obtain the weight centralization of each image recognition sub-model.
[0050] Specifically, the weight information entropy of each layer in each image recognition sub-model can be determined according to the weight parameter of each layer in each image recognition sub-model.
[0051] Calculate the weight information entropy of each layer of the image recognition sub-model , as follows: in, Represents the image recognition sub-model Layer weight parameters, is the number of weights per layer.
[0052] The weight information entropy of each layer is summed to obtain the weight concentration of each image recognition sub-model , as follows: in, Indicates the number of layers of the deep learning model corresponding to the image recognition sub-model.
[0053] Figure 2 It is a schematic diagram of iterative optimization of the image recognition integrated model provided by the present invention.
[0054] Reference Figure 2 , the diversity of the image recognition integrated model is analyzed for the parameter matrix of each layer of the image recognition sub-model in the image recognition integrated model. The present invention calculates the loss function of the image recognition integrated model by integrating the cross entropy loss, the weight correlation of the image recognition integrated model, and the weight concentration of each image recognition sub-model, iteratively optimizes the corresponding image recognition sub-model until the loss function converges, and obtains the final result, that is, the trained image recognition integrated model, thereby optimizing the diversity of the image recognition integrated model and providing better adversarial defense.
[0055] The following describes the experimental settings of four main parts of an example of applying the image recognition method provided by the present invention: 1. Dataset selection: Three benchmark datasets were selected based on the complexity of the task: CIFAR-10, CIFAR-100, and Tiny-ImageNet. CIFAR-10 contains 60,000 color images belonging to 10 categories, and CIFAR-100 contains 60,000 color images belonging to 100 categories. The resolution of both images is 32*32*3 pixels. Tiny-ImageNet contains 120,000 color images belonging to 200 categories with a resolution of 64*64*3 pixels.
[0056] 2. Target model selection: We selected an image recognition ensemble model consisting of K ResNet-18 networks, and used a 3-model and a 5-model for testing. and The test model evaluates the robustness of the image recognition ensemble model by the recognition accuracy under adversarial attacks. The higher the recognition accuracy, the higher the robustness of the image recognition ensemble model against adversarial attacks.
[0057] 3. Threat model classification: Based on the attacker's knowledge of the target model, threat models are divided into four types: (1) White-box attack: The attacker has full access to all information of the target model, including gradients and parameters.
[0058] (2) Type I black box attack: The attacker does not know all the information of the target model.
[0059] (3) Type II black-box attack: The attacker knows the basic architecture of the target model but lacks specific details, such as the exact structure of the neural network and access to its gradients and parameters.
[0060] (4) Type III black-box attack: The attacker knows the specific structure of the target model but cannot obtain any information about its gradients and parameters.
[0061] 4. Attack method: We select a diverse set of untargeted adversarial attack strategies, including the Fast Gradient Sign Method (FGSW), the Basic Iterative Method (BIM), the Projected Gradient Descent (PGD), and the C&W attack method. We set control parameters for each attack method and implement the experiment based on Pytorch and the Adversarial Robustness Toolbox (ART).
[0062] Figure 3 is a schematic diagram of the acquisition process of the image recognition integrated model provided by the present invention, referring to Figure 3 In this example, obtaining the image recognition integrated model may include: Step 1) Image recognition integration model , train multiple image recognition sub-models in parallel, and obtain the cross entropy loss of each image recognition sub-model and the integrated cross entropy loss of the image recognition ensemble model ; Step 2) Use the cosine similarity between layers to calculate the weight correlation between each image recognition sub-model , calculate the arithmetic mean to get the weight correlation of the image recognition ensemble model .
[0063] Calculating weight correlations for image recognition ensemble models The method is: Step 3) Using the weight parameters of each layer of the image recognition sub-model, we calculate the weight information entropy of each layer in the image recognition sub-model. Get the weight concentration of each image recognition sub-model .
[0064] Step 4) Determine the weight correlation of the image recognition ensemble model The weight parameter and weight concentration The weight parameter , using integrated cross entropy loss , Weight Correlation of Image Recognition Ensemble Model and weight concentration Constructing the loss function of the image recognition ensemble model . Train the image recognition sub-models in parallel until Converge and obtain the parameters of the trained image recognition ensemble model , and promote the maximization of diversity among its image recognition sub-models.
[0065] For the convenience of description, assume the following simplified application example: The input image set consists of 60,000 32*32 RGB color images, with a total of 10 categories and 6,000 images in each category.
[0066] According to the calculation steps mentioned above, implement them in sequence: The first step is to construct an image recognition integrated model F. The image recognition integrated model consists of three ResNet18 networks. The ResNet18 network consists of five convolutional layers and one pooling layer. The network structure is shown in Table 1: Table 1
[0067] The training set of the above pictures Input image recognition ensemble model , using a deep integration-based synchronous training method for image recognition ensemble models Train each image recognition sub-model in to obtain the trained image recognition integrated model , get the weight parameters of each layer .
[0068] Calculate the cross entropy loss for each image recognition sub-model ,in, Indicates The image recognition sub-model is used to input the model The predicted value of Represents input The true value of To calculate the The cross entropy loss of each image recognition sub-model is summed to get the integrated cross entropy loss .
[0069] In the second step, the cosine similarity between layers is used to calculate the weight correlation between each image recognition sub-model: in, Representation Model No. The layer weight matrix, represents the 2-norm, is the number of image recognition sub-models.
[0070] The weight correlation of each two image recognition sub-models is taken as the arithmetic average to obtain the weight correlation of the image recognition integrated model. .
[0071] The third step is to use the weight parameters of each layer of the deep learning model to calculate the weight information entropy of each layer in the image recognition sub-model. ,in, Indicates Layer weight parameters, is the number of weights per layer.
[0072] The weight information entropy of each layer is summed to obtain the weight concentration of each image recognition sub-model .
[0073] Step 4: Determine weight relevance The weight parameter = 0.1 and weight concentration The weight parameter , using integrated cross entropy loss , weight correlation and weight concentration Constructing the loss function of the image recognition ensemble model . Train the image recognition sub-models in parallel until Converge and obtain the trained image recognition integrated model And promote the maximization of diversity among its image recognition sub-models.
[0074] Among them, the integrated cross entropy loss obtained in each round of training , weight correlation and weight concentration Some of the data are shown in Table 2: Table 2
[0075] When the attacker can fully understand the structure of the image recognition integrated model and the specific parameters of each layer of each image recognition sub-model, the robustness on simple data sets (such as CIFAR-10) is not lower than that of existing methods; it shows better robustness on complex data sets (such as Tiny-Image). When the attacker does not understand the internal details of the image recognition integrated model, the image recognition integrated model obtained by the method provided by the present invention has better robustness.
[0076] The image recognition method provided by the present invention can improve the robustness of the image recognition integrated model by changing the loss function in the training process of the image recognition integrated model and optimizing the weight parameters between the image recognition sub-models. Compared with a single image recognition model, the robustness is improved while ensuring similar training time; compared with the method based on loss gradient, the training speed is increased and the training results are optimized, thereby improving the efficiency and accuracy of image recognition.
[0077] The image recognition device provided by the present invention is described below. The image recognition device described below and the image recognition method described above can refer to each other and can achieve the same technical effects, which will not be repeated here.
[0078] Figure 4 Schematic diagram of the structure of the image recognition device provided by the present invention. Figure 4 As shown, the image recognition device provided by the present invention may include: A training module 410 is used to train multiple image recognition sub-models in the image recognition integrated model in parallel based on the image sample data and the corresponding image recognition result labels until the loss function of the image recognition integrated model converges to obtain a trained image recognition integrated model; The recognition module 420 is used to input the image to be recognized into the trained image recognition integrated model to obtain the image recognition result output by the trained image recognition integrated model; The loss function is determined as follows: Based on the image sample data and the corresponding image recognition result labels, determining the weight correlation between the image recognition sub-models and the weight information entropy of each layer in each image recognition sub-model; Determining the weight correlation of the image recognition integrated model based on the weight correlation between the various image recognition sub-models; Based on the weight information entropy of each layer in each image recognition sub-model, the weight centralization of each image recognition sub-model is determined; Determining the integrated cross entropy loss of the image recognition integrated model based on the cross entropy losses of each image recognition sub-model; The loss function is determined based on the integrated cross entropy loss, the weight correlation of the image recognition integrated model and the weight centrality of each image recognition sub-model.
[0079] In one embodiment, the training module 410 is specifically used to: The weight correlations between the image recognition sub-models are arithmetic averaged to obtain the weight correlation of the image recognition integrated model.
[0080] In one embodiment, the training module 410 is further used to: Based on the similarity of each layer of the weight matrix between the image recognition sub-models, the weight correlation between the image recognition sub-models is determined.
[0081] In one embodiment, the training module 410 is specifically used to: The weight information entropy of each layer in each image recognition sub-model is summed to obtain the weight centralization of each image recognition sub-model.
[0082] In one embodiment, the training module 410 is further used to: Based on the weight parameters of each layer in each image recognition sub-model, the weight information entropy of each layer in each image recognition sub-model is determined.
[0083] In one embodiment, the training module 410 is further used to: The cross entropy losses of each image recognition sub-model are summed to obtain the integrated cross entropy loss of the image recognition integrated model.
[0084] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the image recognition method described in any of the above embodiments, for example, including: Based on the image sample data and the corresponding image recognition result labels, multiple image recognition sub-models in the image recognition integrated model are trained in parallel until the loss function of the image recognition integrated model converges to obtain a trained image recognition integrated model; Inputting the image to be recognized into the trained image recognition integrated model to obtain an image recognition result output by the trained image recognition integrated model; The loss function is determined as follows: Based on the image sample data and the corresponding image recognition result labels, determining the weight correlation between the image recognition sub-models and the weight information entropy of each layer in each image recognition sub-model; Determining the weight correlation of the image recognition integrated model based on the weight correlation between the various image recognition sub-models; Based on the weight information entropy of each layer in each image recognition sub-model, the weight centralization of each image recognition sub-model is determined; Determining the integrated cross entropy loss of the image recognition integrated model based on the cross entropy losses of each image recognition sub-model; The loss function is determined based on the integrated cross entropy loss, the weight correlation of the image recognition integrated model and the weight centrality of each image recognition sub-model.
[0085] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0086] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program may be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the image recognition method described in any of the above embodiments, for example, including: Based on the image sample data and the corresponding image recognition result labels, multiple image recognition sub-models in the image recognition integrated model are trained in parallel until the loss function of the image recognition integrated model converges to obtain a trained image recognition integrated model; Inputting the image to be recognized into the trained image recognition integrated model to obtain an image recognition result output by the trained image recognition integrated model; The loss function is determined as follows: Based on the image sample data and the corresponding image recognition result labels, determining the weight correlation between the image recognition sub-models and the weight information entropy of each layer in each image recognition sub-model; Determining the weight correlation of the image recognition integrated model based on the weight correlation between the various image recognition sub-models; Based on the weight information entropy of each layer in each image recognition sub-model, the weight centralization of each image recognition sub-model is determined; Determining the integrated cross entropy loss of the image recognition integrated model based on the cross entropy losses of each image recognition sub-model; The loss function is determined based on the integrated cross entropy loss, the weight correlation of the image recognition integrated model and the weight centrality of each image recognition sub-model.
[0087] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the image recognition method described in any of the above embodiments, for example, including: Based on the image sample data and the corresponding image recognition result labels, multiple image recognition sub-models in the image recognition integrated model are trained in parallel until the loss function of the image recognition integrated model converges to obtain a trained image recognition integrated model; Inputting the image to be recognized into the trained image recognition integrated model to obtain an image recognition result output by the trained image recognition integrated model; The loss function is determined as follows: Based on the image sample data and the corresponding image recognition result labels, determining the weight correlation between the image recognition sub-models and the weight information entropy of each layer in each image recognition sub-model; Determining the weight correlation of the image recognition integrated model based on the weight correlation between the various image recognition sub-models; Based on the weight information entropy of each layer in each image recognition sub-model, the weight centralization of each image recognition sub-model is determined; Determining the integrated cross entropy loss of the image recognition integrated model based on the cross entropy losses of each image recognition sub-model; The loss function is determined based on the integrated cross entropy loss, the weight correlation of the image recognition integrated model and the weight centrality of each image recognition sub-model.
[0088] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0089] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An image recognition method, characterized in that: include: Based on the image sample data and the corresponding image recognition result labels, multiple image recognition sub-models in the image recognition integrated model are trained in parallel until the loss function of the image recognition integrated model converges to obtain a trained image recognition integrated model; Inputting the image to be recognized into the trained image recognition integrated model to obtain an image recognition result output by the trained image recognition integrated model; The loss function is determined as follows: Based on the image sample data and the corresponding image recognition result labels, determining the weight correlation between the image recognition sub-models and the weight information entropy of each layer in each image recognition sub-model; Determining the weight correlation of the image recognition integrated model based on the weight correlation between the various image recognition sub-models; Based on the weight information entropy of each layer in each image recognition sub-model, the weight centralization of each image recognition sub-model is determined; Determining the integrated cross entropy loss of the image recognition integrated model based on the cross entropy losses of each image recognition sub-model; The loss function is determined based on the integrated cross entropy loss, the weight correlation of the image recognition integrated model and the weight centrality of each image recognition sub-model.
2. The image recognition method according to claim 1, characterized in that: The step of determining the weight correlation of the image recognition integrated model based on the weight correlation between the image recognition sub-models includes: The weight correlations between the image recognition sub-models are arithmetic averaged to obtain the weight correlation of the image recognition integrated model.
3. The image recognition method according to claim 2, characterized in that: Also includes: Based on the similarity of each layer of the weight matrix between the image recognition sub-models, the weight correlation between the image recognition sub-models is determined.
4. The image recognition method according to claim 1, characterized in that: The step of determining the weight centrality of each image recognition sub-model based on the weight information entropy of each layer in each image recognition sub-model includes: The weight information entropy of each layer in each image recognition sub-model is summed to obtain the weight centralization of each image recognition sub-model.
5. The image recognition method according to claim 4, characterized in that: Also includes: Based on the weight parameters of each layer in each image recognition sub-model, the weight information entropy of each layer in each image recognition sub-model is determined.
6. The image recognition method according to claim 1, characterized in that: Also includes: The cross entropy losses of each image recognition sub-model are summed to obtain the integrated cross entropy loss of the image recognition integrated model.
7. An image recognition device, characterized in that: include: A training module, used to train multiple image recognition sub-models in the image recognition integrated model in parallel based on the image sample data and the corresponding image recognition result labels, until the loss function of the image recognition integrated model converges to obtain a trained image recognition integrated model; A recognition module, used for inputting the image to be recognized into the trained image recognition integrated model to obtain the image recognition result output by the trained image recognition integrated model; The loss function is determined as follows: Based on the image sample data and the corresponding image recognition result labels, determining the weight correlation between the image recognition sub-models and the weight information entropy of each layer in each image recognition sub-model; Determining the weight correlation of the image recognition integrated model based on the weight correlation between the various image recognition sub-models; Based on the weight information entropy of each layer in each image recognition sub-model, the weight centralization of each image recognition sub-model is determined; Determining the integrated cross entropy loss of the image recognition integrated model based on the cross entropy losses of each image recognition sub-model; The loss function is determined based on the integrated cross entropy loss, the weight correlation of the image recognition integrated model and the weight centrality of each image recognition sub-model.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the image recognition method according to any one of claims 1 to 6 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image recognition method according to any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image recognition method according to any one of claims 1 to 6 is implemented.