Model training method, device, electronic device and storage medium

Abstract

The embodiments of the present invention are applicable to the field of computer technology, and provide a model training method, device, electronic device and storage medium, wherein the model training method includes: blocking part of the image content of each first image in at least two first images to obtain at least two second images; the at least two first images correspond to at least two image categories, and the similarity between the first images of different categories in the at least two first images is greater than a set value; training a first model based on the at least two first images and at least two second images; the trained first model is used to identify the image category of the input image.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a model training method, device, electronic equipment and storage medium. Background Art

[0002] At present, many office scenarios require the use of image recognition models to identify image categories. For sensitive office images that are easy to confuse, such as invoices and red-stamp documents, related technologies are prone to identification errors when using image recognition models, and the recognition accuracy of image recognition models is low. Summary of the invention

[0003] In order to solve the above problems, the embodiments of the present invention provide a model training method, device, electronic device and storage medium to at least solve the problem of low recognition accuracy of the image recognition model in the related art.

[0004] The technical solution of the present invention is achieved in this way:

[0005] In a first aspect, an embodiment of the present invention provides a model training method, the method comprising:

[0006] Obstructing a portion of image content of each of at least two first images to obtain at least two second images; the at least two first images correspond to at least two image categories, and the similarity between first images of different categories in the at least two first images is greater than a set value;

[0007] A first model is trained based on the at least two first images and the at least two second images; the first model obtained by training is used to identify the image category of the input image.

[0008] In the above solution, the step of blocking a partial image area of ​​each of the at least two first images includes:

[0009] selecting a set area in each of the at least two first images;

[0010] The set area is filled with pixel blocks of a set color.

[0011] In the above solution, the step of blocking a partial image area of ​​each of the at least two first images includes:

[0012] A partial image area is randomly selected in each of the at least two first images for blocking, so as to obtain at least two second images corresponding to each of the at least two first images.

[0013] In the above solution, the training of the first model based on the at least two first images and the at least two second images includes:

[0014] Inputting the at least two first images and the at least two second images into the first model, obtaining a first probability value of the first image and a second probability value of the corresponding second image output by the first model; the first probability value represents the image category of the corresponding first image; and the second probability value represents the image category of the corresponding second image;

[0015] Determine, based on the first probability value and the second probability value, a loss value of a set loss function corresponding to each of the at least two image categories;

[0016] The weight parameters of the first model are updated based on the loss value.

[0017] In the above solution, updating the weight parameters of the first model based on the loss value includes:

[0018] Performing weighted calculation on the loss value of the set loss function corresponding to each image category of the at least two image categories to obtain a weighted value;

[0019] A weight parameter of the first model is updated based on the weighted value.

[0020] In the above solution, before training the first model based on the at least two first images and the at least two second images, the method further includes:

[0021] Delete the set number of network layers and / or number of network channels of the neural network model to obtain the first model.

[0022] In the above solution, after the first model is obtained by training based on the at least two first images and the at least two second images, the method further includes:

[0023] The first model is converted into a second model; the second model is used to identify the image category of the at least one third image; the first model represents the PyTorch model; the second model represents the PaddlePaddle model.

[0024] In the above solution, converting the first model into the second model includes:

[0025] Converting the first model into an open neural network exchange format;

[0026] The first model in the open neural network exchange format is converted into the second model through a predetermined model conversion tool.

[0027] In a second aspect, an embodiment of the present invention provides an image recognition method, the method comprising:

[0028] Inputting a third image into the first model or the second model to obtain a third probability value output by the first model or the second model; the third probability value represents an image category of the third image;

[0029] determining a difference between the third probability value and a fourth probability value corresponding to each of the at least two categories;

[0030] An image category of the third image is determined based on the difference value.

[0031] In a third aspect, an embodiment of the present invention provides a model training device, the device comprising:

[0032] a masking module, configured to mask part of the image content of each of at least two first images to obtain at least two second images; the at least two first images correspond to at least two image categories, and the similarity between first images of different categories in the at least two first images is greater than a set value;

[0033] A training module is used to train a first model based on the at least two first images and the at least two second images; the first model obtained by training is used to identify the image category of an input image.

[0034] In a fourth aspect, an embodiment of the present invention provides an image recognition device, the device comprising:

[0035] An input module, used for inputting a third image into the first model or the second model to obtain a third probability value output by the first model or the second model; the third probability value represents the image category of the third image;

[0036] a determination module, configured to determine a difference between the third probability value and a fourth probability value corresponding to each of at least two categories;

[0037] A recognition module is used to determine the image category of the third image based on the difference.

[0038] In a fifth aspect, an embodiment of the present invention provides an electronic device, comprising a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the steps of the model training method provided in the first aspect or the steps of the image recognition method provided in the second aspect of the embodiment of the present invention.

[0039] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, including: the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the model training method provided in the first aspect of the embodiment of the present invention or the steps of the image recognition method provided in the second aspect are implemented.

[0040] The embodiment of the present invention obtains at least two second images by blocking part of the image content of each first image in at least two first images. A first model is trained based on the at least two first images and the at least two second images, and the trained first model is used to identify the image category of the input image. Among them, the at least two first images correspond to at least two image categories, and the similarity between the first images of different categories in the at least two first images is greater than a set value. The embodiment of the present invention trains the first model by at least two first images and at least two second images, and can expand the number of training samples of the first model for various categories of images, thereby enhancing the model's recognition ability for various categories of images, achieving the distinction between easily confused categories of images, and improving the recognition accuracy of the image recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of an implementation flow of a model training method provided by an embodiment of the present invention;

[0042] Figure 2 It is a schematic diagram of the implementation flow of another model training method provided by an embodiment of the present invention;

[0043] Figure 3 It is a schematic diagram of the implementation flow of another model training method provided by an embodiment of the present invention;

[0044] Figure 4 It is a schematic diagram of the implementation flow of another model training method provided by an embodiment of the present invention;

[0045] Figure 5 It is a schematic diagram of an implementation flow of a model conversion method provided by an embodiment of the present invention;

[0046] Figure 6 It is a schematic diagram of an implementation flow of an image recognition method provided by an embodiment of the present invention;

[0047] Figure 7 is a schematic diagram of an image processing flow provided by an application embodiment of the present invention;

[0048] Figure 8 is a schematic diagram of a model training device provided by an embodiment of the present invention;

[0049] Fig. 9 is a schematic diagram of an image recognition device provided by an embodiment of the present invention;

[0050] Fig.10 It is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] In office scenarios, there is often a need for image classification. A small number of images can be classified through manual recognition, but in the case of a large number of images, image recognition models are needed for image classification. For easily confused sensitive office images, such as invoices, seal documents, and red-headed documents, these easily confused images have similar features for computers and are difficult to distinguish. Therefore, the related technology has a low recognition accuracy when identifying easily confused sensitive office images.

[0053] Moreover, the current image recognition model for sensitive offices requires the design of specific algorithm structures for specific application scenarios, and has slow reasoning speed and low image recognition efficiency.

[0054] In view of the shortcomings of the above-mentioned related technologies, an embodiment of the present invention provides a model training method, which can at least improve the recognition accuracy of the image recognition model. In order to illustrate the technical solution of the present invention, a specific embodiment is used for illustration below.

[0055] Figure 1 1 is a schematic diagram of an implementation flow of a model training method provided by an embodiment of the present invention. The execution subject of the model training method is an electronic device, and the electronic device includes a desktop computer, a laptop computer, and a server. Figure 1 , model training methods include:

[0056] S101, masking part of the image content of each of at least two first images to obtain at least two second images; the at least two first images correspond to at least two image categories, and the similarity between first images of different categories in the at least two first images is greater than a set value.

[0057] Here, at least two first images are training data for the first model. The first model is an image recognition model with a pre-set network structure. After model training, the first model can be used to identify the image category of the input image.

[0058] The at least two first images include at least two categories of first images, such as at least two of the seven categories: code screenshots, invoices, seal documents, design drawings, waybill numbers, red-headed documents, and red-headed seal documents. In addition, in the at least two first images, the similarity between the first images of different categories is greater than a set value.

[0059] If the similarity between the first images of different categories is greater than the set value, it means that the first images of different categories are easily confused, and the features are similar and difficult to distinguish for the computer. Here, the similarity can be cosine similarity, which represents the image as a vector and characterizes the similarity of the two images by calculating the cosine distance between the vectors. The similarity can also be histogram similarity. The histogram can describe the global distribution of colors in an image. For example, if there are images A and B, the histograms of the two images, HistA and HistB, are calculated respectively, and then the normalized correlation coefficient (Bhattacharyya distance, histogram intersection distance) of the two histograms is calculated as the similarity. The similarity can also be calculated by the structural similarity measure (SSIM) and the perceptual hashing algorithm. The perceptual hashing algorithm generates a "fingerprint" string for each image, and then compares the fingerprints of different images. The closer the fingerprints are, the more similar the images are. SSIM measures the image similarity from three aspects: brightness, contrast, and structure. The SSIM value range is [0,1]. The larger the value, the smaller the image distortion. In practical applications, a sliding window can be used to divide the image into blocks. Let the total number of blocks be N. Considering the influence of window shape on the blocks, Gaussian weighting is used to calculate the mean, variance and covariance of each window, and then the structural similarity SSIM of the corresponding blocks is calculated. Finally, the average value is used as the structural similarity measure of the two images, that is, the average structural similarity SSIM.

[0060] Part of the image content of each of the at least two first images is blocked, for example, for image A, a rectangular frame with a solid background is inserted into image A, and a part of the image content of image A is blocked by the rectangular frame. It should be understood that the area of ​​the rectangular frame should be smaller than the area of ​​image A.

[0061] refer to Figure 2 In one embodiment, the randomly blocking a partial image area of ​​each of the at least two first images includes:

[0062] S201: Select a set area in each of the at least two first images.

[0063] Here, the set area is an area with a set shape and size. For example, the set area is a rectangular frame with a length and a width of 4 mm, and the rectangular frame is randomly inserted into any position of the first image.

[0064] In practical applications, the set region cannot completely cover the first image, so the area of ​​the set region should be smaller than the area of ​​the first image.

[0065] S202: Fill the set area with pixel blocks of a set color.

[0066] Here, pixel blocks of any color may be used to fill the set area. For example, white pixel blocks may be used to fill the set area to cover a partial image area of ​​the first image.

[0067] In practical applications, multiple occlusion areas can be generated on a first image at the same time. For example, three set areas can be selected on a first image, and the shapes of the three set areas can be different. In this way, three parts of the image content of the first image will be occluded.

[0068] In one embodiment, the step of blocking a partial image area of ​​each of the at least two first images includes:

[0069] A partial image area is randomly selected in each of the at least two first images for blocking, so as to obtain at least two second images corresponding to each of the at least two first images.

[0070] Here, each of the at least two first images generates multiple second images. For example, a set area is selected in the first image each time, and the position of the set area selected each time is different, that is, the image content blocked each time is different, so the multiple second images obtained are also different.

[0071] Blocking part of the image content of the first image does not change the category of the first image. For example, if the image content of the first image is a cat, the image content of the second image obtained by blocking the cat's ears is still a cat.

[0072] The more second images generated by the first image of each category, the more the model's recognition ability for images of that category can be enhanced during model training, the more robust the model can be, and the more easily confusing category images can be distinguished.

[0073] S102, training a first model based on the at least two first images and the at least two second images; the first model obtained by training is used to identify the image category of the input image.

[0074] refer to Figure 3 In one embodiment, the training of the first model based on the at least two first images and the at least two second images comprises:

[0075] S301, input the at least two first images and the at least two second images into the first model, and obtain a first probability value of the first image and a second probability value of the corresponding second image output by the first model; the first probability value represents the image category of the corresponding first image; the second probability value represents the image category of the corresponding second image.

[0076] At least two first images and at least two second images are training images of the first model. In practical applications, before using the training images to start training the first model, the training images need to be labeled, and the labels represent the categories of the corresponding images.

[0077] The first model is trained based on at least two labeled first images and at least two second images to enhance the model's ability to distinguish images of easily confused categories.

[0078] Here, during the model training process, the first model outputs a probability value, which represents the image category of the input image predicted by the model. The first probability value represents the image category of the corresponding first image, and the second probability value represents the image category of the corresponding second image.

[0079] Assume that there are 5 categories in total, and each category corresponds to a set probability value. For example, the probability value corresponding to the invoice category is 0.2; the probability value corresponding to the seal category is 0.4; the probability value corresponding to the red-headed document category is 0.6; the probability value corresponding to the waybill category is 0.8; and the probability value corresponding to the design drawing category is 1.0.

[0080] The closer the probability value output by the first model is to the set probability value corresponding to a certain category, the first model determines the corresponding category as the category of the input first image. For example, the first probability value output by the first model is 0.22, which is closest to the probability value corresponding to the invoice category, and the first model identifies the category of the input first image as the invoice category.

[0081] S302: Determine a loss value of a set loss function corresponding to each of the at least two image categories based on the first probability value and the second probability value.

[0082] Here, a loss function is set for each of the at least two image categories, and the first model completes training only when all loss functions converge. The set loss function is a matching loss function, and the set loss function is used to make the first probability value and the second probability value as similar as possible.

[0083] For the set loss function corresponding to each image category, the first probability value and the second probability value are substituted into the set loss function to solve the loss value. In an embodiment of the present invention, the smaller the difference between the first probability value and the second probability value, the smaller the loss value corresponding to the set loss function; the larger the difference between the first probability value and the second probability value, the larger the loss value corresponding to the set loss function. When the set loss function has the smallest loss value for , it can be considered that the set loss function converges. In practical applications, the set loss function can be an L2 loss function, which is also called the minimum square error, and the minimum value of the set loss function can be found by gradient descent.

[0084] S303: Update the weight parameters of the first model based on the loss value.

[0085] Model training is an iterative training process, which continuously uses training images to train the first model and update model parameters until the model converges.

[0086] Here, the weight parameter is the configuration variable inside the first model. Whether the first model is applicable or how effective it is depends largely on the setting of the weight parameter. By tuning the weight parameter, the first model can be optimized and the performance of the first model can be improved. In practical applications, the update of the weight parameter includes increasing the parameter value and decreasing the parameter value.

[0087] refer to Figure 4 In one embodiment, updating the weight parameters of the first model based on the loss value includes:

[0088] S401, performing weighted calculation on the loss value of the set loss function corresponding to each image category of the at least two image categories to obtain a weighted value.

[0089] S402: Update weight parameters of the first model based on the weighted value.

[0090] Since the set loss function corresponding to each image category has a loss value, if the weight parameters of the first model are adjusted according to each loss value alone, the weight parameters of the first model need to be adjusted multiple times, and it is difficult to balance the weight parameters of the first model. The embodiment of the present invention performs weighted calculation on the loss value of the set loss function corresponding to each image category to obtain a weighted value, and adjusts the weight parameters of the first model based on the weighted value. In this way, the weight parameters of the first model only need to be adjusted once, reducing the complexity of updating the weight parameters of the first model. In the weighted calculation, the weights of the loss values ​​corresponding to each image category can be the same or different.

[0091] In practical applications, the correspondence between the loss value and the weight parameter that needs to be adjusted can be set. After the loss value is obtained, the weight parameter of the first model is adjusted to the weight parameter corresponding to the loss value.

[0092] After updating the weight parameters of the first model, continue to train the first model, and iterate the training repeatedly until all loss functions of the first model converge, and the training of the first model is completed.

[0093] By using at least two first images and at least two second images to train the first model, the number of training samples of the first model for each type of image can be expanded, and the recognition ability of the first model for each type of image can be enhanced. Since the loss functions corresponding to different image categories converge at the same time, in the actual image classification process, if multiple images of easily confused categories are input at the same time, the first model can distinguish images of easily confused categories.

[0094] The embodiment of the present invention obtains at least two second images by blocking part of the image content of each first image in at least two first images. A first model is trained based on the at least two first images and the at least two second images, and the trained first model is used to identify the image category of the input image. Among them, the at least two first images correspond to at least two image categories, and the similarity between the first images of different categories in the at least two first images is greater than a set value. The embodiment of the present invention trains the first model by at least two first images and at least two second images, and can expand the number of training samples of the first model for various categories of images, thereby enhancing the model's recognition ability for various categories of images, improving the robustness of the model, achieving the distinction of easily confused categories of images, and improving the recognition accuracy of the image recognition model.

[0095] In one embodiment, before training the first model based on the at least two first images and the at least two second images, the method further includes:

[0096] Delete the set number of network layers and / or number of network channels of the neural network model to obtain the first model.

[0097] For example, the neural network model is set to the original MobileNet_V3 model. The network structure of the original MobileNet_V3 is shown in Table 1:

[0098] Input Operator exp size #out SE NL s 224x224x3 conv2d - 16 - HS 2 112x112x16 bneck,3x3 16 16 - RE 1 112x112x16 bneck,3x3 64 24 - RE 2 56x56x24 bneck,3x3 72 24 - RE 1 56x56x24 bneck,5x5 72 40 √ RE 2 28x28x40 bneck,5x5 120 40 √ RE 1 28x28x40 bneck,5x5 120 40 √ RE 1 28x28x40 bneck,3x3 240 80 - HS 2 14x14x80 bneck,3x3 200 80 - HS 1 14x14x80 bneck,3x3 184 80 - HS 1 14x14x80 bneck,3x3 184 80 - HS 1 14x14x80 bneck,3x3 480 112 √ HS 1 14x14x112 bneck,3x3 672 112 √ HS 1 14x14x112 bneck,5x5 672 160 √ HS 2 7x7x160 bneck,5x5 960 160 √ HS 1 7x7x160 bneck,5x5 960 160 √ HS 1 7x7x160 conv2d,1x1 - 960 - HS 1 7x7x960 pool,7x7 - - - - 1 1x1x960 conv2d,1x1,NBN - 1280 - HS 1 1x1x1280 conv2d,1x1,NBN - k - - 1

[0099] Table 1

[0100] The original MobileNet_V3 model is streamlined, and the number of network layers and / or network channels of the original MobileNet_V3 model is deleted. The network structure of the streamlined MobileNet_V3 is shown in Table 2:

[0101] Input Operator expsize #out SE NL s 224x224x3 conv2d - 16 - HS 2 112x112x16 bneck,3x3 16 16 √ RE 2 56x56x16 bneck,3x3 72 24 - RE 2 28x28x24 bneck,3x3 88 24 - RE 1 28x28x24 bneck,5x5 96 40 √ HS 2 14x14x40 bneck,5x5 240 40 √ HS 1 14x14x40 bneck,5x5 120 48 √ HS 1 14x14x48 bneck,5x5 288 96 √ HS 2 7x7x96 bneck,5x5 576 96 √ HS 1 7x7x96 bneck,5x5 576 96 √ HS 1 7x7x96 conv2d,1x1 - 480 - HS 1 7x7x480 pool,7x7 - - - - 1 1x1x480 conv2d,1x1,NBN - 512 - HS 1 1x1x512 conv2d,1x1,NBN - k - - 1

[0102] Table 2

[0103] Among them, Input is the input size; Operator represents the structure; expsize and #out represent the number of channels; SE is checked to indicate that the attention mechanism is used; NL represents different activation functions, RE represents ReLU, HS represents h-wish; s represents the convolution step size.

[0104] By reducing the number of network layers and channels, the network parameters are reduced. At the same time, a large step size is used to reduce the amount of calculation, thereby improving the processing speed of the first model and accelerating the recognition efficiency of the first model.

[0105] In one embodiment, after the first model is obtained by training based on the at least two first images and the at least two second images, the method further includes:

[0106] The first model is converted into a second model; the second model is used to identify the image category of the at least one third image; the first model represents the PyTorch model; the second model represents the PaddlePaddle model.

[0107] For example, the first model is a lightweight MobileNet_V3 network model under the pytorch framework, and the second model is a Paddle model.

[0108] The reason for converting the first model to the second model is that the Paddle model is built on C++, while the PyTorch model is built on Python. C++ is faster than Python. Moreover, the Paddle model is specially designed for inference, with better underlying optimization, and faster computing speed for image recognition.

[0109] refer to Figure 5 In the above embodiment, converting the first model into the second model includes:

[0110] S501, converting the first model into an open neural network exchange format.

[0111] Here, the open neural network exchange format is ONNX format. ONNX is an open file format designed for machine learning and used to store trained models. Mainstream frameworks such as Caffe2, PyTorch, Microsoft CognitiveToolkit and Apache MXNet all support ONNX to varying degrees. This facilitates the migration of models between different frameworks.

[0112] S502: Convert the first model in the open neural network exchange format into the second model using a set model conversion tool.

[0113] Here, the model conversion tool set is x2paddle. x2paddle is a model conversion tool that can convert TensorFlow and Caffe models into a format that can be loaded by PaddlePaddle's core framework Paddle Fluid. At the same time, X2Paddle also supports ONNX format model conversion, which is equivalent to supporting many frameworks that can be converted to ONNX format, such as PyTorch, MXNet, and CNTK.

[0114] Through x2paddle, the first model in ONNX format can be converted into the second model in Paddle structure. The converted Paddle model can be used for image recognition to speed up the model's image processing speed and reduce memory consumption. In actual applications, a single-core CPU can achieve a processing speed of 20 images per second and a recognition accuracy of 95% for images of different categories.

[0115] refer to Figure 6 , Figure 6 The present invention provides an embodiment of an image recognition method provided by the present invention. The image recognition method uses the first model or the second model trained by the above embodiment to perform image recognition. The image recognition method is performed by an electronic device, including a desktop computer, a laptop computer, and a server. Figure 6 , the image recognition method includes:

[0116] S601, input a third image into the first model or the second model to obtain a third probability value output by the first model or the second model; the third probability value represents the image category of the third image.

[0117] Here, both the first model and the second model can be used for image recognition, and the second model has more efficient image recognition capabilities.

[0118] For the input third image, the first model or the second model outputs a third probability value, and the third probability value represents the image category of the third image.

[0119] S602: Determine a difference between the third probability value and a fourth probability value corresponding to each of at least two categories.

[0120] Here, a fourth probability value is set for each category. Assuming there are 5 categories in total, for example, the fourth probability value corresponding to the invoice category is 0.2; the fourth probability value corresponding to the seal category is 0.4; the fourth probability value corresponding to the red-headed document category is 0.6; the fourth probability value corresponding to the waybill category is 0.8; and the fourth probability value corresponding to the design drawing category is 1.0.

[0121] A difference between the third probability value and a fourth probability value corresponding to each of the at least two categories is calculated.

[0122] S603: Determine an image category of the third image based on the difference.

[0123] The closer the third probability value output by the first model or the second model is to the fourth probability value corresponding to a certain image category, the first model determines the corresponding image category as the image category of the input first image. For example, the third probability value output by the first model or the second model is 0.22, and the difference between the third probability value and the fourth probability value corresponding to the invoice category is the smallest, so the category of the input third image is identified as the invoice category.

[0124] The embodiment of the present invention uses the trained first model or the second model to perform image recognition, which can achieve the distinction of easily confused category images and enhance the accuracy of image recognition.

[0125] refer to Figure 7 , Figure 7 Schematic diagram of an image processing process provided by an application embodiment of the present invention. The image processing process includes:

[0126] First, we obtain office-sensitive images from information sources such as the enterprise and the Internet, and partially block the image content of the office-sensitive images to obtain blocked images. We label the blocked images and the original public-sensitive images as training data. We then use the training data to train the MobileNet_V3 model under the pytorch framework. By adding blocked images as training data, we can enhance the model's ability to recognize images of each category and distinguish images of easily confused categories.

[0127] Convert the trained MobileNet_V3 model to ONNX format, and then convert the ONNX format MobileNet_V3 model to a Paddle model through x2paddle. This can increase the model's image reasoning speed and improve the model's image recognition efficiency.

[0128] Finally, the converted Paddle model is used for image classification to achieve high accuracy and high efficiency image recognition.

[0129] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0130] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0131] It should be noted that the technical solutions described in the embodiments of the present invention can be arbitrarily combined without conflict.

[0132] In addition, in the embodiments of the present invention, "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0133] refer to Figure 8 , Figure 8 is a schematic diagram of a model training device provided by an embodiment of the present invention, such as Figure 8 As shown, the device includes: an occlusion module and a training module.

[0134] a masking module, configured to mask part of the image content of each of at least two first images to obtain at least two second images; the at least two first images correspond to at least two image categories, and the similarity between first images of different categories in the at least two first images is greater than a set value;

[0135] A training module is used to train a first model based on the at least two first images and the at least two second images; the first model obtained by training is used to identify the image category of an input image.

[0136] In one embodiment, when the shielding module shields a partial image area of ​​each of the at least two first images, it is used to:

[0137] selecting a set area in each of the at least two first images;

[0138] The set area is filled with pixel blocks of a set color.

[0139] In one embodiment, when the shielding module shields a partial image area of ​​each of the at least two first images, it is used to:

[0140] A partial image area is randomly selected in each of the at least two first images for blocking, so as to obtain at least two second images corresponding to each of the at least two first images.

[0141] In one embodiment, when training the first model based on the at least two first images and the at least two second images, the training module is used to:

[0142] Inputting the at least two first images and the at least two second images into the first model, obtaining a first probability value of the first image and a second probability value of the corresponding second image output by the first model; the first probability value represents the image category of the corresponding first image; and the second probability value represents the image category of the corresponding second image;

[0143] Determine, based on the first probability value and the second probability value, a loss value of a set loss function corresponding to each of the at least two image categories;

[0144] The weight parameters of the first model are updated based on the loss value.

[0145] In one embodiment, the training module updates the weight parameters of the first model based on the loss value, for:

[0146] Performing weighted calculation on the loss value of the set loss function corresponding to each image category of the at least two image categories to obtain a weighted value;

[0147] A weight parameter of the first model is updated based on the weighted value.

[0148] In one embodiment, the device further comprises:

[0149] A deletion module is used to delete the number of network layers and / or the number of network channels of the set neural network model to obtain the first model.

[0150] In one embodiment, the device further comprises:

[0151] A conversion module, used to convert the first model into a second model; the second model is used to identify the image category of the at least one third image; the first model represents a PyTorch model; the second model represents a PaddlePaddle model.

[0152] In one embodiment, when the conversion module converts the first model into the second model, it is used to:

[0153] Converting the first model into an open neural network exchange format;

[0154] The first model in the open neural network exchange format is converted into the second model through a predetermined model conversion tool.

[0155] refer to Fig. 9 , Fig. 9 is a schematic diagram of a model training device provided by an embodiment of the present invention, such as Fig. 9 As shown, the device includes: an input module, a determination module and an identification module.

[0156] An input module, used for inputting a third image into the first model or the second model to obtain a third probability value output by the first model or the second model; the third probability value represents the image category of the third image;

[0157] a determination module, configured to determine a difference between the third probability value and a fourth probability value corresponding to each of at least two categories;

[0158] A recognition module is used to determine the image category of the third image based on the difference.

[0159] In actual application, the input module, determination module and identification module can be implemented by a processor in an electronic device, such as a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU) or a programmable gate array (FPGA).

[0160] It should be noted that: the model training device provided in the above embodiment only uses the division of the above modules as an example when performing model training. In practical applications, the above processing can be assigned to different modules as needed, that is, the internal structure of the device can be divided into different modules to complete all or part of the processing described above. In addition, the model training device provided in the above embodiment and the model training method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0161] Based on the hardware implementation of the above program modules and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides an electronic device. Fig.10 Schematic diagram of the hardware structure of the electronic device of the present application embodiment. Fig.10 As shown, the electronic equipment includes:

[0162] Communication interface, which can exchange information with other devices such as network equipment;

[0163] The processor is connected to the communication interface to realize information exchange with other devices, and is used to execute the method provided by one or more technical solutions of the electronic device side when running the computer program. The computer program is stored in the memory.

[0164] Of course, in actual applications, the various components in the electronic device are coupled together through the bus system. It is understandable that the bus system is used to achieve connection and communication between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig.10 In the specification, various buses are labeled as bus systems.

[0165] The memory in the embodiment of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program used to operate on the electronic device.

[0166] It can be understood that the memory can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0167] The method disclosed in the above embodiment of the present application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the program in the memory and completes the steps of the above method in combination with its hardware.

[0168] Optionally, when the processor executes the program, the corresponding processes implemented by the electronic device in each method of the embodiments of the present application are implemented, which will not be described in detail here for the sake of brevity.

[0169] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, including a first memory storing a computer program, and the computer program can be executed by a processor of an electronic device to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0170] In the several embodiments provided in the present application, it should be understood that the disclosed devices, electronic devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0171] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0172] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0173] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.

[0174] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0175] It should be noted that the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0176] In addition, in the examples of the present application, "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0177] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model training method, characterized in that: The method comprises: Obstructing part of the image content of each of at least two first images to obtain at least two second images; the at least two first images correspond to at least two image categories, and the similarity between first images of different categories in the at least two first images is greater than a set value; each of the first images and the second image obtained by obstructing the first image are of the same category; Training a first model based on the at least two first images and the at least two second images; the first model obtained by training is used to identify the image category of the input image; The training of the first model based on the at least two first images and the at least two second images comprises: Inputting the at least two first images and the at least two second images into the first model, obtaining a first probability value of the first image and a second probability value of the corresponding second image output by the first model; the first probability value represents the image category of the corresponding first image; and the second probability value represents the image category of the corresponding second image; Determine, based on the first probability value and the second probability value, a loss value of a set loss function corresponding to each of the at least two image categories; The weight parameters of the first model are updated based on the loss value.

2. The method according to claim 1, characterized in that The step of blocking a partial image area of ​​each of the at least two first images comprises: selecting a set area in each of the at least two first images; The set area is filled with pixel blocks of a set color.

3. The method according to claim 1, characterized in that The step of blocking a partial image area of ​​each of the at least two first images comprises: A partial image area is randomly selected in each of the at least two first images for blocking, so as to obtain at least two second images corresponding to each of the at least two first images.

4. The method according to claim 1, characterized in that: The updating of the weight parameters of the first model based on the loss value comprises: Performing weighted calculation on the loss value of the set loss function corresponding to each image category of the at least two image categories to obtain a weighted value; A weight parameter of the first model is updated based on the weighted value.

5. The method according to claim 1, characterized in that Before training the first model based on the at least two first images and the at least two second images, the method further includes: Delete the set number of network layers and / or number of network channels of the neural network model to obtain the first model.

6. The method according to claim 1, characterized in that After the first model is obtained by training based on the at least two first images and the at least two second images, the method further includes: The first model is converted into a second model; the second model is used to identify the image category of at least one third image; the first model represents the PyTorch model; the second model represents the PaddlePaddle model.

7. The method according to claim 6, characterized in that The converting the first model into a second model comprises: Converting the first model into an open neural network exchange format; The first model in the open neural network exchange format is converted into the second model through a predetermined model conversion tool.

8. An image recognition method, characterized in that: Image recognition is performed based on the first model according to any one of claims 1 to 7 or the second model according to any one of claims 6 to 7, the method comprising: Inputting a third image into the first model or the second model to obtain a third probability value output by the first model or the second model; the third probability value represents an image category of the third image; determining a difference between the third probability value and a fourth probability value corresponding to each of the at least two categories; An image category of the third image is determined based on the difference value.

9. A model training device, characterized in that: include: A masking module, used to mask part of the image content of each of the at least two first images to obtain at least two second images; The at least two first images correspond to at least two image categories, and the similarity between first images of different categories in the at least two first images is greater than a set value; each of the first images and a second image obtained by shielding the first image are of the same category; A training module, used for training a first model based on the at least two first images and the at least two second images; the first model obtained by training is used for identifying the image category of an input image; The training module is specifically used for: Inputting the at least two first images and the at least two second images into the first model, obtaining a first probability value of the first image and a second probability value of the corresponding second image output by the first model; the first probability value represents the image category of the corresponding first image; and the second probability value represents the image category of the corresponding second image; Determine, based on the first probability value and the second probability value, a loss value of a set loss function corresponding to each of the at least two image categories; The weight parameters of the first model are updated based on the loss value.

10. An image recognition device, characterized in that: Performing image recognition based on the first model according to any one of claims 1 to 7 or the second model according to any one of claims 6 to 7 comprises: An input module, used for inputting a third image into the first model or the second model to obtain a third probability value output by the first model or the second model; the third probability value represents the image category of the third image; a determination module, configured to determine a difference between the third probability value and a fourth probability value corresponding to each of at least two categories; A recognition module is used to determine the image category of the third image based on the difference.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the model training method according to any one of claims 1 to 7 or the image recognition method according to claim 8.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor executes the model training method according to any one of claims 1 to 7 or the image recognition method according to claim 8.

Images