Image recognition method and device, computer device and storage medium

Abstract

The application relates to an image recognition method and device, computer equipment and a storage medium, and belongs to the technical field of image recognition. The method comprises the following steps: adding disturbance in an original sample image to obtain an initial adversarial sample image; inputting the initial adversarial sample image into a semantic segmentation network model, and using a feature deception algorithm based on cosine similarity to attack the semantic segmentation network model to obtain an adversarial sample image; using a training sample set to perform optimization training on the semantic segmentation network model to obtain a semantic segmentation network optimization model, wherein the training sample set is a set in which the adversarial sample image is added in a specified proportion; and using the semantic segmentation network optimization model to recognize a to-be-recognized image. The semantic segmentation network optimization model obtained by using the method has strong anti-interference capability and good recognition effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and particularly relates to an image recognition method and device, a computer device and a storage medium. BACKGROUND

[0002] With the development of deep learning technology, artificial intelligence technology based on deep learning has a profound impact on various fields of human society, and image recognition is one of the fields.

[0003] In the prior art, a trained semantic segmentation network model is usually used to recognize pictures.

[0004] However, research has found that a small perturbation of a normal image, even though the change brought by the perturbation to the picture is difficult to perceive by the human eye, can cause the trained semantic segmentation network model in the prior art to produce incorrect recognition. For example, for a cat image, adding a small perturbation will cause the trained semantic segmentation network model in the prior art to recognize the cat image as a dog or other animals. That is, the trained semantic segmentation network model in the prior art has low recognition accuracy in the case of a small perturbation. SUMMARY

[0005] Therefore, it is necessary to provide an image recognition method, device, computer device and storage medium to solve the above technical problems.

[0006] In a first aspect, an image recognition method is provided, which includes:

[0007] adding a perturbation to an original sample image to obtain an initial adversarial sample image; inputting the initial adversarial sample image into a semantic segmentation network model and using a feature deception algorithm based on cosine similarity to attack the semantic segmentation network model to obtain an adversarial sample image; optimizing and training the semantic segmentation network model using a training sample set to obtain a semantic segmentation network optimization model, wherein the training sample set is a set in which the adversarial sample image is added according to a specified proportion; and recognizing a to-be-recognized image using the semantic segmentation network optimization model.

[0008] In one embodiment, the loss function in the semantic segmentation network model aims to maximize the included angle between a first vector and a second vector output by each layer of the semantic segmentation network model, the first vector being a vector corresponding to the original sample image output by each layer of the semantic segmentation network model, and the second vector being a vector corresponding to the initial adversarial sample image output by each layer of the semantic segmentation network model.

[0009] In one embodiment, the loss function in the semantic segmentation network model is:

[0010]

[0011]

[0012] Such than||δ|| ∞ <ξ

[0013] wherein, K is the range of the attacked layer, θ is the maximum angle between the original output of each layer and the output of the attack, ξ is an infinitesimal, l i (x) is the output of the i-th layer of the semantic segmentation network model, l i (x+δ) is the output of the i-th layer of the semantic segmentation network model in the case of adding perturbation to the data matrix of the original sample image, ||l i (x+δ)||2 is the second norm of the matrix formed after adding perturbation to the original sample image data matrix, ||l i (x)||2 is the second norm of the original sample image data matrix, and δ is the adversarial perturbation.

[0014] In one embodiment, the initial adversarial sample image is input into the semantic segmentation network model, and the feature deception algorithm based on cosine similarity is used to attack the semantic segmentation network model to obtain an adversarial sample image, comprising: inputting the initial adversarial sample image into the semantic segmentation network model, and using the feature deception algorithm based on cosine similarity to attack the semantic segmentation network model, calculating the saturation of the output features of each convolutional layer in the semantic segmentation network model and the saturation difference of the output features of adjacent two convolutional layers, if the saturation of a specified convolutional layer is greater than a first preset threshold and the saturation difference of the output features of adjacent convolutional layers is less than a second preset threshold, the candidate adversarial sample image output by the specified convolutional layer is saved; the perturbation in the candidate adversarial sample image is compressed according to a preset ratio, the compressed candidate adversarial sample image is input into the semantic segmentation network model, and the feature deception algorithm based on cosine similarity is used to attack the semantic segmentation network model, and iterative calculation is performed until the number of iterations reaches a specified number, and the candidate adversarial sample image corresponding to each iteration is saved; one of the candidate adversarial sample images is selected as the adversarial sample image.

[0015] In one embodiment, the selection of one of the candidate adversarial sample images as the adversarial sample image comprises: sequentially adding the perturbation in each candidate adversarial sample image to a test sample image, sequentially inputting the test sample images with different perturbations into the semantic segmentation network model, calculating the deception rate of the perturbation in each candidate adversarial sample image, and selecting the candidate adversarial sample image corresponding to the perturbation with the highest deception rate as the adversarial sample image.

[0016] In one of the embodiments, the selecting one of the candidate adversarial sample images as the adversarial sample image further comprises: adding the perturbation in the obtained candidate adversarial sample image to the test sample image after each iteration to obtain the candidate adversarial sample image, inputting the test sample image with the added perturbation into the semantic segmentation network model, and calculating the deception rate of the perturbation in the candidate adversarial sample image; and if the deception rate of the perturbation in the candidate adversarial sample image obtained by the continuous specified number of iterations is less than the deception rate of the perturbation in the candidate adversarial sample image obtained by the previous iteration, stopping the iteration and selecting the candidate adversarial sample image corresponding to the perturbation with the highest deception rate as the adversarial sample image.

[0017] In one of the embodiments, the calculating the deception rate of the perturbation in the candidate adversarial sample image comprises: using an evaluation index of semantic segmentation to calculate the deception rate of the perturbation in the candidate adversarial sample image.

[0018] In a second aspect, an image recognition device is provided, which comprises: a first obtaining module configured to add a perturbation to an original sample image to obtain an initial adversarial sample image; a second obtaining module configured to input the initial adversarial sample image into a semantic segmentation network model and attack the semantic segmentation network model using a feature deception algorithm based on cosine similarity to obtain an adversarial sample image; a third obtaining module configured to optimize and train the semantic segmentation network model using a training sample set to obtain a semantic segmentation network optimization model, wherein the training sample set is a set in which the adversarial sample image is added in a specified proportion; and a recognition module configured to recognize a to-be-recognized image using the semantic segmentation network optimization model.

[0019] In one of the embodiments, the loss function in the semantic segmentation network model aims to maximize the included angle between a first vector output by each layer of the semantic segmentation network model and a second vector output by each layer of the semantic segmentation network model, the first vector being a vector corresponding to the original sample image, and the second vector being a vector corresponding to the initial adversarial sample image.

[0020] In one of the embodiments, the loss function in the semantic segmentation network model in the second obtaining module is:

[0021]

[0022]

[0023] Such than||δ|| ∞ <ξ

[0024] wherein, K is the range of the attacked layer number, θ is the maximum angle between the original output of each layer and the adversarial output, ξ is an infinitesimal, and l i (x) is the output of the i-th layer of the semantic segmentation network model, l i (x+δ) is the output of the i-th layer of the semantic segmentation network model in the case of adding perturbation to the data matrix of the original sample image, ||l i (x+δ)||2 is the second norm of the matrix formed after adding perturbation to the original sample image data matrix, ||l i (x)||2 is the second norm of the original sample image data matrix, and δ is the adversarial perturbation.

[0025] In one of the embodiments, the second acquisition module is specifically configured to: input the initial adversarial sample image into the semantic segmentation network model, and attack the semantic segmentation network model using a feature deception algorithm based on cosine similarity, calculate the saturation of the output features of each convolutional layer in the semantic segmentation network model and the saturation difference of the output features of adjacent two convolutional layers, if the saturation of a specified convolutional layer is greater than a first preset threshold and the saturation difference of the output features of the adjacent convolutional layer is less than a second preset threshold, save the candidate adversarial sample image output by the specified convolutional layer; compress the perturbation in the candidate adversarial sample image according to a preset ratio, input the compressed candidate adversarial sample image into the semantic segmentation network model, and attack the semantic segmentation network model using the feature deception algorithm based on cosine similarity, and perform iterative calculation until the number of iterations reaches a specified number, save the candidate adversarial sample image corresponding to each iteration calculation; select one of the candidate adversarial sample images as the adversarial sample image.

[0026] In one of the embodiments, the second acquisition module is specifically configured to: add the perturbation in each of the candidate adversarial sample images to the test sample image in sequence, input the test sample image with different perturbations added to the semantic segmentation network model in sequence, calculate the deception rate of the perturbation in each of the candidate adversarial sample images, and select the candidate adversarial sample image corresponding to the perturbation with the highest deception rate as the adversarial sample image.

[0027] In one of the embodiments, the second obtaining module is specifically configured to: after obtaining the candidate adversarial sample image in each iteration calculation, add the perturbation in the obtained candidate adversarial sample image to the test sample image, input the test sample image with the added perturbation to the semantic segmentation network model, and calculate the deception rate of the perturbation in the candidate adversarial sample image; if the deception rates of the perturbations in the candidate adversarial sample images obtained in the continuous specified number of iteration calculations are all less than the deception rate of the perturbation in the candidate adversarial sample image obtained in the previous iteration calculation, stop the iteration calculation, and select the candidate adversarial sample image corresponding to the perturbation with the highest deception rate as the adversarial sample image.

[0028] In one of the embodiments, the second obtaining module is specifically configured to: calculate the deception rate of the perturbation in the candidate adversarial sample image using the evaluation index of semantic segmentation.

[0029] In a third aspect, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the method of any one of the first aspect when executing the computer program.

[0030] In a fourth aspect, a computer readable storage medium is provided, storing a computer program, and the computer program implementing the steps of the method of any one of the first aspect when executed by a processor.

[0031] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects:

[0032] In the embodiments of the present application, first, a perturbation is added to an original sample image to obtain an initial adversarial sample image; second, the initial adversarial sample image is input to a semantic segmentation network model, and a feature deception algorithm based on cosine similarity is used to attack the semantic segmentation network model to obtain an adversarial sample image; third, the semantic segmentation network model is optimized and trained using a training sample set to which the adversarial sample image is added in a specified proportion to obtain a semantic segmentation network optimization model; and finally, the semantic segmentation network optimization model is used to identify a to-be-identified image. Since the feature deception algorithm based on cosine similarity has a high deception rate, the adversarial sample image generated by the algorithm is added to the training sample set, and the semantic segmentation network optimization model trained based on the training sample set has strong anti-interference ability and good identification effect. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 A schematic diagram of an implementation environment provided by the embodiments of the present application;

[0034] Figure 2 A flowchart of an image identification method provided by the embodiments of the present application;

[0035] Figure 3 A flowchart of a technical means for obtaining an adversarial sample image provided by an embodiment of the present application is provided.

[0036] Figure 4 A flowchart of another image recognition method provided by an embodiment of the present application is provided.

[0037] Figure 5 A block diagram of an image recognition device provided by an embodiment of the present application is provided.

[0038] Figure 6 An internal structure diagram of a computer device provided by an embodiment of the present application is provided. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0040] With the development of deep learning technology, artificial intelligence technology based on deep learning has a profound impact on various fields of human society, and image recognition is one of the fields.

[0041] In the prior art, a trained semantic segmentation network model is usually used to recognize pictures.

[0042] However, research has found that a small perturbation to a normal image, even though the changes brought by these perturbations are difficult to detect by the human eye, can cause the trained semantic segmentation network model in the prior art to produce incorrect recognition. For example, for a cat image, adding a small perturbation will cause the trained semantic segmentation network model in the prior art to recognize the cat image as a dog or other animals. That is, the accuracy of image recognition of the trained semantic segmentation network model in the prior art is not high when facing a small perturbation.

[0043] Therefore, embodiments of the present application provide an image recognition method, device, computer device and storage medium, which improves the accuracy of image recognition of the semantic segmentation network model by optimizing the semantic segmentation network model in the prior art.

[0044] Please refer to Figure 1 which shows a schematic diagram of an implementation environment involved in the image recognition method provided by an embodiment of the present application. As shown in Figure 1As shown, the execution subject of the image recognition method provided in the embodiments of the present application can be a computer device, or a computer device cluster composed of multiple computer devices. Different computer devices can communicate through wired or wireless means. Wireless means can be achieved through WIFI, operator network, NFC (near field communication) or other technologies.

[0045] Please refer to Figure 2 , which shows a flowchart of an image recognition method provided in the embodiments of the present application. The image recognition method can be applied to Figure 1 the computer device as shown. As Figure 2 shown, the image recognition method can include the following steps:

[0046] Step 201, the computer device adds perturbation to the original sample image to obtain an initial adversarial sample image.

[0047] In the embodiments of the present application, the computer device can add perturbation to the original sample image based on various algorithms, wherein the various algorithms can be selected according to the actual business scenario, and the embodiments of the present application do not make specific limitations. Wherein the original sample image can be understood as selecting a certain number of original images as samples from the handling process of the business scenario based on the needs of the actual business scenario. The sample can be used for training or testing of a semantic segmentation network model, etc. Optionally, the number, size and type of the original sample image can be selected based on actual needs, and the embodiments of the present application do not make specific limitations.

[0048] Step 202, the computer device inputs the initial adversarial sample image into the semantic segmentation network model, and uses a feature deception algorithm based on cosine similarity to attack the semantic segmentation network model to obtain an adversarial sample image.

[0049] In the optional embodiments of the present application, the output of the semantic segmentation network is a feature vector, and each value on the vector represents the probability of the corresponding class. The class corresponding to the value with the largest probability represents the final result of recognition. We assume that the feature vector output by a semantic segmentation network is x(x1,x2,...,x n ), if there is a vector y(y1,y2,...y n ) that is equal in length and orthogonal to the feature vector, then it satisfies the following formula:

[0050] x·y=0

[0051] Specifically, it satisfies x1y1+x2y2+...x n y n= 0, since the values on the classification vector cannot be all zeros and cannot have negative values, so the all-zero vector does not meet the requirements. In this case, it is obvious that the equality cannot be achieved, that is, the above formula becomes:

[0052] x·y * >0

[0053] where y * is a characteristic vector of x with the same length but not orthogonal, and we need to obtain y * that minimizes the value on the left side of the above formula. According to the conclusion of the ordering inequality, for two arrays x and y, the product of the two arrays is maximum if and only if the maximum number in the array x matches the maximum number in the array y (multiplied), the second maximum number matches the second maximum number, and so on until the minimum number matches the minimum number. Conversely, if the matching is in the opposite order, the resulting product is minimum. For the feature vector output by the semantic segmentation network, that is, the class corresponding to the maximum probability in the vector x corresponds to the class to which the minimum probability in the vector y * belongs, which causes the classification error of the semantic segmentation network.

[0054] Based on the above principle, in the embodiments of the present application, the target of the loss function in the semantic segmentation network model is to maximize the angle between the first vector and the second vector output by each layer of the semantic segmentation network model. The first vector is the vector corresponding to the original image output by each layer of the semantic segmentation network model, and the second vector is the vector corresponding to the initial adversarial sample image output by each layer of the semantic segmentation network model. Optionally, the first vector is obtained by inputting the original sample image into the semantic segmentation network model, and the output of the semantic segmentation network model is the first vector; the second vector is obtained by inputting the initial adversarial sample image into the semantic segmentation network model, and the output of the semantic segmentation network model is the second vector. After obtaining the first vector and the second vector, the first vector and the second vector are input into the loss function to obtain the loss value, and the adversarial perturbation is adjusted according to the loss value until the loss function converges based on the adjusted adversarial perturbation.

[0055] Based on this, semantic segmentation can be regarded as pixel-level target recognition, and the loss function of the semantic segmentation network model can be designed as:

[0056]

[0057]

[0058] Such that ||δ|| ∞ <ξ

[0059] where K is the range of the number of layers attacked, θ is the maximum angle between the original output and the adversarial output of each layer, and ξ is an infinitesimal.i (x) is the output of the i-th layer of the semantic segmentation network model, l i (x+δ) is the output of the i-th layer of the semantic segmentation network model in the case of adding a perturbation to the data matrix of the original sample image, ||l i (x+δ)||2 is the second norm of the matrix formed after adding the perturbation to the original sample image data matrix, ||l i (x)||2 is the second norm of the original sample image data matrix, and δ is the adversarial perturbation.

[0060] It should be noted that l i (x) can be understood as the first vector output by each layer of the semantic segmentation network model as described above, l i (x+δ) can be understood as the second vector output by each layer of the semantic segmentation network model as described above.

[0061] The meaning of the loss function in the above formula can be: for the adversarial perturbation δ, maximize the angle θ between the original output of each layer and the adversarial output to minimize the cosine similarity, so as to maximize the directional difference between the perturbed result and the original result. In order to prevent the loss value from being too small, the present embodiment suppresses the loss value from tending to 0 in the later training by subtracting the cosine value from 1, and takes the logarithm of the result to prevent the loss value from being too large, and continuously changes the loss function in the present embodiment based on the characteristics of the cosine similarity.

[0062] The feature spoofing algorithm based on cosine similarity in the present embodiment has a higher spoofing rate than the GD-UAP algorithm in the prior art, where the GD-UAP algorithm is the current international mainstream attack algorithm, including three types of perturbations: fast feature spoofing, statistical information-based feature spoofing, and data-driven feature spoofing.

[0063] Step 203, the computer device optimizes and trains the semantic segmentation network model using the training sample set to obtain a semantic segmentation network optimization model.

[0064] The training sample set is a set in which adversarial sample images are added according to a specified proportion; the training sample set includes adversarial sample images and general sample images; and the specified proportion is a proportion that makes the obtained semantic segmentation network optimization model optimal.

[0065] In the embodiment of the present application, because the recognition effect and anti-interference ability of the semantic segmentation network optimization model trained by the training sample set change with the change of the proportion of the adversarial sample image in the training sample set, in order to make the semantic segmentation network optimization model trained by the training sample set have the best recognition effect and anti-interference ability, it is necessary to determine the best proportion of the adversarial sample image in the training sample set, which is the specified proportion. Optionally, the specified proportion can be a best proportion value obtained according to a large amount of experimental data, or it can be a best proportion range. Whether it is a best proportion value or a best proportion range, it is necessary to ensure that the semantic segmentation network optimization model trained by the training sample set containing the specified proportion of adversarial sample images is optimal, that is, the semantic segmentation network optimization model has the best recognition effect and anti-interference ability.

[0066] In the prior art, the adversarial sample image is usually obtained by attacking the semantic segmentation network model based on the GD-UAP algorithm, and the training sample set with the adversarial sample image added according to the specified proportion is used to optimize and train the semantic segmentation network model, thereby obtaining the semantic segmentation network optimization model. In the embodiment of the present application, the adversarial sample image is obtained by attacking the semantic segmentation network model based on the cosine similarity feature deception algorithm, and the training sample set with the adversarial sample image added according to the specified proportion is used to optimize and train the semantic segmentation network model, thereby obtaining the semantic segmentation network optimization model. Since the cosine similarity feature deception algorithm has a higher deception rate than the GD-UAP algorithm in the prior art, the semantic segmentation network trained in the embodiment of the present application has better anti-interference ability and stronger recognition ability.

[0067] Step 204, the computer device identifies the to-be-identified image by using the semantic segmentation network optimization model.

[0068] In the embodiment of the present application, first, a disturbance is added to the original sample image to obtain an initial adversarial sample image; second, the initial adversarial sample image is input into the semantic segmentation network model, and the semantic segmentation network model is attacked using the cosine similarity feature deception algorithm to obtain an adversarial sample image; third, the training sample set with the adversarial sample image added according to the specified proportion is used to optimize and train the semantic segmentation network model, thereby obtaining the semantic segmentation network optimization model; and finally, the semantic segmentation network optimization model is used to identify the to-be-identified image. Since the cosine similarity feature deception algorithm has a high deception rate, the adversarial sample image generated by the algorithm is added to the training sample set, and the semantic segmentation network optimization model trained based on the training sample set has strong anti-interference ability and good recognition effect.

[0069] Please refer to Figure 3 which shows a technical means for obtaining an adversarial sample image provided by an embodiment of the present application. As shown in Figure 3 , the technical means for obtaining an adversarial sample image includes the following steps:

[0070] Step 301, the computer device inputs an initial adversarial sample image into a semantic segmentation network model, and uses a feature deception algorithm based on cosine similarity to attack the semantic segmentation network model, calculates the saturation of the output features of each convolutional layer in the semantic segmentation network model and the saturation difference of the output features of two adjacent convolutional layers, and if the saturation corresponding to a specified convolutional layer is greater than a first preset threshold and the saturation difference between the specified convolutional layer and an adjacent convolutional layer is less than a second preset threshold, the candidate adversarial sample image output by the specified convolutional layer is saved.

[0071] Since the convolutional layer is responsible for feature extraction, the attack algorithm in the present application can attack all convolutional layers. The feature deception algorithm based on cosine similarity in the present application is a multi-step iterative algorithm, but it does not have obvious directionality like deepfool, and needs a large number of iterations to continuously approach the optimal solution. The present application considers avoiding local optimal solution, and sets Sat t variable represents the saturation of the perturbed image (the ratio of pixels reaching the upper and lower limits of attack intensity) and Sat t the difference between the two training SatC t If Sat t is too high, SatC t is too small, then it can be considered that the training has reached a "saturation state".

[0072] In the present application, the initial adversarial sample image is input into the semantic segmentation network model, and the feature deception algorithm based on cosine similarity is used to attack the semantic segmentation network model, that is, to attack all convolutional layers in the semantic segmentation network model, and to calculate the saturation of the output features of each convolutional layer in the semantic segmentation network model and the saturation difference of the output features of two adjacent convolutional layers. If the saturation corresponding to a certain convolutional layer is greater than a first preset threshold, and the saturation difference between the convolutional layer and an adjacent convolutional layer is less than a second preset threshold, the candidate adversarial sample image output by the convolutional layer is saved. The size of the first preset threshold and the second preset threshold can be set based on actual needs, and can be the same value or different values, which are not limited in the present application.

[0073] In the embodiment of the present application, the perturbation in the candidate adversarial sample image is compressed according to a preset ratio. Optionally, if the preset ratio is 0.5, that is, the compression rate is 0.5, the perturbation can be reduced by half. The compressed candidate adversarial sample image is input into the semantic segmentation network model, and the above steps are repeated to calculate the saturation and saturation difference value of each convolution layer of the semantic segmentation network model, obtain the candidate adversarial sample image corresponding to a new round of iteration calculation until the iteration number reaches a specified number, and save the candidate adversarial sample image corresponding to each iteration calculation.

[0074] In the embodiment of the present application, the perturbation in the candidate adversarial sample image is compressed according to a preset ratio. Optionally, if the preset ratio is 0.5, that is, the compression rate is 0.5, the perturbation can be reduced by half. The compressed candidate adversarial sample image is input into the semantic segmentation network model, and the above steps are repeated to calculate the saturation and saturation difference value of each convolution layer of the semantic segmentation network model, obtain the candidate adversarial sample image corresponding to a new round of iteration calculation until the iteration number reaches a specified number, and save the candidate adversarial sample image corresponding to each iteration calculation.

[0075] Step 303, the computer device selects one from each candidate adversarial sample image as an adversarial sample image.

[0076] In the optional embodiment of the present application, the perturbation in each candidate adversarial sample image is added to the test sample image in turn, and the test sample image to which different perturbations are added is input into the semantic segmentation network model in turn, the deception rate of the perturbation in each candidate adversarial sample image is calculated, and the candidate adversarial sample image corresponding to the perturbation with the highest deception rate is selected as the adversarial sample image. The test sample image can be understood as a test sample set used to test the accuracy of the semantic segmentation network model, and there can be multiple test sample images. In the embodiment of the present application, the candidate adversarial sample image corresponding to the perturbation with the highest deception rate is selected as the adversarial sample image. The adversarial sample image can cause the greatest interference to the semantic segmentation network model, thereby laying a data foundation for the subsequent optimization training of the semantic segmentation network model.

[0077] In the embodiment of the present application, the optimal perturbation in the convolution layer in each round is obtained through iterative calculation to avoid local solutions, thereby laying a data foundation for the subsequent optimization training of the semantic segmentation network model and improving the accuracy and stability of image recognition.

[0078] In another optional embodiment of the present application, after obtaining a candidate adversarial sample image in each iteration calculation, the perturbation in the obtained candidate adversarial sample image is added to the test sample image, the test sample image to which the perturbation is added is input into the semantic segmentation network model, and the deception rate of the perturbation in the candidate adversarial sample image is calculated; if the deception rates of the perturbations of the candidate adversarial sample images obtained in the continuous specified number of iteration calculations are less than the deception rate of the perturbation of the candidate adversarial sample image obtained in the previous iteration calculation, the iteration calculation is stopped, and the candidate adversarial sample image corresponding to the perturbation with the highest deception rate is selected as the adversarial sample image.

[0079] Optionally, the perturbation of the candidate adversarial sample image obtained after each iteration calculation can be tested. For example, the perturbation in the obtained candidate adversarial sample image is added to the test sample image, the test sample image to which the perturbation is added is input into the semantic segmentation network model, and the deception rate of the perturbation in the candidate adversarial sample image is calculated. If the perturbation obtained in the new round of iteration calculation can achieve a higher deception rate on the validation set, it is considered that a better result is obtained, and the candidate adversarial sample image at this time is saved to overwrite the previous result. If the deception rates of the perturbations of the candidate adversarial sample images obtained in the continuous specified number of iteration calculations are less than the deception rate of the perturbation of the candidate adversarial sample image obtained in the previous iteration calculation, the iteration calculation can be stopped, and the latest saved candidate adversarial sample image is taken as the adversarial sample image or the candidate adversarial sample image with the highest deception rate is taken as the adversarial sample image. If there is no case in which the deception rates of the perturbations of the candidate adversarial sample images obtained in the continuous specified number of iteration calculations are less than the deception rate of the perturbation of the candidate adversarial sample image obtained in the previous iteration calculation in the iteration process, the loop iteration is continued until the number of iterations reaches the specified number.

[0080] In the embodiment of the present application, the deception rate of the perturbation in the candidate adversarial sample image can be calculated using the evaluation index of semantic segmentation. The intersection-over-union (IOU) is a commonly used evaluation index for measuring the accuracy of semantic segmentation, which can be used to calculate the deception rate of the perturbation. The IOU is simply the intersection rate of the target window obtained by the model and the labeled window, and the formula for calculating the IOU is as follows:

[0081]

[0082] wherein TP, FP and FN represent the statistical data of true positive, false positive and false negative respectively, the range of IOU is 0-100%, the higher the value of IOU is, the more accurate the segmentation result is, the lower the value of IOU is, the worse the segmentation result is, and the worse the segmentation result is, the better the effect of the perturbation attack is.

[0083] In the embodiment of the present application, the computer device timely stops the iterative calculation in the case that the deception rates of the candidate adversarial sample images obtained for multiple times in succession do not exceed the deception rate calculated in the previous iteration, improves the data processing efficiency, and also obtains a relatively good result.

[0084] From the above description, compared with the semantic segmentation network in the prior art, the semantic segmentation network optimized model trained in the embodiment of the present application has the following two advantages:

[0085] 1. The classification accuracy of the semantic segmentation network model trained by the training sample set composed of adversarial sample images and ordinary sample images is higher. As follows:

[0086] Model Baseline FCN-AlexNet 46.75 DL-VGG16 59.20 Adv-FCN-AlexNet 53.40 Adv-DL-VGG16 68.22

[0087] Among them, FCN-AlexNet and DL-VGG16 represent the semantic segmentation network model not trained by adversarial samples, Adv-FCN-AlexNet and Adv-DL-VGG16 represent the semantic segmentation network model trained by adversarial samples, and the values in the table are the values of IOU.

[0088] From the above table, it can be seen that the IOU value of the Adv-FCN-AlexNet semantic segmentation network model is 53.40, the IOU value of the FCN-AlexNe semantic segmentation network model is 46.75, the IOU value of the Adv-DL-VGG16 semantic segmentation network model is 68.22, and the IOU value of the DL-VGG16 semantic segmentation network model is 59.40. Therefore, the IOU value of Adv-FCN-AlexNet is 6.65 higher than that of FCN-AlexNe, and the IOU value of Adv-DL-VGG16 is 9.02 higher than that of DL-VGG16. That is, the classification accuracy of the semantic segmentation network model trained by adversarial samples is about 5% higher than that of the semantic segmentation network model not trained by adversarial samples.

[0089] 2. The anti-interference ability of the semantic segmentation network trained based on the cosine similarity feature deception is stronger. As follows:

[0090]

[0091]

[0092] No data is a no-data fast deception algorithm of the GD-UAP, All Data is a feature deception attack algorithm based on cosine similarity, simulates white-box attack and black-box attack, wherein FCN-AlexNet and DL-VGG16 represent the semantic segmentation network model trained by using the no-data fast deception algorithm, Adv-FCN-AlexNet and Adv-DL-VGG16 represent the semantic segmentation network model trained by using the feature deception attack algorithm based on cosine similarity, and the numerical value in the table is the value of IOU.

[0093] As can be seen from the above table, the IOU value of the semantic segmentation network trained based on the feature deception based on cosine similarity is higher than the IOU value of the semantic segmentation network model trained by using the no-data fast deception algorithm, and therefore, the anti-interference ability of the semantic segmentation network trained based on the feature deception based on cosine similarity is stronger.

[0094] See Figure 4 , which shows a flowchart of an image recognition method provided by an embodiment of the present application, which can be applied to Figure 1 the computer device shown. As Figure 4 shown, the image recognition method can include the following steps:

[0095] Step 401, the computer device adds perturbation in the original sample image to obtain an initial adversarial sample image.

[0096] In the embodiments of the present application, the computer device can add perturbation in the original sample image based on various algorithms, wherein the various algorithms can be selected according to the actual business scenario, and the embodiments of the present application do not make specific limitations. Wherein the original sample image can be understood as selecting a certain number of original images as samples from the handling process of the business scenario based on the needs of the actual business scenario, which can be used for training or testing of semantic segmentation network model, optionally, the number, size and type of original sample images can be selected based on actual needs, and the embodiments of the present application do not make specific limitations.

[0097] Step 402, the computer device inputs the initial adversarial sample image into the semantic segmentation network model, and uses the feature deception attack algorithm based on cosine similarity to attack the semantic segmentation network model, calculates the saturation of the output features of each convolutional layer in the semantic segmentation network model and the saturation difference of the output features of adjacent two convolutional layers, if the saturation of the specified convolutional layer is greater than the first preset threshold and the saturation difference of the output features of the adjacent convolutional layer is less than the second preset threshold, the candidate adversarial sample image output by the specified convolutional layer is saved.

[0098] Since the convolutional layer is responsible for feature extraction, the attack algorithm in the embodiments of the present application can attack all convolutional layers. The feature deception algorithm based on cosine similarity in the embodiments of the present application is a multi-step iterative algorithm, but it does not have obvious directionality like deepfool, and needs a large number of iterations to continuously approach the optimal solution. The embodiments of the present application consider avoiding local optimal solution, and set Sat t The variable represents the saturation of the perturbed image (the ratio of pixels reaching the upper and lower limits of attack intensity) and Sat t The difference between the two training SatC t If SatC t is too high, SatC t If the value is too small, it can be considered that the training has reached a "saturation state".

[0099] In the embodiments of the present application, the initial adversarial sample image is input into the semantic segmentation network model, the feature deception algorithm based on cosine similarity is used to attack the semantic segmentation network model, that is, to attack all convolutional layers in the semantic segmentation network model, and the saturation of the output features of each convolutional layer in the semantic segmentation network model and the saturation difference between adjacent two convolutional layers are calculated. If the saturation corresponding to a certain convolutional layer is greater than a first preset threshold, and the saturation difference between the convolutional layer and the adjacent convolutional layer is less than a second preset threshold, the candidate adversarial sample image corresponding to the output of the convolutional layer is saved. The size of the first preset threshold and the second preset threshold can be set based on actual needs, and can be the same value or different values, which are not limited in the embodiments of the present application.

[0100] In step 403, the computer device compresses the perturbation in the candidate adversarial sample image according to a preset ratio, inputs the compressed candidate adversarial sample image into the semantic segmentation network model, and uses the feature deception algorithm based on cosine similarity to attack the semantic segmentation network model. Iterative calculation is performed until the number of iterations reaches a specified number, and the candidate adversarial sample image corresponding to each iteration is saved.

[0101] In the embodiments of the present application, the perturbation in the candidate adversarial sample image is compressed according to a preset ratio. Optionally, if the preset ratio is 0.5, that is, the compression rate is 0.5, the perturbation can be reduced by half. The compressed candidate adversarial sample image is input into the semantic segmentation network model, and the above steps are repeated to calculate the saturation and saturation difference corresponding to each convolutional layer of the semantic segmentation network model, obtain the candidate adversarial sample image corresponding to a new round of iteration calculation until the number of iterations reaches a specified number, and save the candidate adversarial sample image corresponding to each iteration.

[0102] Step 404, the computer device sequentially adds the perturbation in each candidate adversarial sample image to the test sample image, sequentially inputs the test sample image added with different perturbations to the semantic segmentation network model, calculates the deception rate of the perturbation in each candidate adversarial sample image, and selects the candidate adversarial sample image corresponding to the perturbation with the highest deception rate as the adversarial sample image.

[0103] The test sample image can be understood as a test sample set used to test the accuracy of the semantic segmentation network model, and there can be multiple test sample images. In the embodiment of the present application, the candidate adversarial sample image corresponding to the perturbation with the highest deception rate is selected as the adversarial sample image. The adversarial sample image can cause the greatest interference to the semantic segmentation network model, thereby laying a data foundation for subsequent optimization training of the semantic segmentation network model.

[0104] Step 405, optimizing and training the semantic segmentation network model by using the training sample set to obtain a semantic segmentation network optimization model, wherein the training sample set is a set in which the adversarial sample image is added according to a specified proportion.

[0105] The training sample set is a set in which the adversarial sample image is added according to a specified proportion; the training sample set includes the adversarial sample image and the general sample image; and the specified proportion is a proportion that makes the semantic segmentation network optimization model obtained by training optimal.

[0106] In the embodiment of the present application, because the recognition effect and the anti-interference ability of the semantic segmentation network optimization model obtained by training using the training sample set vary with the proportion of the adversarial sample image in the training sample set, in order to make the semantic segmentation network optimization model obtained by training using the training sample set have the best recognition effect and anti-interference ability, it is necessary to determine the best proportion of the adversarial sample image in the training sample set, which is the specified proportion. Optionally, the specified proportion can be a best proportion value obtained according to a large amount of experimental data, or can be a best proportion range. Whether it is a best proportion value or a best proportion range, it is necessary to ensure that the semantic segmentation network optimization model obtained by training using the training sample set containing the specified proportion of adversarial sample images is optimal, that is, the semantic segmentation network optimization model has the best recognition effect and anti-interference ability.

[0107] In the prior art, the semantic segmentation network model is usually attacked based on the GD-UAP algorithm to obtain an adversarial sample image, and the training sample set with the adversarial sample image added according to a specified proportion is used to optimize and train the semantic segmentation network model, so as to obtain a semantic segmentation network optimization model. However, in the embodiment of the present application, the semantic segmentation network model is attacked based on the feature deception algorithm of cosine similarity to obtain an adversarial sample image, and the training sample set with the adversarial sample image added according to a specified proportion is used to optimize and train the semantic segmentation network model, so as to obtain a semantic segmentation network optimization model. Since the feature deception algorithm of cosine similarity has a higher deception rate than the GD-UAP algorithm in the prior art, the semantic segmentation network trained in the embodiment of the present application has better anti-interference ability and stronger recognition ability.

[0108] Step 406, identifying the to-be-identified image by using the semantic segmentation network optimization model.

[0109] In the embodiment of the present application, first, a disturbance is added to the original sample image to obtain an initial adversarial sample image; second, the initial adversarial sample image is input into the semantic segmentation network model, and the semantic segmentation network model is attacked by using the feature deception algorithm of cosine similarity to obtain an adversarial sample image; third, the training sample set with the adversarial sample image added according to a specified proportion is used to optimize and train the semantic segmentation network model, so as to obtain a semantic segmentation network optimization model; and finally, the to-be-identified image is identified by using the semantic segmentation network optimization model. Since the feature deception algorithm of cosine similarity has a high deception rate, the adversarial sample image generated by the algorithm is added to the training sample set, and the semantic segmentation network optimization model trained based on the training sample set has strong anti-interference ability and good recognition effect.

[0110] It should be understood that, although each step in the flowchart is displayed in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the figure can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps.

[0111] Please refer to Figure 5 , which shows a block diagram of an image recognition device 500 provided by an embodiment of the present application. The device 500 can be configured in the computer device of Figure 1 . As shown in the figure, the device 500 includes a feature deception algorithm of cosine similarity 501, a semantic segmentation network model 502, a training sample set 503, and a semantic segmentation network optimization model 504.Figure 5 As shown, the apparatus 500 includes a first obtaining module 501, a second obtaining module 502, a third obtaining module 503, and an identifying module 504.

[0112] The first obtaining module 501 is configured to add a perturbation to an original sample image to obtain an initial adversarial sample image. The second obtaining module 502 is configured to input the initial adversarial sample image into a semantic segmentation network model and attack the semantic segmentation network model using a feature deception algorithm based on cosine similarity to obtain an adversarial sample image. The third obtaining module 503 is configured to optimize and train the semantic segmentation network model using a training sample set to obtain a semantic segmentation network optimization model, wherein the training sample set is a set in which the adversarial sample image is added according to a specified proportion. The identifying module 504 is configured to identify a to-be-identified image using the semantic segmentation network optimization model.

[0113] In an optional embodiment of the present application, a loss function in the semantic segmentation network model aims to maximize an included angle between a first vector and a second vector output by each layer of the semantic segmentation network model, the first vector being a vector corresponding to the original sample image output by each layer of the semantic segmentation network model, and the second vector being a vector corresponding to the initial adversarial sample image output by each layer of the semantic segmentation network model.

[0114] In an optional embodiment of the present application, the loss function in the semantic segmentation network model in the second obtaining module 502 is:

[0115]

[0116]

[0117] Such than||δ|| ∞ <ξ

[0118] wherein K is a range of layers to be attacked, θ is to maximize an included angle between an original output and an adversarial output of each layer, ξ is an infinitesimal, and l i (x) is an output of the i-th layer of the semantic segmentation network model, l i (x+δ) is an output of the i-th layer of the semantic segmentation network model in the case of adding a perturbation to a data matrix of the original sample image, ||l i (x+δ)||2 is a second norm of a matrix formed after adding a perturbation to the original sample image data matrix, ||l i (x)||2 is a second norm of the original sample image data matrix, and δ is an adversarial perturbation.

[0119] In an optional embodiment of the present application, the second obtaining module 502 is specifically configured to: input the initial adversarial sample image into the semantic segmentation network model, attack the semantic segmentation network model using the feature deception algorithm based on cosine similarity, calculate the saturation of the output features of each convolutional layer in the semantic segmentation network model and the saturation difference of the output features of two adjacent convolutional layers, save the candidate adversarial sample image output by a specified convolutional layer if the saturation of the specified convolutional layer is greater than a first preset threshold and the saturation difference of the output features of the adjacent convolutional layers is less than a second preset threshold, compress the perturbation in the candidate adversarial sample image according to a preset ratio, input the compressed candidate adversarial sample image into the semantic segmentation network model, attack the semantic segmentation network model using the feature deception algorithm based on cosine similarity, and perform iterative calculation until the number of iterations reaches a specified number, and save the candidate adversarial sample image corresponding to each iteration.

[0120] In an optional embodiment of the present application, the second obtaining module 502 is specifically configured to: sequentially add the perturbations in the candidate adversarial sample images to the test sample image, input the test sample image with different perturbations added to the semantic segmentation network model in sequence, calculate the deception rates of the perturbations in the candidate adversarial sample images, and select the candidate adversarial sample image corresponding to the perturbation with the highest deception rate as the adversarial sample image.

[0121] In an optional embodiment of the present application, the second obtaining module 502 is specifically configured to: after obtaining the candidate adversarial sample image in each iteration, add the perturbation in the obtained candidate adversarial sample image to the test sample image, input the test sample image with the added perturbation to the semantic segmentation network model, and calculate the deception rate of the perturbation in the candidate adversarial sample image; if the deception rates of the perturbations in the candidate adversarial sample images obtained by continuous specified number of iterations are less than the deception rate of the perturbation in the candidate adversarial sample image obtained by previous iteration, stop the iteration, and select the candidate adversarial sample image corresponding to the perturbation with the highest deception rate as the adversarial sample image.

[0122] In an optional embodiment of the present application, the second obtaining module 502 is specifically configured to: calculate the deception rate of the perturbation in the candidate adversarial sample image using the evaluation index of semantic segmentation.

[0123] The specific limitations of the image recognition device can be referred to the limitations of the image recognition method in the above, which will not be repeated here. Each module in the above image recognition device can be realized by software, hardware and combinations thereof, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above modules.

[0124] In one embodiment, a computer device is provided, and an internal structure diagram of the computer device can be as shown in Figure 6 The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved through WIFI, a carrier network, NFC (Near Field Communication) or other technologies. The computer program is executed by the processor to implement an image recognition method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0125] Those skilled in the art can understand that Figure 6 The structure shown in the above embodiment is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. Specifically, the computer device can include more or fewer components than those shown in the diagram, or combine certain components, or have a different arrangement of components.

[0126] In one embodiment of the present application, a computer device is provided, which includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following steps are implemented: adding a disturbance to an original sample image to obtain an initial adversarial sample image; inputting the initial adversarial sample image into a semantic segmentation network model and using a feature deception algorithm based on cosine similarity to attack the semantic segmentation network model to obtain an adversarial sample image; optimizing and training the semantic segmentation network model using a training sample set to obtain a semantic segmentation network optimization model, wherein the training sample set is a set in which the adversarial sample image is added according to a specified proportion; and identifying a to-be-identified image using the semantic segmentation network optimization model.

[0127] In one embodiment of the present application, the loss function in the semantic segmentation network model aims to maximize the included angle between a first vector and a second vector output by each layer of the semantic segmentation network model. The first vector is a vector corresponding to the original sample image output by each layer of the semantic segmentation network model, and the second vector is a vector corresponding to the initial adversarial sample image output by each layer of the semantic segmentation network model.

[0128] In an embodiment of the present application, the loss function in the semantic segmentation network model is:

[0129]

[0130]

[0131] Such than||δ|| ∞ <ξ

[0132] Wherein, K is the range of the number of layers attacked, θ is the maximum angle between the original output of each layer and the output of the attack, ξ is an infinitesimal, l i (x) is the output of the i-th layer of the semantic segmentation network model, l i (x+δ) is the output of the i-th layer of the semantic segmentation network model in the case of adding perturbation to the data matrix of the original sample image, ||l i (x+δ)||2 is the second norm of the matrix formed after adding perturbation to the original sample image data matrix, ||l i (x)||2 is the second norm of the original sample image data matrix, and δ is the adversarial perturbation.

[0133] In an embodiment of the present application, when the processor executes the computer program, the following steps are also implemented: inputting the initial adversarial sample image into the semantic segmentation network model, and using the feature deception algorithm based on cosine similarity to attack the semantic segmentation network model, calculating the saturation of the output features of each convolutional layer in the semantic segmentation network model and the saturation difference of the output features of adjacent two convolutional layers, if the saturation of the specified convolutional layer is greater than the first preset threshold and the saturation difference of the output features of the adjacent convolutional layers is less than the second preset threshold, then the candidate adversarial sample image output by the specified convolutional layer is saved; compressing the perturbation in the candidate adversarial sample image according to a preset ratio, inputting the compressed candidate adversarial sample image into the semantic segmentation network model, and using the feature deception algorithm based on cosine similarity to attack the semantic segmentation network model, and performing iterative calculation until the number of iterations reaches a specified number, and saving the candidate adversarial sample image corresponding to each iteration calculation; selecting one of the candidate adversarial sample images as the adversarial sample image.

[0134] In an embodiment of the present application, when the processor executes the computer program, the following steps are also implemented: sequentially adding the perturbation in each candidate adversarial sample image to the test sample image, sequentially inputting the test sample images with different perturbations added into the semantic segmentation network model, calculating the deception rate of the perturbation in each candidate adversarial sample image, and selecting the candidate adversarial sample image corresponding to the perturbation with the highest deception rate as the adversarial sample image.

[0135] In an embodiment of the present application, the processor, when executing the computer program, also implements the following steps: after obtaining the candidate adversarial sample image in each iteration calculation, adding the disturbance in the obtained candidate adversarial sample image to the test sample image, inputting the test sample image added with the disturbance to the semantic segmentation network model, and calculating the deception rate of the disturbance in the candidate adversarial sample image; if the deception rate of the disturbance in the candidate adversarial sample image obtained by the continuous specified number of iteration calculations is less than the deception rate of the disturbance in the candidate adversarial sample image obtained by the previous iteration calculation, stopping the iteration calculation, and selecting the candidate adversarial sample image corresponding to the disturbance with the highest deception rate as the adversarial sample image.

[0136] In an embodiment of the present application, the processor, when executing the computer program, also implements the following steps: using the evaluation index of semantic segmentation to calculate the deception rate of the disturbance in the candidate adversarial sample image.

[0137] The computer device provided by the embodiment of the present application has similar implementation principles and technical effects to the above method embodiments, and will not be described here.

[0138] In an embodiment of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the following steps:

[0139] adding the disturbance to the original sample image to obtain an initial adversarial sample image; inputting the initial adversarial sample image to the semantic segmentation network model, and using the feature deception algorithm based on the cosine similarity to attack the semantic segmentation network model to obtain an adversarial sample image; optimizing and training the semantic segmentation network model by using a training sample set, to obtain a semantic segmentation network optimization model, wherein the training sample set is a set in which the adversarial sample image is added according to a specified proportion; and identifying the image to be identified by using the semantic segmentation network optimization model.

[0140] In an embodiment of the present application, the loss function in the semantic segmentation network model aims to maximize the included angle between the first vector and the second vector output by each layer of the semantic segmentation network model, the first vector is the vector corresponding to the original sample image output by each layer of the semantic segmentation network model, and the second vector is the vector corresponding to the initial adversarial sample image output by each layer of the semantic segmentation network model.

[0141] In an embodiment of the present application, the loss function in the semantic segmentation network model is:

[0142]

[0143]

[0144] Such than||δ||| ∞ <ξ

[0145] wherein, K is the range of the layer to be attacked, θ is the maximum angle between the original output of each layer and the output of the adversarial, ξ is an infinitesimal, and l i (x) is the output of the i-th layer of the semantic segmentation network model, l i (x+δ) is the output of the i-th layer of the semantic segmentation network model in the case of adding perturbation to the data matrix of the original sample image, ||l i (x+δ)||2 is the second norm of the matrix formed after adding the perturbation to the original sample image data matrix, ||l i (x)||2 is the second norm of the original sample image data matrix, and δ is the adversarial perturbation.

[0146] In an embodiment of the present application, when the computer program is executed by the processor, the following steps are further implemented: inputting the initial adversarial sample image into the semantic segmentation network model, and using the feature deception algorithm based on cosine similarity to attack the semantic segmentation network model, calculating the saturation of the output features of each convolutional layer in the semantic segmentation network model and the saturation difference of the output features of adjacent two convolutional layers, if the saturation corresponding to the specified convolutional layer is greater than a first preset threshold and the saturation difference of the output features of the adjacent convolutional layers is less than a second preset threshold, then saving the candidate adversarial sample image output by the specified convolutional layer; compressing the perturbation in the candidate adversarial sample image according to a preset proportion, inputting the compressed candidate adversarial sample image into the semantic segmentation network model, and using the feature deception algorithm based on cosine similarity to attack the semantic segmentation network model, and iteratively calculating until the number of iterations reaches a specified number, saving the candidate adversarial sample image corresponding to each iteration calculation; selecting one of the candidate adversarial sample images as the adversarial sample image.

[0147] In an embodiment of the present application, when the computer program is executed by the processor, the following steps are further implemented: sequentially adding the perturbation in each candidate adversarial sample image to the test sample image, sequentially inputting the test sample images with different perturbations added into the semantic segmentation network model, calculating the deception rate of the perturbation in each candidate adversarial sample image, and selecting the candidate adversarial sample image corresponding to the perturbation with the highest deception rate as the adversarial sample image.

[0148] In an embodiment of the present application, the computer program, when executed by the processor, further implements the following steps: after obtaining the candidate adversarial sample image in each iteration calculation, adding the disturbance in the obtained candidate adversarial sample image to the test sample image, inputting the test sample image added with the disturbance into the semantic segmentation network model, and calculating the deception rate of the disturbance in the candidate adversarial sample image; if the deception rates of the disturbances in the candidate adversarial sample images obtained in the continuous specified number of iteration calculations are less than the deception rate of the disturbance in the candidate adversarial sample image obtained in the previous iteration calculation, stopping the iteration calculation, and selecting the candidate adversarial sample image corresponding to the disturbance with the highest deception rate as the adversarial sample image.

[0149] In an embodiment of the present application, the computer program, when executed by the processor, further implements the following steps: using the evaluation index of semantic segmentation to calculate the deception rate of the disturbance in the candidate adversarial sample image.

[0150] The computer readable storage medium provided in the embodiment has similar implementation principles and technical effects to the above method embodiments, and details are not described herein.

[0151] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the computer program can include the processes of the above-mentioned embodiments. In the embodiments provided by the present application, any reference to the memory, storage, database or other medium can include at least one of the non-volatile and volatile memories. The non-volatile memory can include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory or an optical storage, etc. The volatile memory can include a random access memory (RAM) or an external cache memory. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0152] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0153] The above embodiments only express several implementation ways of the present application, and the description is more specific and detailed, but it should not be understood as a limitation to the patent scope of the application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. An image recognition method, characterized in that, The method includes: Add perturbations to the original sample image to obtain the initial adversarial sample image; The initial adversarial sample image is input into the semantic segmentation network model, and the semantic segmentation network model is attacked using a feature deception algorithm based on cosine similarity to obtain the adversarial sample image. The semantic segmentation network model is optimized and trained using a training sample set to obtain an optimized semantic segmentation network model, wherein the training sample set is a set in which the adversarial sample images are added according to a specified ratio; The semantic segmentation network optimization model is used to identify the image to be identified; The objective of the loss function in the semantic segmentation network model is to maximize the angle between the first vector and the second vector output by each layer of the semantic segmentation network model. The first vector is the vector output by each layer of the semantic segmentation network model corresponding to the original sample image, and the second vector is the vector output by each layer of the semantic segmentation network model corresponding to the initial adversarial sample image. The loss function in the semantic segmentation network model is: Make |δ| ∞ <ξ Where K is the range of the attacked layers, θ is the angle between the original output and the adversarial output of each layer to minimize the cosine similarity, ξ is an infinitesimal quantity, li(x) is the output of the i-th layer of the semantic segmentation network model, li(x+δ) is the output of the i-th layer of the semantic segmentation network model when perturbation is added to the data matrix of the original sample image, ‖li(x+δ)‖2 is the second normal form of the matrix formed after perturbation is added to the data matrix of the original sample image, ‖li(x)‖2 is the second normal form of the original sample image data matrix, and δ is the adversarial perturbation; Wherein, li(x) in the loss function is the first vector output by each layer of the semantic segmentation network model, and li(x+δ) is the second vector output by each layer of the semantic segmentation network model; The loss function suppresses the loss value from approaching 0 in the later stages of training by subtracting the cosine value from 1, and limits the range of the loss value by logarithmic operation.

2. The method according to claim 1, characterized in that, The step of inputting the initial adversarial sample image into a semantic segmentation network model and attacking the semantic segmentation network model using a feature deception algorithm based on cosine similarity to obtain the adversarial sample image includes: The initial adversarial sample image is input into the semantic segmentation network model, and the semantic segmentation network model is attacked using a feature deception algorithm based on cosine similarity. The saturation of the output features of each convolutional layer in the semantic segmentation network model and the saturation difference between the output features of two adjacent convolutional layers are calculated. If the saturation of the specified convolutional layer is greater than a first preset threshold and the saturation difference between the output features of the specified convolutional layer and the output features of the adjacent convolutional layer is less than a second preset threshold, the candidate adversarial sample image output by the specified convolutional layer is saved. The perturbations in the candidate adversarial sample image are compressed according to a preset ratio. The compressed candidate adversarial sample image is input into the semantic segmentation network model. The semantic segmentation network model is attacked using a feature deception algorithm based on cosine similarity. Iterative calculation is performed until the number of iterations reaches a specified number. The candidate adversarial sample image corresponding to each iteration calculation is saved. One of the candidate adversarial sample images is selected as the adversarial sample image.

3. The method according to claim 2, characterized in that, The step of selecting one image from each of the candidate adversarial sample images as the adversarial sample image includes: The perturbations in each of the candidate adversarial sample images are added to the test sample image in sequence. The test sample images with different perturbations are then input into the semantic segmentation network model. The deception rate of the perturbations in each of the candidate adversarial sample images is calculated. The candidate adversarial sample image corresponding to the perturbation with the highest deception rate is selected as the adversarial sample image.

4. The method according to claim 2, characterized in that, The step of selecting one image from each of the candidate adversarial sample images as the adversarial sample image includes: After obtaining the candidate adversarial sample image in each iteration, the perturbation in the obtained candidate adversarial sample image is added to the test sample image, and the perturbation-added test sample image is input into the semantic segmentation network model to calculate the deception rate of the perturbation in the candidate adversarial sample image. If the deception rate of the perturbation of the candidate adversarial sample image obtained by the consecutive iteration calculation of a specified number of times is less than the deception rate of the perturbation of the candidate adversarial sample image obtained by the previous iteration calculation, then the iteration calculation is stopped, and the candidate adversarial sample image corresponding to the perturbation with the highest deception rate is selected as the adversarial sample image.

5. The method according to claim 4, characterized in that, The calculation of the deception rate of the perturbation in the candidate adversarial sample image includes: The deception rate of perturbations in the candidate adversarial sample images is calculated using an evaluation metric for semantic segmentation.

6. An image recognition device, characterized in that, The device includes: The first acquisition module is used to add perturbations to the original sample image to obtain the initial adversarial sample image; The second acquisition module is used to input the initial adversarial sample image into the semantic segmentation network model and use a feature deception algorithm based on cosine similarity to attack the semantic segmentation network model in order to obtain the adversarial sample image. The third acquisition module is used to optimize and train the semantic segmentation network model using the training sample set to obtain the optimized semantic segmentation network model, wherein the training sample set is a set in which the adversarial sample images have been added according to a specified ratio; The recognition module is used to recognize the image to be recognized using the semantic segmentation network optimization model; The objective of the loss function in the semantic segmentation network model is to maximize the angle between the first vector and the second vector output by each layer of the semantic segmentation network model. The first vector is the vector output by each layer of the semantic segmentation network model corresponding to the original sample image, and the second vector is the vector output by each layer of the semantic segmentation network model corresponding to the initial adversarial sample image. The loss function in the semantic segmentation network model is: Make |δ| ∞ <ξ Where K is the range of the attacked layers, θ is the angle between the original output and the adversarial output of each layer to minimize the cosine similarity, ξ is an infinitesimal quantity, li(x) is the output of the i-th layer of the semantic segmentation network model, li(x+δ) is the output of the i-th layer of the semantic segmentation network model when perturbation is added to the data matrix of the original sample image, ‖li(x+δ)‖2 is the second normal form of the matrix formed after perturbation is added to the data matrix of the original sample image, ‖li(x)‖2 is the second normal form of the original sample image data matrix, and δ is the adversarial perturbation; Wherein, li(x) in the loss function is the first vector output by each layer of the semantic segmentation network model, and li(x+δ) is the second vector output by each layer of the semantic segmentation network model; The loss function suppresses the loss value from approaching 0 in the later stages of training by subtracting the cosine value from 1, and limits the range of the loss value by logarithmic operation.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images