A method and device for identifying disease types of livestock and poultry diseases

Abstract

The present invention provides a method and device for identifying the type of disease in livestock and poultry disease images, which relates to the technical field of image data processing. The method includes: obtaining livestock and poultry images to be processed, and calling a livestock and poultry image attribute extractor based on an inverse mapping generative adversarial network to extract appearance latent variable features; generating appearance latent variable features through a healthy livestock and poultry image generator based on a generative adversarial network to obtain healthy livestock and poultry images; determining a corrected difference map between the healthy livestock and poultry image and the livestock and poultry image to be processed, and calling an image classifier to classify the corrected difference map; when the classification result of the corrected difference map is a lesion area image, locating the type of disease lesion area of ​​the livestock and poultry image to be processed by using the corrected difference map. Through this application, the technical problem that the recognition accuracy and lesion area positioning accuracy still need to be improved in the application of livestock and poultry disease recognition in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a method and device for identifying animal and poultry disease image types. Background Art

[0002] In recent years, as livestock and poultry farming has gradually become larger, more intensive, standardized and intelligent, the density of livestock and poultry farming has also been increasing, and the problem of livestock and poultry diseases has become increasingly prominent.

[0003] Being able to diagnose livestock and poultry diseases in a timely and accurate manner is the key to taking effective prevention and control measures, reducing the risk of disease transmission, and protecting the healthy development of the livestock and poultry industry. However, in livestock and poultry farms, individual detection, disease diagnosis, and abnormal behavior identification of livestock and poultry are currently mainly carried out by professional inspectors who constantly patrol the breeding houses. This method has the problems of large workload, strong subjectivity, poor environment, and the risk of cross-infection. In the application of livestock and poultry disease identification in the prior art, deep learning technology is used to collect livestock and poultry disease image data and build livestock and poultry disease identification models to achieve intelligent identification of livestock and poultry diseases, but the recognition accuracy and the accuracy of disease area positioning still need to be improved. Summary of the invention

[0004] The present invention provides a method and device for identifying animal and poultry disease images, which are used to solve the technical problem that in the application of animal and poultry disease identification in the prior art, deep learning technology is used to realize intelligent identification of animal and poultry diseases, but the recognition accuracy still needs to be improved.

[0005] The present invention provides a method for identifying livestock and poultry disease images, the method comprising the following steps:

[0006] Acquire a livestock image to be processed, and call a livestock image attribute extractor based on an inverse mapping generative adversarial network to extract appearance latent variable features of the livestock image to be processed;

[0007] The appearance latent variable features are generated by a healthy livestock and poultry image generator based on a generative adversarial network to obtain a healthy livestock and poultry image;

[0008] Determining a corrected difference image between the healthy livestock and poultry image and the livestock and poultry image to be processed, and calling an image classifier to classify the corrected difference image;

[0009] When the classification result of the corrected difference image is a lesion region image, the lesion-like region of the livestock and poultry image to be processed is located by using the corrected difference image.

[0010] In some embodiments, the livestock and poultry image attribute extractor and the healthy livestock and poultry image generator are connected in series, the network layer structures of the livestock and poultry image attribute extractor and the healthy livestock and poultry image generator are symmetrical, and in the livestock and poultry image attribute extractor and the healthy livestock and poultry image generator, at least one pair of symmetrical network layers are connected by a skip connection module, and the processing process of the skip connection module includes:

[0011] Obtaining a feature map output by a network layer of the livestock and poultry image attribute extractor;

[0012] The feature map is subjected to low-pass filtering, and the feature map after the low-pass filtering is input into the network layer of the healthy livestock and poultry image generator.

[0013] In some embodiments, the healthy livestock and poultry image is obtained by calling an image correction model to correct the livestock and poultry image to be processed, the image correction model includes an image attribute extractor, an image generator and an image discriminator, and the training process of the image correction model includes:

[0014] Acquire livestock and poultry image samples, and input the livestock and poultry image samples into the image correction model for forward propagation to obtain a corrected image;

[0015] Constructing a total loss function of the image attribute extractor and the image discriminator according to the livestock and poultry image sample and the corrected image;

[0016] Determine the Wasserstein distance between the true probability distribution of the livestock and poultry image sample and the generated probability distribution of the corrected image, and use the Wasserstein distance as a generation loss function of the image generator;

[0017] The sum of the total loss function and the generated loss function is used as the final loss function of the image correction model;

[0018] The final loss function is used to perform back propagation in the image correction model to update the parameters of the image correction model.

[0019] In some embodiments, the total loss function of the image attribute extractor and the image discriminator is constructed according to the livestock and poultry image sample and the corrected image, including:

[0020] Determine a first discriminant value of the image discriminator for the livestock and poultry image sample and a second discriminant value of the image discriminator for the corrected image, and construct a discriminant loss function of the image discriminator according to a difference between the first discriminant value and the second discriminant value;

[0021] Determine the mean square error between the livestock and poultry image sample and the corrected image, and use the mean square error as a reconstruction loss function of the image attribute extractor when reconstructing the image;

[0022] Preset reconstruction weights and discrimination weights;

[0023] The reconstruction loss function is weighted by the reconstruction weight to obtain a reconstruction weighted result, and the discrimination loss function is weighted by the discrimination weight to obtain a discrimination weighted result;

[0024] The sum of the reconstruction weighted result and the discrimination weighted result is used as the total loss function of the image attribute extractor and the image discriminator.

[0025] In some embodiments, the image classifier is trained by collecting lesion area image samples, the lesion area image samples carry real disease labels, and the training process of the image classifier is:

[0026] Acquire livestock and poultry disease images, and perform correction processing on the livestock and poultry disease images to obtain corresponding disease image samples;

[0027] Determine a lesion difference map between the livestock and poultry disease image and the disease image sample, and extract a lesion area image from the lesion difference map as a lesion area image sample;

[0028] Inputting the lesion area image sample into the image classifier for forward propagation to obtain a disease prediction label;

[0029] A disease classification loss is constructed based on the disease true label and the disease predicted label, and back-propagation is performed in the image classifier through the disease classification loss to update the parameters of the image classifier.

[0030] In some embodiments, extracting the lesion area image from the lesion difference map includes:

[0031] Preset pixel value threshold;

[0032] For each pixel point in the lesion difference image, when the threshold of the pixel point is greater than the pixel value threshold, determining the pixel point as the first pixel point of the abnormal symptom area;

[0033] When the threshold of the pixel point is less than or equal to the pixel value threshold, determining that the pixel point is a second pixel point in the normal area;

[0034] A lesion area image is determined based on the first pixel point and the second pixel point.

[0035] The present invention also provides a device for identifying livestock and poultry disease images, which includes the following modules:

[0036] An extraction module, used for acquiring livestock and poultry images to be processed, and calling a livestock and poultry image attribute extractor based on an inverse mapping generative adversarial network to extract appearance latent variable features of the livestock and poultry images to be processed;

[0037] A generation module, used to generate the appearance latent variable features through a healthy livestock and poultry image generator based on a generative adversarial network to obtain a healthy livestock and poultry image;

[0038] A determination module, used to determine a corrected difference image between the healthy livestock and poultry image and the livestock and poultry image to be processed, and call an image classifier to classify the corrected difference image;

[0039] The identification module is used to locate the disease-like disease area of ​​the livestock and poultry image to be processed through the corrected difference image when the classification result of the corrected difference image is a disease area image.

[0040] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for identifying animal and poultry disease image types as described in any one of the above is implemented.

[0041] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for identifying animal and poultry disease image types as described in any one of the above is implemented.

[0042] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for identifying animal and poultry disease image categories.

[0043] The method and device for identifying the type of disease images of livestock and poultry provided by the present invention correct the livestock and poultry images to be processed through a livestock and poultry image attribute extractor based on an inverse mapping generative adversarial network and a healthy livestock and poultry image generator based on a generative adversarial network to obtain healthy livestock and poultry images, and then calculate the corrected difference map between the healthy livestock and poultry images and the livestock and poultry images to be processed, and call an image classifier to classify the corrected difference map to locate the type of disease lesion area of ​​the livestock and poultry images to be processed, thereby realizing the type of disease identification of the livestock and poultry images to be processed. Compared with directly using the original livestock and poultry images to be processed for type of disease identification, the present invention can effectively remove image features that are not related to livestock and poultry lesions, reduce the impact on the classification results, thereby improving the accuracy of type of disease identification and improving the positioning accuracy of the lesion area. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 It is a flow chart of the method for identifying livestock and poultry disease images provided by the present invention.

[0046] Figure 2 It is a schematic diagram of the framework of the image correction model provided by the present invention.

[0047] Figure 3 It is a model architecture diagram of the image generator provided by the present invention.

[0048] Figure 4 This is a model architecture diagram of the attribute extractor provided by the present invention.

[0049] Figure 5 It is a model architecture diagram of the discriminator provided by the present invention.

[0050] Figure 6 It is a structural schematic diagram of the device for identifying livestock and poultry disease image types provided by the present invention.

[0051] Figure 7 It is a schematic diagram of the physical structure of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Combine the following Figure 1-Figure 6 The present invention describes a method and device for identifying livestock and poultry disease images. Figure 1 FIG. 1 is a flow chart of the method for identifying livestock and poultry disease images provided by the present invention. Figure 1 As shown, the method includes the following steps 101 to 104.

[0054] Step 101: Obtain a livestock image to be processed, and call a livestock image attribute extractor based on an inverse mapping generative adversarial network to extract appearance latent variable features of the livestock image to be processed.

[0055] First, the images of livestock and poultry to be processed are obtained. Here, an image collector is used to collect the image data corresponding to the livestock and poultry in a specific scene environment, and the image data is uploaded as the livestock and poultry images to be processed for subsequent symptom identification. The symptom in the embodiment of the present invention refers to common diseases that livestock and poultry may suffer from. It can be an external disease, so the image collector directly collects the appearance images of the livestock and poultry life. In some implementation scenarios, it can also be an internal disease, and the image collector collects pathological images of the tissues or organs inside the livestock and poultry body. The image data collected by the image collector varies in size and needs to be preprocessed. Here, image adjustment is performed uniformly, that is, each livestock and poultry image to be processed is resized to an appropriate image size.

[0056] After the preprocessing is completed, the livestock image attribute extractor based on the inverse mapping generative adversarial network is called to extract the appearance latent variable features of the livestock image to be processed. The livestock image attribute extractor here is actually a network model that uses the generative adversarial network inversion mapping technology (GAN Inversion), which can map the input livestock image to be processed to the latent variable space to extract the latent variable features corresponding to the appearance features, specifically including the appearance attribute features of the livestock image to be processed, such as color, posture, contour, etc.

[0057] See also Figure 4 , Figure 4 This is the model architecture diagram of the attribute extractor. Figure 4 The network layer structure of the livestock and poultry image attribute extractor in the embodiment of the present invention is illustrated, which is specifically composed of 5 convolution modules. Here, the width, height, and channel size are 64. 64 3 as an example, except for the first and last two convolution modules, the remaining convolution modules can halve the width and height of the livestock and poultry image to be processed and double the number of channels to obtain a width, height, and channel size of 32. 32 32, 16 16 64, 8 8 128, 4 4 256 feature maps, the final layer sets the width and height to 1, and finally extracts the appearance latent variable features containing appearance attributes. This appearance latent variable feature is a feature vector containing image appearance attribute information, and the width, height, and channel size are 1 1 128.

[0058] Step 102: Generate appearance latent variable features through a healthy livestock and poultry image generator based on a generative adversarial network to obtain healthy livestock and poultry images.

[0059] After extracting the appearance latent variable features through step 101, the appearance latent variable features are generated by the healthy livestock and poultry image generator based on the generative adversarial network to obtain healthy livestock and poultry images. The healthy livestock and poultry image generator in the embodiment of the present invention is essentially an image generator based on the generative adversarial network (GAN). The specific structure is as follows Figure 3 As shown in the figure, the network layer structure of the image generator is also composed of 5 convolution modules. The input is the appearance latent variable features output by the livestock and poultry image attribute extractor. The width, height, and channel size are 1 1 128. Except for the first and last convolution modules, the remaining convolution blocks double the dimension of the appearance latent variable features and halve the number of channels, resulting in a width, height, and channel size of 4. 4 256, 8 8 128, 16 16 64, 32 32 32 feature maps, the final output width, height, and channel size are 64 64 The healthy livestock and poultry images of 3 are completely consistent with the width, height, and channel size of the livestock and poultry images to be processed.

[0060] Combining 101 and step 102, it can be understood that the output of the livestock image attribute extractor is the healthy livestock image generator. Therefore, the livestock image attribute extractor and the healthy livestock image generator in the embodiment of the present invention can be connected in series, which is essentially an image correction or image reconstruction process, which can input the livestock image to be processed and then directly generate a healthy livestock image with the same size. Figure 4 and Figure 5 It can be seen from the network layer structure that the network layer structure of the image attribute extractor and the healthy livestock and poultry image generator are symmetrical, both have 5 convolution modules, and each convolution module is symmetrical one by one, and the feature map size and channel processed by the two symmetrical convolution modules are the same.

[0061] The discriminator in the generative adversarial network is mainly used to evaluate the generation results of the healthy livestock and poultry image generator. Figure 5 The network structure of the discriminator is shown. Figure 5 It can be seen that the discriminator also specifically includes a 5-layer network structure. Figure 3 , Figure 4It can be seen that the network structure of the discriminator is symmetrical with that of the healthy livestock image generator, but similar to the network structure of the livestock image attribute extractor, but the output feature map. The input of the discriminator is the healthy livestock image output by the healthy livestock image generator, with a width, height, and channel size of 64. 64 3. Then, after five layers of network structure, except for the first and last two layers, each layer will halve the width and height and double the channel size, and the width, height, and channel size will be 32 in turn. 32 16, 16 16 32, 8 8 64, 4 4 The last layer of the feature map of 128 sets the width, height and number of channels to 1, that is, outputs the corresponding evaluation score, which is used to evaluate and feedback the generation results of the healthy livestock and poultry image generator, so that the healthy livestock and poultry image generator can generate more realistic healthy livestock and poultry images.

[0062] In some embodiments, it is considered that when the livestock and poultry image attribute extractor processes the image to be processed, the feature extraction is continuously performed through 5 convolution modules, which may cause the loss of feature information. In order to retain more feature information, especially low-frequency signals, before the healthy livestock and poultry image generator generates the image, the embodiment of the present invention connects at least one pair of symmetrical network layers in the livestock and poultry image attribute extractor and the healthy livestock and poultry image generator through a skip connection module. The specific structure of the skip connection model can be a residual network structure.

[0063] Here, the livestock and poultry image attribute extractor and certain specific network layers in the healthy livestock and poultry image generator are connected through a skip connection module to realize the transmission of feature information. Of course, there may be more than one pair of specific network layers, which can be a pair of symmetrical network layers or two pairs of symmetrical network layers. The following example takes a pair of symmetrical network layers as an example.

[0064] For example, Figure 4 The feature map size output by the third convolutional module of the livestock and poultry image attribute extractor is 16 16 64, and Figure 3 The feature map size output by the third convolutional module of the healthy livestock and poultry image generator is also 16 16 64, indicating that the two convolution modules are symmetrical and the processed feature maps have the same size and channels. Therefore, the two convolution modules can be connected through a skip connection module, so that the feature map output by the third-layer convolution module of the livestock and poultry image attribute extractor can be directly input into the third-layer convolution module of the livestock and poultry image attribute extractor through the skip connection module to help generate the image, thereby completing the transmission of feature information. Because 16 16 The feature map size of 64 is compared with 1 1 The appearance latent variable features of 128 can retain more feature information, thereby reducing the loss of feature information and improving the structural similarity between the corrected image (i.e., the healthy livestock image) and the original image (the livestock image to be processed), that is, improving the image correction effect.

[0065] The processing process of the skip connection module is as follows: obtain the feature map output by the network layer of the livestock and poultry image attribute extractor, then perform low-pass filtering on the feature map, and finally input the feature map after low-pass filtering into the network layer of the healthy livestock and poultry image generator, so that the feature information of the livestock and poultry image to be processed can be better preserved in the healthy livestock and poultry image.

[0066] The healthy livestock image generator uses the generator in the generative adversarial network, which is mainly composed of two opposing neural networks: the generator G (Generator) and the discriminator D (Discriminator). The task of the generator is to generate realistic images from random noise, while the discriminator tries to distinguish these generated images from the original data. The objective function of its training uses the cross entropy loss function, denoted as , as shown in formula (1):

[0067] (1)

[0068] In the above formula (1), x represents the input image, and z represents any image with the same structure as the input image. represents the image generated by the generator G, represents the discriminant value output by the discriminator D for the input image x, and Represents the image generated by the discriminator D for the generator G Output discriminant value. Indicates the calculation of expected value. represents the true probability distribution obeyed by the input image x, Represents the image generated by the generator G Obey the generated probability distribution.

[0069] The generator and the discriminator are improved together through adversarial training. The generator gradually generates more and more realistic images, while the discriminator continuously improves its recognition ability. Eventually, the generator can generate images that are indistinguishable from real ones. However, the use of the cross entropy loss function is often unstable in the actual training process and prone to mode collapse (i.e., the generator only generates a certain type of sample and lacks diversity). This makes the corrected healthy livestock and poultry images less effective and affects the recognition of subsequent types of diseases.

[0070] Based on the above scenario, the embodiment of the present invention also proposes an image correction model, which corrects the livestock and poultry images to be processed by calling the image correction module to obtain healthy livestock and poultry images. Figure 2 As shown, the image correction model includes an image attribute extractor, an image generator (i.e. Figure 2 The generator in ) and the image discriminator (i.e. Figure 2 The following describes the training process of the image correction model.

[0071] First, livestock and poultry image samples are obtained as input images. The livestock and poultry image samples are healthy images, that is, livestock and poultry images without class symptoms. Then, the livestock and poultry image samples are input into the image correction model for forward propagation to obtain the corrected image. Here, the poultry image samples are input as the input image x and input into the image attribute extractor to extract the appearance latent variable features. Then, the appearance latent variable features are input into the image generator for regeneration to obtain the corresponding corrected image. , as the output image.

[0072] Next, according to the livestock and poultry image sample x and the corrected image Construct the total loss function Loss of the image attribute extractor and the image discriminator. Here, the total loss function Loss is composed of the sum of the reconstruction loss function and the discriminant loss function, that is, Figure 2 The reconstruction loss and the discrimination loss shown in .

[0073] For the discriminant loss, determine the first discriminant value of the image discriminator for the livestock and poultry image sample and the second discriminant value of the image discriminator for the corrected image. These two discriminant values ​​are the image discriminator for the input image x and the output image The discriminant value (i.e., score) generated after discrimination can be used to calculate the loss based on the difference in the discriminant value when training the image discriminator. Therefore, the discriminant loss function of the image discriminator is constructed here based on the difference between the first discriminant value and the second discriminant value. After taking the absolute value of the calculated difference, it can be used as the discriminant loss function of the image discriminator.

[0074] The reconstruction loss can be obtained based on the mean square error generated by the image attribute extractor when reconstructing the image. Therefore, when constructing the reconstruction loss function, the mean square error between the livestock and poultry image samples and the corrected image is determined, and the mean square error is used as the reconstruction loss function of the image attribute extractor when reconstructing the image.

[0075] The discrimination loss and reconstruction loss represent the losses generated at different stages in the image correction process, so they should be focused on during training. Therefore, the embodiment of the present invention presets corresponding reconstruction weights for the discrimination loss and reconstruction loss respectively. And the discriminant weight , to balance the two losses.

[0076] Here, by reconstructing the weights The reconstruction loss function is weighted to obtain the reconstruction weighted result, and the weight is determined by The discriminant loss function is weighted to obtain the discriminant weighted result. Finally, the sum of the reconstruction weighted result and the discriminant weighted result is used as the total loss function Loss of the image attribute extractor and the image discriminator, which is expressed as the following formula (2):

[0077] (2)

[0078] In the above formula (2), represents the reconstruction weight, represents the discriminant weight, D() represents the corresponding discriminant value output by the image discriminator, N represents the total number of livestock and poultry image samples, represents the i-th livestock and poultry image sample input, represents the i-th corrected image generated and output, x represents the livestock and poultry image sample, Indicates a corrected image.

[0079] Next, it is necessary to construct the loss function of the image generator. In the embodiment of the present invention, when the image generator generates the corrected image, the Wasserstein distance between the true probability distribution of the livestock and poultry image sample and the generated probability distribution of the corrected image is first determined, which is denoted as , expressed as the following formula (3):

[0080] (3)

[0081] In the above formula (3), Indicates the true probability distribution of livestock and poultry image samples, represents the generation probability distribution obeyed by the modified image, represents all possible joint distributions composed of the true probability distribution and the generated probability distribution, represents all joint distributions One of the joint distributions, inf represents the infimum, and the minimum value under all possible joint distributions is calculated as the Wasserstein distance, and In the joint distribution The expected value of the distance between pixels in the corrected image distribution from the true probability distribution.

[0082] Here, Wasserstein distance is used as the generation loss function of the image generator Finally, the total loss function Loss is combined with the generated loss function The sum of is used as the final loss function of the image correction model. During training, the final loss function is used to back-propagate in the image correction model to update the parameters of the image correction model. The gradient of the model is calculated during the back-propagation process, and the gradient is optimized using the gradient descent algorithm to update the weight parameters of the image correction model. When the number of training iterations reaches the specified number or the final loss function begins to converge, the training ends.

[0083] The trained image correction model can directly generate healthy livestock and poultry images based on the input livestock and poultry images to be processed. In this way, the image correction model is obtained only through training with healthy image samples, which can make the obtained corrected image more similar to the original image, thereby effectively improving the effect of image correction. In addition, the image correction model constructs a generation loss function through the Wasserstein distance, which can effectively improve the generation effect of the image generator. Compared with the use of the traditional cross entropy loss function, it can make the training process more stable, avoid mode collapse, and generate multiple types of sample images, thereby improving the generation diversity of the corrected image.

[0084] Step 103: determine a corrected difference image between the healthy livestock and poultry image and the livestock and poultry image to be processed, and call an image classifier to classify the corrected difference image.

[0085] After the livestock and poultry image to be processed is corrected through step 102, the corrected difference map between the healthy livestock and poultry image and the livestock and poultry image to be processed is then determined. The purpose of calculating the corrected difference map here is to determine whether there is a lesion area in the livestock and poultry image to be processed for subsequent classification. Because the target correction model is trained with healthy images, the generated healthy livestock and poultry image must be a healthy image without a lesion area. Because the difference between the healthy image and the lesion image before and after correction is huge, by comparing the image difference before and after correction, the lesion condition can be effectively predicted.

[0086] The embodiment of the present invention here characterizes the difference before and after the correction image by calculating the correction difference map. Because the size specifications of the image to be processed and the healthy livestock and poultry image are the same, the number of pixels of the two is the same, and the positions of the pixels correspond one to one. Therefore, the difference between the pixel values ​​of the two can be calculated for the pixels in the same position, so that a pixel difference can be calculated for each pixel position, and finally a correction difference map is formed, whose width, height and channels are also consistent with the size specifications of the image to be processed and the healthy livestock and poultry image.

[0087] The corrected differential image can characterize the difference before and after the image is corrected. In the application scenario of the embodiment of the present invention, although there are many categories or types of abnormal symptoms of livestock and poultry, no matter what kind of abnormal symptom, the difference between the pixels of the lesion area of ​​the symptom before and after correction is large, while the difference between the pixels of the normal healthy image area before and after correction is small, or even no difference. Based on this feature of the abnormal symptom, the image classifier is called to classify the lesion area image. The image classifier will extract the pixel value features of each pixel in the lesion area image, and then map the pixel features to the probability of predicting that the pixel is a lesion area. In this way, it can be determined by classification whether the corrected differential image is a lesion area image or a healthy image.

[0088] However, before inputting the image classifier, it is necessary to perform image filtering on the lesion area image corresponding to the corrected difference map, filter out some noise pixel information that is far away from the lesion area or irrelevant, and then input the filtered lesion area image into the trained image classifier for symptom recognition. This can eliminate the influence of noise and irrelevant factors on the classification results to a certain extent.

[0089] In some embodiments, the image classifier is trained by collecting lesion area image samples, which carry true disease labels, such as the location of the corresponding lesion area, or a numerical label 1. The training process of the image classifier is introduced below.

[0090] First, obtain the livestock and poultry disease images, and perform correction processing on the livestock and poultry disease images to obtain the corresponding disease image samples. Here, the livestock and poultry disease images are images with lesion areas, and the correction processing can be implemented by calling the above-mentioned image correction model or other image generation models. The correction process is not repeated here.

[0091] Next, the lesion difference map between the livestock and poultry disease image and the disease image sample is determined. The calculation method of the lesion difference map is similar to the calculation of the corrected difference map in step 103, which will not be repeated here. Then, the lesion area image is extracted from the lesion difference map as the lesion area image sample.

[0092] Here, the extraction process of the lesion area image specifically includes: first, preset the pixel value threshold, which can be determined according to actual experience or manual prediction. Then traverse the pixel points in the lesion difference map. For each pixel point in the lesion difference map, when the threshold of the pixel point is greater than the pixel value threshold, it means that the difference in pixel value of the livestock and poultry disease image before and after correction is large, and this pixel point may belong to the pixel point of the abnormal disease lesion area, then the pixel point is determined to be the first pixel point of the abnormal disease area; when the threshold of the pixel point is less than or equal to the pixel value threshold, it means that the difference in pixel value of the livestock and poultry disease image before and after correction is small, and this pixel point belongs to the pixel point of the normal healthy area, then the pixel point is determined to be the second pixel point of the normal area. In this way, each pixel point in the lesion difference map can be divided, so as to preliminarily determine whether there is an abnormal disease lesion area in the livestock and poultry image to be processed, and finally determine the lesion area image according to the first pixel point and the second pixel point.

[0093] During model training, the image samples of the lesion area are input into the image classifier for forward propagation to obtain the disease prediction label, that is, to determine whether the lesion area exists in the image sample of the lesion area. Then, the disease classification loss is constructed by combining the true disease label with the disease prediction label, such as the cross entropy loss. The disease classification loss is used to perform back propagation in the image classifier to update the parameters of the image classifier. During the back propagation process, the model gradient is calculated using the disease classification loss, and the gradient is optimized using the gradient descent algorithm to update the weight parameters of the image correction model. When the number of training iterations reaches the specified number or the final loss function begins to converge, the training ends.

[0094] The trained image classifier can effectively identify the image of the lesion area. Therefore, for the corrected difference image after image filtering, the trained image classifier can be directly predicted and the corresponding classification recognition result can be output.

[0095] However, in some embodiments, considering that there are multiple categories or types of abnormal diseases of livestock and poultry, the classification task of the image classifier can be set as a multi-classification task, which not only needs to predict whether the corrected difference image is a lesion area image, but also needs to distinguish which type of abnormal disease it is. In this way, it is necessary to collect lesion area image samples of various types of diseases to train the image classifier, so that the image classifier can accurately determine whether it is a lesion area image, and can further determine which type of disease it belongs to, so that a more accurate classification of diseases can be achieved.

[0096] Step 104: When the classification result of the corrected difference image is a lesion region image, the lesion-like region of the livestock and poultry image to be processed is located by using the corrected difference image.

[0097] After the image classifier in step 103 classifies the corrected difference image after image filtering, the image classifier outputs a corresponding classification result, that is, determines whether the corrected difference image is a diseased area image or a healthy image.

[0098] When the classification result of the corrected differential image is a lesion area image, the lesion area of ​​the livestock and poultry image to be processed is located by the corrected differential image. Here, if the classification result is a lesion area image, it means that there is a lesion area in the livestock and poultry image to be processed, and there may be abnormal symptoms. The image classifier will also output the category or type of the corresponding abnormal symptom, that is, which abnormal symptom or disease type it belongs to. However, at this time, final recognition is still required to determine the lesion area of ​​the abnormal symptom from the livestock and poultry image to be processed. Here, according to the corresponding corrected differential image, the lesion area of ​​the abnormal symptom can be located in the livestock and poultry image to be processed. By analyzing the corrected differential image, the pixel points of the lesion area and the pixel points of the healthy area in the corrected differential image can be judged according to the division of the pixel value threshold. That is, the pixel points greater than the pixel value threshold can be determined as the pixel points of the lesion area, otherwise they are the pixel points of the healthy area.

[0099] Based on this, the pixel points of the lesion area and the pixel points of the healthy area can also be determined on the livestock and poultry image to be processed corresponding to the corrected difference map. The image area formed by the pixel points of the lesion area is the lesion area of ​​the abnormal disease, which realizes the positioning of the lesion area of ​​the disease, thereby completing the disease identification of the livestock and poultry image to be processed. Subsequently, targeted treatment can be carried out on the livestock and poultry according to the lesion area of ​​the abnormal disease.

[0100] If the classification result of the image classifier is a healthy image, it means that there is no diseased area in the corrected differential image. This also means that the livestock and poultry image to be processed is a healthy image and there is no corresponding diseased area, that is, there is no abnormal disease, so there is no need to locate the abnormal diseased area.

[0101] The embodiment of the present invention trains the image classifier by using a lesion difference map that represents the pixel difference before and after image correction, which can avoid overfitting of the classifier during the training process and improve the training effect of the classifier. The corrected difference map of the livestock and poultry image to be processed is classified by the trained image classifier, and then the abnormal symptom identification of the livestock and poultry image to be processed is realized according to the classification result. Compared with the direct image classifier to identify the symptom of the original livestock and poultry image to be processed, the possibility of interference in classification by areas unrelated to the disease can be reduced, the classification effect of the image classifier can be effectively improved, the recognition accuracy of abnormal symptoms can be improved, and the precise positioning of the lesion area can be realized.

[0102] The following is a description of the device for identifying the types of livestock and poultry disease images provided by the present invention. The device for identifying the types of livestock and poultry disease images described below and the method for identifying the types of livestock and poultry disease images described above can be referenced to each other.

[0103] like Figure 6 As shown, the device for identifying the class symptoms of livestock and poultry disease images includes an extraction module 601, a generation module 602, a determination module 603, and an identification module 604. Specifically, the extraction module 601 is used to obtain the livestock and poultry image to be processed, and call the livestock and poultry image attribute extractor based on the inverse mapping generative adversarial network to extract the appearance latent variable features of the livestock and poultry image to be processed; the generation module 602 is used to generate the appearance latent variable features through the healthy livestock and poultry image generator based on the generative adversarial network to obtain the healthy livestock and poultry image; the determination module 603 is used to determine the corrected difference map between the healthy livestock and poultry image and the livestock and poultry image to be processed, and call the image classifier to classify the corrected difference map; the identification module 604 is used to locate the class symptom lesion area of ​​the livestock and poultry image to be processed through the corrected difference map when the class classification result of the corrected difference map is a lesion area image.

[0104] It should be noted that the beneficial effects of the livestock and poultry disease image symptom identification device here and the livestock and poultry disease image symptom identification method mentioned above can correspond to each other, so the beneficial effects of the livestock and poultry disease image symptom identification device are not repeated here.

[0105] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the method for identifying the class symptoms of livestock and poultry disease images, the method comprising: obtaining the livestock and poultry image to be processed, and calling the livestock and poultry image attribute extractor based on the inverse mapping generative adversarial network to extract the appearance latent variable features of the livestock and poultry image to be processed; generating the appearance latent variable features through the healthy livestock and poultry image generator based on the generative adversarial network to obtain the healthy livestock and poultry image; determining the corrected difference map between the healthy livestock and poultry image and the livestock and poultry image to be processed, and calling the image classifier to classify the corrected difference map; when the corrected difference map is classified as a lesion area image, locating the class lesion area of ​​the livestock and poultry image to be processed through the corrected difference map.

[0106] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0107] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the livestock and poultry disease image classification and symptom identification method provided by the above-mentioned methods, and the method includes: obtaining a livestock and poultry image to be processed, and calling a livestock and poultry image attribute extractor based on an inverse mapping generative adversarial network to extract the appearance latent variable features of the livestock and poultry image to be processed; generating the appearance latent variable features through a healthy livestock and poultry image generator based on a generative adversarial network to obtain a healthy livestock and poultry image; determining a corrected difference image between the healthy livestock and poultry image and the livestock and poultry image to be processed, and calling an image classifier to classify the corrected difference image; when the classification result of the corrected difference image is a lesion area image, locating the symptom-class lesion area of ​​the livestock and poultry image to be processed through the corrected difference image.

[0108] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the livestock and poultry disease image classification and symptom identification method provided by the above-mentioned methods, the method comprising: obtaining a livestock and poultry image to be processed, and calling a livestock and poultry image attribute extractor based on an inverse mapping generative adversarial network to extract the appearance latent variable features of the livestock and poultry image to be processed; generating the appearance latent variable features by a healthy livestock and poultry image generator based on a generative adversarial network to obtain a healthy livestock and poultry image; determining a corrected difference image between the healthy livestock and poultry image and the livestock and poultry image to be processed, and calling an image classifier to classify the corrected difference image; when the classification result of the corrected difference image is a lesion area image, locating the symptom-class lesion area of ​​the livestock and poultry image to be processed by using the corrected difference image.

[0109] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0110] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying disease types in livestock and poultry disease images, characterized in that: The method comprises: Acquire a livestock image to be processed, and call a livestock image attribute extractor to extract appearance latent variable features of the livestock image to be processed, wherein the livestock image attribute extractor is a network model using a generative adversarial network inverse mapping technology; The appearance latent variable features are generated by a healthy livestock and poultry image generator to obtain a healthy livestock and poultry image, wherein the healthy livestock and poultry image generator is an image generator based on a generative adversarial network; Determining a corrected difference image between the healthy livestock and poultry image and the livestock and poultry image to be processed, and calling an image classifier to classify the corrected difference image; When the classification result of the corrected difference image is a lesion region image, the lesion-like region of the livestock and poultry image to be processed is located by using the corrected difference image.

2. The method for identifying animal and poultry disease images according to claim 1, characterized in that: The livestock and poultry image attribute extractor and the healthy livestock and poultry image generator are connected in series, the network layer structures of the livestock and poultry image attribute extractor and the healthy livestock and poultry image generator are symmetrical, and in the livestock and poultry image attribute extractor and the healthy livestock and poultry image generator, at least one pair of symmetrical network layers are connected through a skip connection module, and the processing process of the skip connection module includes: Obtaining a feature map output by a network layer of the livestock and poultry image attribute extractor; The feature map is subjected to low-pass filtering, and the feature map after the low-pass filtering is input into the network layer of the healthy livestock and poultry image generator.

3. The method for identifying animal and poultry disease images according to claim 1, characterized in that: The image classifier is trained by collecting lesion area image samples, the lesion area image samples carry real disease labels, and the training process of the image classifier is: Acquire livestock and poultry disease images, and perform correction processing on the livestock and poultry disease images to obtain corresponding disease image samples; Determine a lesion difference map between the livestock and poultry disease image and the disease image sample, and extract a lesion area image from the lesion difference map as a lesion area image sample; Inputting the lesion area image sample into the image classifier for forward propagation to obtain a disease prediction label; A disease classification loss is constructed based on the disease true label and the disease predicted label, and back-propagation is performed in the image classifier through the disease classification loss to update the parameters of the image classifier.

4. The method for identifying animal and poultry disease images according to claim 3, characterized in that: The step of extracting a lesion area image from the lesion difference image comprises: Preset pixel value threshold; For each pixel point in the lesion difference image, when the threshold of the pixel point is greater than the pixel value threshold, determining the pixel point as the first pixel point of the abnormal symptom area; When the threshold of the pixel point is less than or equal to the pixel value threshold, determining that the pixel point is a second pixel point in the normal area; A lesion area image is determined based on the first pixel point and the second pixel point.

5. A method for identifying disease types in livestock and poultry disease images, characterized in that: The method comprises: Acquire images of livestock and poultry to be processed; Calling the image correction model to correct the livestock and poultry image to be processed to obtain a healthy livestock and poultry image; Determining a corrected difference image between the healthy livestock and poultry image and the livestock and poultry image to be processed, and calling an image classifier to classify the corrected difference image; When the classification result of the corrected difference image is a lesion region image, locating the lesion-like region of the livestock and poultry image to be processed by using the corrected difference image; The image correction model includes an image attribute extractor, an image generator and an image discriminator. The training process of the image correction model includes: Acquire livestock and poultry image samples, and input the livestock and poultry image samples into the image correction model for forward propagation to obtain a corrected image; Constructing a total loss function of the image attribute extractor and the image discriminator according to the livestock and poultry image sample and the corrected image; Determine the Wasserstein distance between the true probability distribution of the livestock and poultry image sample and the generated probability distribution of the corrected image, and use the Wasserstein distance as a generation loss function of the image generator; The sum of the total loss function and the generated loss function is used as the final loss function of the image correction model; The final loss function is used to perform back propagation in the image correction model to update the parameters of the image correction model.

6. The method for identifying animal and poultry disease images according to claim 5, characterized in that: The total loss function of the image attribute extractor and the image discriminator is constructed according to the livestock and poultry image sample and the corrected image, including: Determine a first discriminant value of the image discriminator for the livestock and poultry image sample and a second discriminant value of the image discriminator for the corrected image, and construct a discriminant loss function of the image discriminator according to a difference between the first discriminant value and the second discriminant value; Determine the mean square error between the livestock and poultry image sample and the corrected image, and use the mean square error as a reconstruction loss function of the image attribute extractor when reconstructing the image; Preset reconstruction weights and discrimination weights; The reconstruction loss function is weighted by the reconstruction weight to obtain a reconstruction weighted result, and the discrimination loss function is weighted by the discrimination weight to obtain a discrimination weighted result; The sum of the reconstruction weighted result and the discrimination weighted result is used as the total loss function of the image attribute extractor and the image discriminator.

7. A device for identifying disease images of livestock and poultry, characterized in that: The device comprises: An extraction module is used to obtain the livestock and poultry image to be processed, and call the livestock and poultry image attribute extractor to extract the appearance latent variable features of the livestock and poultry image to be processed, wherein the livestock and poultry image attribute extractor is a network model using a generative adversarial network inverse mapping technology; A generation module, used to generate the appearance latent variable features through a healthy livestock and poultry image generator to obtain a healthy livestock and poultry image, wherein the healthy livestock and poultry image generator is an image generator based on a generative adversarial network; A determination module, used to determine a corrected difference image between the healthy livestock and poultry image and the livestock and poultry image to be processed, and call an image classifier to classify the corrected difference image; The identification module is used to locate the disease-like disease area of ​​the livestock and poultry image to be processed through the corrected difference image when the classification result of the corrected difference image is a disease area image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for identifying livestock and poultry disease image categories as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying livestock and poultry disease image categories as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for identifying livestock and poultry disease image categories as described in any one of claims 1 to 6 is implemented.

Images