Pig farm stain monitoring image decontamination method, device and electronic equipment

Abstract

The present invention provides a method, device and electronic device for decontamination of stain monitoring images of pig farms, the method comprising: obtaining a stain image to be decontaminated; inputting the stain image into an image generator to obtain a decontamination image corresponding to the stain image output by the image generator; wherein the image generator is obtained by training based on a sample non-stain image, a sample stain image corresponding to the sample non-stain image and an image discriminator on the basis of introducing an attention mechanism, the image generator and the image discriminator constitute a generative adversarial network, and the image discriminator is used to distinguish the decontamination image generated by the image generator from the sample non-stain image. The present invention can effectively realize the removal of stain noise of stain images of pig farm monitoring videos, has high generalization ability and robustness, improves the video monitoring capability of the farm, reduces the risk of human-animal contact and artificial stress, and is conducive to effectively ensuring the safe production of the farm.

Description

Technical Field

[0001] The present invention relates to the technical field of animal husbandry, and in particular to a method, a device and an electronic device for decontaminating stain monitoring images of a pig farm. Background Art

[0002] In recent years, with the rapid development of animal husbandry informatization, efficient and environmentally friendly management mechanisms have been increasingly valued in the industry, and all pig farms have been equipped with new monitoring equipment. In the complex breeding environment of livestock and poultry farms, video monitoring can help farmers effectively prevent pigs from jumping over fences, biting, fighting and trampling each other, and other animals from entering the farm. At the same time, when monitoring the breeding situation, it can also timely grasp the estrus period of pigs, facilitate accurate control of pig breeding, and be conducive to successful pig breeding and reduce abortion.

[0003] Remote video monitoring systems can help farmers, especially those engaged in large-scale farming, reduce costs and enhance farm management, while also reducing farming risks. However, in the complex farming environment of pig farms, monitoring lenses are often easily contaminated by water stains, flies, insects, feces and other stains, resulting in spots on the captured video images, incomplete information on the monitoring screen and missing important information.

[0004] At present, in order to ensure the clear monitoring images of pig farms, the monitoring cameras are usually wiped manually to remove dirt. However, the above traditional treatment method is not only cumbersome and time-consuming, but also requires cleaning staff to go deep into the breeding house, which brings certain risks to the biological prevention of individual livestock and poultry, and may also cause biological stress to livestock, posing a threat to the personal safety of cleaning staff.

[0005] Therefore, how to better decontaminate the stained images taken by contaminated surveillance cameras in pig farms has become a technical problem that needs to be urgently solved in the industry. Summary of the invention

[0006] The present invention provides a method, a device and an electronic device for decontaminating a stained monitoring image of a pig farm, which are used for better decontaminating a stained image taken by a contaminated monitoring lens in a pig farm.

[0007] The present invention provides a method for decontaminating a stain monitoring image of a pig farm, comprising:

[0008] Acquire a stain image to be decontaminated;

[0009] Inputting the stain image into an image generator to obtain a decontamination image corresponding to the stain image output by the image generator;

[0010] Among them, the image generator is trained based on the introduction of the attention mechanism, based on the sample clean image, the sample stain image corresponding to the sample clean image and the image discriminator. The image generator and the image discriminator constitute a generative adversarial network, and the image discriminator is used to distinguish the decontamination image generated by the image generator from the sample clean image.

[0011] According to a method for decontamination of a pig farm stain monitoring image provided by the present invention, the image generator includes an attention generation module and an image encoding module;

[0012] The attention generation module is used to extract image stain attention features from the sample stain image based on an attention mechanism according to a pixel difference between the sample stain-free image and a sample stain image corresponding to the sample stain-free image;

[0013] The image encoding module is used to encode the sample stain image based on the image stain attention feature to obtain a decontamination image corresponding to the sample stain image.

[0014] According to a method for decontaminating a stain monitoring image of a pig farm provided by the present invention, before inputting the stain image into an image generator, the method further comprises:

[0015] Acquire multiple groups of sample data sets; wherein each group of the sample data sets includes the sample clean image and a sample stain image corresponding to the sample clean image; the sample clean image in each group of the sample data sets is the same real clean image, the sample stain image includes multiple types of stain images, and each type of stain image includes multiple images containing different stain noises;

[0016] Performing image calibration on the sample clean images and the sample stain images in each group of the sample data groups, respectively, to obtain calibrated sample clean images and calibrated sample stain images in each group of the sample data groups;

[0017] Performing pixel difference processing on the calibrated sample clean image and the calibrated sample stain image in each group of the sample data group to obtain a pixel difference point matrix corresponding to each group of the sample data group;

[0018] The generative adversarial network is trained using each group of the sample data groups and the pixel difference point matrix corresponding to each group of the sample data groups, and when the preset training conditions are met, a trained image generator is obtained.

[0019] According to a method for decontamination of a pig farm stain monitoring image provided by the present invention, the generative adversarial network is trained using each group of sample data groups and the pixel difference point matrix corresponding to each group of sample data groups, and when the preset training conditions are met, a trained image generator is obtained, including:

[0020] For any set of sample data groups, inputting the calibrated sample stain image in the sample data group and the pixel difference point matrix corresponding to the sample data group into the attention generation module of the image generator in the generative adversarial network to obtain an attention image;

[0021] Inputting the calibrated sample stain image and the attention image in the sample data group into the image encoding module of the image generator to obtain a decontamination image corresponding to the sample stain image;

[0022] Inputting the decontamination image, the attention image and the calibrated sample non-contamination image in the sample data set into the image discriminator in the generative adversarial network, and outputting the probability that the decontamination image is a true decontamination image;

[0023] When it is determined that the current number of training rounds reaches a preset maximum number, a trained image generator is obtained.

[0024] According to a method for removing stains from monitoring images of pig farms provided by the present invention, the image discriminator is trained based on the following loss function:

[0025] Loss decoder =-log(D(R))-log(1-D(G(O)));

[0026] Among them, Loss decoder represents the total loss of the image discriminator, D(R) represents the network output value of the real non-contaminated image after passing through the image discriminator, and D(G(O)) represents the network output value of the decontaminated image generated by the image generator after passing through the image discriminator.

[0027] According to a method for removing stains from a pig farm monitoring image provided by the present invention, the image generator is trained based on the following loss function:

[0028]

[0029]

[0030] Loss out =Loss att +Loss enconder ;

[0031]

[0032] Among them, Loss out Represents the total loss of the image generator, the Loss att Represents the loss of the attention generation module, Loss encoder represents the loss of the image coding module, Represents the predicted value, y i represents the true value, A t represents the learned attention map, M represents the pixel difference matrix, and compares the effect of the learned attention module. represents the weight, which is a value between 0 and 1; t represents the number of training rounds, i represents the order of sample image input; E i represents the image generated by the image encoding module, R i Represents a true and uncontaminated image.

[0033] The present invention also provides a stain image decontamination device, comprising:

[0034] An acquisition module, used for acquiring a stain image to be decontaminated;

[0035] A decontamination module, used for inputting the stain image into an image generator to obtain a decontamination image corresponding to the stain image output by the image generator;

[0036] Among them, the image generator is trained based on the introduction of the attention mechanism, based on the sample clean image, the sample stain image corresponding to the sample clean image and the image discriminator. The image generator and the image discriminator constitute a generative adversarial network, and the image discriminator is used to distinguish the decontamination image generated by the image generator from the sample clean image.

[0037] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for decontaminating stain monitoring images of a pig farm as described in any one of the above methods is implemented.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for decontaminating pig farm stain monitoring images.

[0039] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for removing contamination from stain monitoring images of pig farms.

[0040] The method, device and electronic device for decontaminating stain monitoring images of pig farms provided by the present invention use a generative adversarial network composed of an image generator and an image discriminator as the main body, combine the advantages of self-attention mechanism and convolutional neural network in multi-scale feature extraction and fusion, use sample non-staining images, sample stain images corresponding to sample non-staining images and image discriminators to train the image generator, the image discriminator is used to distinguish the decontamination image generated by the image generator from the sample non-staining image, thereby forming an attention-guided image generator and image discriminator, the decontamination image restored from the image generator is clear and natural, and is highly similar to the real non-staining image, can effectively realize the removal of stain noise of stain images of pig farm monitoring videos, has high generalization ability and robustness, improves the video monitoring capability of pig farms, reduces the risk of human-animal contact and artificial stress, and is conducive to effectively ensuring the safe production of farms. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 It is a schematic flow chart of the method for decontamination of stain monitoring images of pig farms provided by the present invention;

[0043] Figure 2 It is a schematic diagram of the network structure of the attention generation module in the method for decontamination of stain monitoring images of pig farms provided by the present invention;

[0044] Figure 3 It is a schematic diagram of the network structure of the image encoding module in the method for decontamination of stain monitoring images of pig farms provided by the present invention;

[0045] Figure 4 It is a schematic diagram of the network structure of the image discriminator in the method for decontamination of stain monitoring images of pig farms provided by the present invention;

[0046] Figure 5 It is a schematic structural diagram of a stain image decontamination device provided by the present invention;

[0047] Figure 6 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] In the daily production process of livestock and poultry farming, images bear the function of information dissemination. Therefore, the quality of images plays a vital role in the efficiency of information dissemination. In the production and breeding process, the image degradation phenomenon of images taken by monitoring equipment is mostly caused by various environmental noises in the breeding environment, such as the monitoring lens being contaminated by fly feces, disinfectant mist, etc. Image degradation will lead to problems such as the lack of supervision focus on individual livestock and poultry and incomplete monitoring information, which seriously affects the healthy development and safe production of farms. Therefore, it is of great practical significance and application value to study the image denoising and decontamination algorithm in the breeding environment.

[0050] However, in the current actual breeding process, most of the noise and dirt removal of images taken by breeding monitoring equipment is achieved by breeding personnel regularly cleaning the monitoring lens or cleaning the monitoring equipment when necessary, which poses risks of livestock and poultry disease infection. At the same time, the processing effect of popular mainstream image processing methods such as dark channel, color gradation, and filtering is poor in low-light and low-contrast livestock and poultry breeding environments.

[0051] Deep learning is a hot research direction in the field of machine learning in recent years, and has achieved great success in computer vision, natural language processing and other fields. Image deblurring algorithms based on deep learning have also been widely used in many fields such as medical imaging, object detection and recognition, etc. The emergence of deep learning algorithms provides new ideas for solving the above problems.

[0052] In real-world application scenarios, the decontamination and denoising of monitoring images in a breeding environment has become an important issue that needs to be solved in the industry. To this end, the present invention provides a method for decontaminating monitoring images of pig farms based on a deep learning algorithm to solve the above-mentioned defects.

[0053] Combine the following Figure 1-Figure 6 The present invention describes a method, device and electronic equipment for decontamination of stain monitoring images in a pig farm.

[0054] Figure 1 FIG. 1 is a flow chart of a method for decontamination of a pig farm stain monitoring image provided by the present invention, such as Figure 1 As shown, it includes: step 110 and step 120.

[0055] Step 110, obtaining a stain image to be decontaminated;

[0056] Step 120, inputting the stain image into an image generator to obtain a decontamination image corresponding to the stain image output by the image generator;

[0057] Among them, the image generator is trained based on the sample clean image, the sample stain image corresponding to the sample clean image and the image discriminator on the basis of introducing the attention mechanism. The image generator and the image discriminator constitute a generative adversarial network. The image discriminator is used to distinguish the decontamination image generated by the image generator from the sample clean image.

[0058] Specifically, the stain image to be decontaminated described in the embodiment of the present invention may be a stain monitoring video image of livestock and poultry activities captured by monitoring equipment in a pig breeding environment and contaminated by stains such as water stains, flies and insect feces.

[0059] In the embodiment of the present invention, the sample image data can be acquired by using multiple monitoring cameras, and the multiple monitoring cameras are fixed on a camera stand to ensure that the shooting angle of each camera is consistent. Among them, the multiple monitoring cameras include a normal camera with an uncontaminated lens, and the other monitoring cameras form a certain noise shielding at different lens positions. The main noise comes from the disinfectant mist, fly excrement, and individual flies in the livestock and poultry breeding house.

[0060] Furthermore, a sample clean image can be acquired through a normal camera with a clean lens, and a sample stain image corresponding to the sample clean image can be acquired through other surveillance cameras with contaminated lenses.

[0061] It should be noted that the Generative Adversarial Network (GAN) mainly consists of two parts: the generator and the discriminator. The principle is that the generator generates simulated data and the discriminator judges the authenticity of the data. The two form an adversarial relationship, so it is called an adversarial neural network. Experiments have shown that the network formed by the adversarial relationship between networks has achieved good results in unsupervised and semi-supervised fields, which can be regarded as a self-learning process of using the network to supervise the network.

[0062] In an embodiment of the present invention, a GAN network is formed by an image generator and an image discriminator, wherein the image generator is obtained by training based on sample clean images, sample stain images corresponding to the sample clean images, and an image discriminator on the basis of introducing an attention mechanism, and by combining the advantages of the self-attention learning mechanism with the convolutional neural network in multi-scale feature extraction and fusion, self-supervised target monitoring and learning of clean images and stain images are completed.

[0063] Based on the content of the above embodiment, as an optional embodiment, the image generator includes an attention generation module and an image encoding module;

[0064] The attention generation module is used to extract image stain attention features from the sample stain image based on the attention mechanism according to the pixel difference between the sample stain-free image and the sample stain image corresponding to the sample stain-free image;

[0065] The image encoding module is used to encode the sample stain image based on the image stain attention feature to obtain the decontamination image corresponding to the sample stain image.

[0066] Specifically, in an embodiment of the present invention, by introducing an attention mechanism into an image generator, self-supervised learning of image contaminated areas can be achieved.

[0067] The image stain attention feature described in the embodiment of the present invention may specifically be a feature of occlusion points or defect points in a sample stain image that represents different pollution environments.

[0068] In an embodiment of the present invention, the attention generation module can extract the image stain attention feature from the sample stain image based on the attention mechanism and the pixel difference between the sample clean image and the sample stain image corresponding to the sample clean image, and generate the attention image, thereby learning the occlusion point or defect point features under different pollution environments by utilizing the difference between the sample clean image and the sample stain image. In this way, through the attention learning mechanism, more guidance can be given to the image encoding module, guiding the image encoding module to perform encoding operations on the image pollution part existing in the stain image.

[0069] Furthermore, the image encoding module can encode the sample stain image based on the image stain attention feature to obtain a decontamination image corresponding to the sample stain image.

[0070] It can be understood that in the embodiment of the present invention, the image generator actually includes two functions, one is to generate an attention image containing a stain discrimination area, that is, a region of interest, and the other is to generate a target decontamination image, by fusing the region of interest with the original stain-free image, thereby achieving self-calibration of the region of interest of a single image.

[0071] The method of the embodiment of the present invention introduces an attention learning mechanism in the image generator, utilizes the difference between the original clean image and the corresponding stain image, learns the characteristics of occlusion points or defect points under different pollution environments, so as to provide more guidance to the image encoding module, thereby effectively improving the quality and decontamination effect of the decontamination image generated by the image generator.

[0072] Furthermore, in an embodiment of the present invention, the image discriminator is used to distinguish the decontamination image generated by the image generator from the sample non-contamination image, so as to discriminate the authenticity of the decontamination image generated by the image generator.

[0073] In an embodiment of the present invention, by inputting an attention image containing a region of interest into an image discriminator, the image discriminator can recognize the region of interest. That is, the region of interest is used to generate a decontamination image for the image generator, and is also used to identify stain regions in the image for the image discriminator.

[0074] In an embodiment of the present invention, a GAN network is trained by using a sample clean image, a sample stain image corresponding to the sample clean image, and an image discriminator, and a trained image generator is obtained when preset conditions are met.

[0075] Afterwards, by inputting the stain video images of livestock and poultry activities shot in a breeding environment into the above-mentioned trained image generator, the decontamination image corresponding to the stain video image output by the image generator can be obtained. The obtained decontamination image is of high quality, clear and natural, and highly similar to the real stain-free image, thereby effectively realizing the removal of stain noise from the stain image of the pig farm monitoring video, ensuring that the monitoring picture is clean and clear, and the information is complete.

[0076] Of course, it is understandable that the present invention can also be applied to livestock farms such as cattle and sheep where surveillance cameras are often easily contaminated by water stains, flies, insects, feces, and other stains.

[0077] The method for decontaminating stain monitoring images of pig farms provided by the embodiment of the present invention uses a generative adversarial network composed of an image generator and an image discriminator as the main body, combines the advantages of self-attention mechanism and convolutional neural network in multi-scale feature extraction and fusion, and uses sample non-staining images, sample stain images corresponding to sample non-staining images, and image discriminators to train the image generator. The image discriminator is used to distinguish the decontamination images generated by the image generator from the sample non-staining images, thereby forming an attention-guided image generator and image discriminator. The decontamination images restored from the image generator are clear and natural, and are highly similar to the real non-staining images. The stain noise removal of stain images of pig farm monitoring videos can be effectively realized, and the method has high generalization ability and robustness, thereby improving the video monitoring capabilities of the farms, reducing the risk of human-animal contact and artificial stress, and being conducive to effectively ensuring the safe production of the farms.

[0078] Based on the content of the above embodiment, as an optional embodiment, before inputting the stain image into the image generator, the method further includes:

[0079] Acquire multiple groups of sample data sets; wherein each group of sample data sets includes a sample clean image and a sample stain image corresponding to the sample clean image; the sample clean image in each group of sample data sets is the same real clean image, the sample stain image includes multiple types of stain images, and each type of stain image includes multiple images containing different stain noises;

[0080] Performing image calibration on the sample clean images and sample stain images in each group of sample data groups respectively, to obtain the calibrated sample clean images and calibrated sample stain images in each group of sample data groups;

[0081] Perform pixel difference processing on the calibrated sample clean image and the calibrated sample stain image in each group of sample data to obtain a pixel difference point matrix corresponding to each group of sample data;

[0082] The generative adversarial network is trained using each group of sample data and the pixel difference point matrix corresponding to each group of sample data. When the preset training conditions are met, a trained image generator is obtained.

[0083] Specifically, in an embodiment of the present invention, before the stain image is input into the image generator, it is necessary to perform model training on the image generator.

[0084] In the embodiment of the present invention, it is first necessary to obtain sample data and pre-process the sample data, which may specifically include the following steps:

[0085] Step 1, obtaining multiple groups of sample data groups; wherein each group of sample data groups includes a sample clean image and a sample stain image corresponding to the sample clean image; the sample clean image in each group of sample data groups is the same real clean image, and the sample stain image includes multiple types of stain images, and the stain noise between each type of stain images is different.

[0086] In a specific embodiment, according to the same shooting angle, four monitoring cameras are fixed by camera stands, one of which has a normal lens without pollution, and the other three have lenses with livestock and poultry scene pollution. The three polluted lenses form certain noise shielding at different positions of the lenses. A sample clean image, i.e., a real clean image, can be obtained through a normal lens without pollution; three types of stained images with different stain noises can be obtained through three camera lenses with livestock and poultry scene pollution, i.e., three types of sample stained images can be obtained. A sample clean image and its corresponding sample stained image are respectively combined to form a sample data group, thereby obtaining multiple groups of sample data groups.

[0087] In an embodiment of the present invention, the step of preprocessing the sample data includes the following step two and step three.

[0088] Step 2: perform image calibration on the sample clean images and sample stain images in each group of sample data groups, respectively, to obtain the calibrated sample clean images and calibrated sample stain images in each group of sample data groups.

[0089] In an embodiment of the present invention, after obtaining a data set consisting of paired sample data groups, since there is a slight deviation in the shooting angles of each surveillance camera, a scale-invariant feature transform method can be used to achieve image calibration, that is, the core feature points of the two images in the sample data group are extracted using the scale-invariant feature transform (SIFT) algorithm, and then the core feature points are obtained and combined with their spatial information to form a corrected image, thereby obtaining a calibrated sample clean image and a calibrated sample stain image in each sample data group.

[0090] Assuming (x1, y1) is the coordinate of a pixel in the sample clean image, and (x2, y2) is the coordinate of the same feature point in the corresponding sample stain image, the two images can be linked in the following way:

[0091]

[0092] Where H represents the homography matrix.

[0093] The above formula can be used to implement image calibration of the sample clean image and the sample stain image in each group of sample data groups, thereby obtaining two images in the same direction.

[0094] Step three, performing pixel difference processing on the calibrated sample clean image and the calibrated sample stain image in each group of sample data, and obtaining a pixel difference point matrix corresponding to each group of sample data.

[0095] In an embodiment of the present invention, after obtaining the corrected image data set, in order to achieve difference learning between two images, it can be understood as self-learning of difference points, so as to achieve self-learning of stains and defect points. For the image effects taken by different surveillance cameras, the automatic color gradation image processing method can be used to process the image data.

[0096] The calibrated sample clean image and the calibrated sample stain image are subjected to pixel difference processing to obtain the difference points between the two images. The difference points are the monitoring points of the target self-learning. Therefore, based on the matrix composed of this type of difference points, the pixel difference point matrix M corresponding to each group of sample data groups can be obtained.

[0097] Optionally, in an embodiment of the present invention, in order to ensure the uniformity of the training data, a normalization method may be used to normalize all the above image data to obtain an image between -1 and 1.

[0098]

[0099] In the formula, p ( ' x,y) Represents the pixel value of the pixel in the original image, p(x,y) Represents the normalized pixel value.

[0100] After preprocessing the sample data, the GAN network can be trained using each group of sample data and the pixel difference point matrix corresponding to each group of sample data until it is determined that the preset training conditions are met, thereby obtaining a trained image generator.

[0101] The method of the embodiment of the present invention, before the model training of the generative adversarial network, extracts the difference points between the clean image and the stain image through data preprocessing, realizes self-learning of the characteristics of stains and defects, which is conducive to improving the effect of subsequent model training.

[0102] Based on the content of the above embodiment, as an optional embodiment, the generative adversarial network is trained using each group of sample data groups and the pixel difference point matrix corresponding to each group of sample data groups. When the preset training conditions are met, a trained image generator is obtained, including:

[0103] For any set of sample data groups, the calibrated sample stain image in the sample data group and the pixel difference point matrix corresponding to the sample data group are input into the attention generation module of the image generator in the generative adversarial network to obtain an attention image;

[0104] Inputting the calibrated sample stain image and attention image in the sample data group into the image encoding module of the image generator to obtain the decontamination image corresponding to the sample stain image;

[0105] The decontaminated image, the attention image and the calibrated sample non-contaminated image in the sample data set are input into the image discriminator in the generative adversarial network, and the probability that the decontaminated image is a true decontaminated image is output;

[0106] When it is determined that the current number of training rounds reaches a preset maximum number, a trained image generator is obtained.

[0107] Specifically, in an embodiment of the present invention, the GAN network includes an image generator and an image discriminator. The image generator is used to generate a decontaminated image, and the image discriminator is used to identify whether the decontaminated image generated by the image generator is real, that is, whether the comparison with the real non-contaminated image is close.

[0108] In an embodiment of the present invention, the image generator includes an attention generation module and an image encoding module. The main function of the attention generation module is to realize self-supervised learning of the polluted area in combination with the real clean image, and to learn the characteristics of occlusion points or defect points under different pollution environments by using the difference between the real clean image and the stain image; the image encoding module generates a defect-free image based on the stain image and the defect point characteristics extracted by the attention generation module.

[0109] Furthermore, for any set of sample data groups, the calibrated sample stain image in the sample data group and the pixel difference point matrix corresponding to the sample data group are input into the attention generation module of the image generator to obtain an attention image, which extracts the features of occlusion points or defect points under different pollution environments.

[0110] In a specific embodiment, Figure 2 As shown, the attention generation module can be composed of 8 residual network layers and LSTM network layers, and its input is a combination of a defective image and a calibrated pixel difference matrix, that is, a calibrated sample stain image in the sample data group and a pixel difference matrix corresponding to the sample data group. The output of the attention generation module is an attention image, which can be specifically represented in the form of a matrix. The size of the matrix is ​​exactly the same as the size of the image. The matrix identifies the attention of each pixel in the sample stain image, that is, the influence range of the stains and stains.

[0111] Among them, the LSTM network used in the present invention can be expressed as follows:

[0112] The input layer can be expressed as:

[0113]

[0114] The forgetting network layer can be expressed as:

[0115]

[0116]

[0117] The output layer can be expressed as:

[0118]

[0119] The long memory network layer can be expressed as:

[0120]

[0121] The short memory network layer can be expressed as:

[0122]

[0123] Among them, the Tanh function can be expressed as:

[0124]

[0125] The Sigmoid function can be expressed as:

[0126]

[0127] Among them, Xt represents the features generated by the residual network layer, C t Indicates the encoding of the state to be transferred to the next LSTM, H t Represents the output features of the LSTM unit, operator represents the convolution operation, and W represents the weight coefficient.

[0128] In this embodiment, the output result of the LSTM network is a two-dimensional matrix, whose values ​​represent the positions of the stain points in the original true non-staining image.

[0129] Furthermore, the calibrated sample stain image and attention image in the sample data group are input into the image encoding module of the image generator to obtain the decontamination image corresponding to the sample stain image.

[0130] In an embodiment of the present invention, the image encoding module takes the calibrated sample stain image and the attention image as input, and outputs a stain-removed image, that is, a decontamination image corresponding to the sample stain image.

[0131] In a specific embodiment, Figure 3 As shown in the figure, the structure of the image coding module may include 24 convolutional layers, 12 hole convolutional layers and 6 transposed convolutional layers. Each layer has a corresponding activation layer after the corresponding convolution operation to ensure the output data. At the same time, the corresponding second layer, fourth layer, sixth layer, eighth layer, tenth layer, and twelfth layer have corresponding jump links to the corresponding transposed convolutional layers. The linking method is to sum after the operation to prevent the output of blurred volume.

[0132] Based on the content of the above embodiment, as an optional embodiment, the image generator is trained based on the following loss function:

[0133]

[0134]

[0135] Loss out =Loss att +Loss enconder ;

[0136]

[0137] Among them, Loss out Represents the total loss of the image generator, Loss att Represents the loss of the attention generation module, Loss ancoder represents the loss of the image encoding module, Represents the predicted value, y i represents the true value, A trepresents the learned attention map, M represents the pixel difference matrix, and compares the effect of the learned attention module. represents the weight, which is a value between 0 and 1; t represents the number of training rounds, i represents the order of sample image input; E i represents the image generated by the image encoding module, R i Represents a true and uncontaminated image.

[0138] Specifically, in an embodiment of the present invention, the loss function used in the image generator training includes two parts, one is the attention generation module part, and the other is the image encoding module part. The loss function of the attention generation module part is the mean square error between the image output by the residual network and the LSTM network and the pixel difference point matrix image, and N represents the number of training rounds, and the later the number of training rounds, the greater the impact.

[0139] The loss function of the image encoding module is the mean square error between the encoded image and the true clean image. Therefore, the overall loss of the image generator is the sum of the loss of the attention generation module and the loss of the image encoding module.

[0140] In this embodiment, the Lrelu function can be expressed as:

[0141]

[0142] Among them, a represents the preset coefficient, which is a very small value, which can be taken as 0.02 to ensure that the network is nonlinearly activated.

[0143] The loss of the attention generation module can be expressed as:

[0144]

[0145] in, Represents the weight. In order to ensure that the content learned in the later stage accounts for a larger proportion, it is a value between 0 and 1. The higher the number of training rounds, the higher the proportion.

[0146] The loss of the image encoding module can be expressed as:

[0147]

[0148] In an embodiment of the present invention, the total loss of the image generator can be expressed as:

[0149] Loss out =Loss att +Loss enconder ;

[0150] In this embodiment, the greater the number of training rounds, the closer the total loss of the image generator will be to a convergence state, and the quality of the decontamination image output by the trained image generator will also be higher.

[0151] The method of the embodiment of the present invention controls the loss value of the image generator within a preset range by training the image generator in the generative adversarial network, thereby facilitating improving the quality of the decontamination image output by the image generator.

[0152] Furthermore, the decontamination image, the attention image and the calibrated sample non-contamination image in the sample data group are input into the image discriminator in the generative adversarial network, and the authenticity of the decontamination image is judged by the image discriminator, and the probability that the decontamination image is a true decontamination image is output.

[0153] In an embodiment of the present invention, Figure 4 As shown in the figure, the image discriminator is used to identify the authenticity of the output content of the image generator. It can specifically include 15 network layers, including 12 convolutional layers and three fully connected layers. Finally, the output is distributed between 0 and 1 through the activation function to represent the authenticity of the decontaminated image. At the same time, in order to avoid local convergence of network training, skip links can be introduced at the same time, and the input of the initial network layer is added to the input of the intermediate network layer. A new set is formed through convolution operations as the input of the next layer. Here, the attention image is introduced into the discriminator so that the discriminator can better capture local details. The overall goal is to achieve the minimum global and local loss.

[0154] Based on the content of the above embodiment, as an optional embodiment, the image discriminator is trained based on the following loss function:

[0155] Loss decoder =-log(D(R))-log(1-D(G(O)));

[0156] Among them, Loss decoder represents the total loss of the image discriminator, D(R) represents the network output value of the real non-contaminated image after passing through the image discriminator, and D(G(O)) represents the network output value of the decontaminated image generated by the image generator after passing through the image discriminator.

[0157] The method of the embodiment of the present invention ensures that the loss value of the image discriminator is controlled within a preset range by training the image discriminator in the generative adversarial network, which is beneficial to improving the accuracy of the image discriminator in determining the authenticity of the decontaminated image output by the image generator.

[0158] Furthermore, in an embodiment of the present invention, when it is determined that the current number of training rounds reaches a preset maximum number, the loss values ​​in the model training process can be controlled in a convergence state, so that a trained image generator can be obtained.

[0159] In a specific embodiment, the number of training rounds can be set to 50010 steps, which can be adjusted according to the output effect. The generator learning rate can be set to 0.0002, the image discriminator learning rate can be set to 0.001, the learning rate setting function decays, and the decay step can be set to 10000 steps.

[0160] After the training of the generative adversarial network model is completed, the model of the image generator can be saved and applied. Combined with the weights and parameters saved by the image generator, stain removal and image optimization can be achieved for the stained images taken by the contaminated monitoring lenses of the farm.

[0161] The method of the embodiment of the present invention, combined with the characteristics of the GAN network, introduces an attention learning mechanism into the image generator, and forms an attention-guided image generator and image discriminator through a model training process, thereby improving the image generator's ability to understand images, making the image generator have higher generalization ability and robustness, and greatly improving the quality and decontamination effect of the decontamination image output by the image generator.

[0162] The following is a description of the pig farm stain image decontamination device provided by the present invention. The pig farm stain image decontamination device described below and the pig farm stain monitoring image decontamination method described above can be referenced to each other.

[0163] Figure 5 Schematic diagram of the structure of the stain image removal device for pig farms provided by the present invention. Figure 5 As shown, including:

[0164] An acquisition module 510 is used to acquire a stain image to be decontaminated;

[0165] A decontamination module 520, used for inputting the stain image into the image generator to obtain a decontamination image corresponding to the stain image output by the image generator;

[0166] Among them, the image generator is trained based on the sample clean image, the sample stain image corresponding to the sample clean image and the image discriminator on the basis of introducing the attention mechanism. The image generator and the image discriminator constitute a generative adversarial network. The image discriminator is used to distinguish the decontamination image generated by the image generator from the sample clean image.

[0167] The stain image decontamination device described in this embodiment can be used to execute the above-mentioned pig farm stain monitoring image decontamination method embodiment, and its principles and technical effects are similar and will not be repeated here.

[0168] The device for removing stain images from pig farms provided by the present invention uses a generative adversarial network composed of an image generator and an image discriminator as the main body, combines the advantages of a self-attention mechanism and a convolutional neural network in multi-scale feature extraction and fusion, and uses a sample non-staining image, a sample stain image corresponding to the sample non-staining image, and an image discriminator to train the image generator. The image discriminator is used to distinguish between the decontaminated image generated by the image generator and the sample non-staining image, thereby forming an attention-guided image generator and image discriminator. The decontaminated image restored from the image generator is clear and natural, and is highly similar to the real non-staining image. The device can effectively remove stain noise from stain images in pig farm monitoring videos, has high generalization ability and robustness, improves the video monitoring capability of the farm, reduces the risk of human-animal contact and artificial stress, and is conducive to effectively ensuring the safe production of the farm.

[0169] Figure 6 is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the method for decontamination of the stain monitoring image of the pig farm provided by the above methods, the method comprising: obtaining a stain image to be decontaminated; inputting the stain image into an image generator to obtain a decontamination image corresponding to the stain image output by the image generator; wherein the image generator is obtained by training based on a sample non-stain image, a sample stain image corresponding to the sample non-stain image and the image discriminator on the basis of introducing an attention mechanism, the image generator and the image discriminator constitute a generative adversarial network, and the image discriminator is used to distinguish the decontamination image generated by the image generator from the sample non-stain image.

[0170] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0171] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the pig farm stain monitoring image decontamination method provided by the above methods, the method including: obtaining a stain image to be decontaminated; inputting the stain image into an image generator to obtain a decontamination image corresponding to the stain image output by the image generator; wherein the image generator is trained based on a sample clean image, a sample stain image corresponding to the sample clean image, and the image discriminator on the basis of introducing an attention mechanism, and the image generator and the image discriminator constitute a generative adversarial network, and the image discriminator is used to distinguish the decontamination image generated by the image generator from the sample clean image.

[0172] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for decontaminating pig farm stain monitoring images provided by the above-mentioned methods, the method comprising: obtaining a stain image to be decontaminated; inputting the stain image into an image generator to obtain a decontamination image corresponding to the stain image output by the image generator; wherein the image generator is trained based on a sample clean image, a sample stain image corresponding to the sample clean image, and the image discriminator on the basis of introducing an attention mechanism, the image generator and the image discriminator forming a generative adversarial network, and the image discriminator is used to distinguish between the decontamination image generated by the image generator and the sample clean image.

[0173] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0174] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for decontamination of stain monitoring images in pig farms, It is characterized in that include: Obtaining a stain image to be decontaminated; the stain image includes a stain monitoring video image of livestock and poultry activities captured by monitoring equipment contaminated by water stains and fly and insect feces stains in a pig farm environment; Inputting the stain image into an image generator to obtain a decontamination image corresponding to the stain image output by the image generator; The image generator is obtained by training based on a sample clean image, a sample stain image corresponding to the sample clean image, and the image discriminator on the basis of introducing an attention mechanism. The image generator and the image discriminator constitute a generative adversarial network. The image discriminator is used to distinguish between the decontamination image generated by the image generator and the sample clean image. The attention mechanism is used to learn the occlusion point features or defect point features in the monitoring image under the pig farm environment based on the difference between the original clean image and the corresponding stain image. The image generator includes an attention generation module and an image encoding module; The attention generation module is used to extract image stain attention features from the sample stain image based on an attention mechanism according to a pixel difference between the sample stain-free image and a sample stain image corresponding to the sample stain-free image; The image encoding module is used to encode the sample stain image based on the image stain attention feature to obtain a decontamination image corresponding to the sample stain image; Before inputting the stain image into the image generator, the method further comprises: Acquire multiple groups of sample data sets; wherein each group of the sample data sets includes the sample clean image and a sample stain image corresponding to the sample clean image; the sample clean image in each group of the sample data sets is the same real clean image, the sample stain image includes multiple types of stain images, and each type of stain image includes multiple images containing different stain noises; Performing image calibration on the sample clean images and the sample stain images in each group of the sample data groups, respectively, to obtain the calibrated sample clean images and calibrated sample stain images in each group of the sample data groups; performing pixel difference processing on the calibrated sample clean images and calibrated sample stain images in each group of the sample data groups, to obtain a pixel difference point matrix corresponding to each group of the sample data groups; The generative adversarial network is trained using each group of the sample data groups and the pixel difference point matrix corresponding to each group of the sample data groups, and when the preset training conditions are met, a trained image generator is obtained; The performing image calibration on the sample clean image and the sample stain image in each group of the sample data group to obtain the calibrated sample clean image and the calibrated sample stain image in each group of the sample data group comprises: The scale-invariant feature transformation method is used to achieve image calibration, that is, the core feature points of the two images in the sample data group are extracted using the scale-invariant feature transformation algorithm, and then the core feature points are obtained and combined with their spatial information to form a corrected image, so as to obtain the calibrated sample clean image and the calibrated sample stain image in each group of sample data.

2. The method for decontaminating a pig farm stain monitoring image according to claim 1, It is characterized in that The method of training the generative adversarial network using each group of the sample data groups and the pixel difference point matrix corresponding to each group of the sample data groups, and obtaining a trained image generator when a preset training condition is met, comprises: For any set of sample data groups, inputting the calibrated sample stain image in the sample data group and the pixel difference point matrix corresponding to the sample data group into the attention generation module of the image generator in the generative adversarial network to obtain an attention image; Inputting the calibrated sample stain image and the attention image in the sample data group into the image encoding module of the image generator to obtain a decontamination image corresponding to the sample stain image; Inputting the decontamination image, the attention image and the calibrated sample non-contamination image in the sample data set into the image discriminator in the generative adversarial network, and outputting the probability that the decontamination image is a true decontamination image; When it is determined that the current number of training rounds reaches a preset maximum number, a trained image generator is obtained.

3. The method for decontaminating a pig farm stain monitoring image according to any one of claims 1 to 2, It is characterized in that The image discriminator is trained based on the following loss function: Loss decoder =-log(D(R))-log(1-D(G(O))); Among them, Loss decoder represents the total loss of the image discriminator, D(R) represents the network output value of the real non-contaminated image after passing through the image discriminator, and D(G(O)) represents the network output value of the decontaminated image generated by the image generator after passing through the image discriminator.

4. The method for decontaminating a pig farm stain monitoring image according to any one of claims 1 to 2, It is characterized in that The image generator is trained based on the following loss function: Loss out =Loss att +Loss encoder ; Among them, Loss out Represents the total loss of the image generator, the Loss att Represents the loss of the attention generation module, Loss encoder represents the loss of the image coding module, Represents the predicted value, y i represents the true value, A t represents the learned attention map, M represents the pixel difference matrix, and compares the effect of the learned attention module. represents the weight, which is a value between 0 and 1; t represents the number of training rounds, i represents the order of sample image input; E i represents the image generated by the image encoding module, R i Represents a true and uncontaminated image.

5. A stain image decontamination device, It is characterized in that include: An acquisition module is used to acquire a stain image to be decontaminated; the stain image includes a stain monitoring video image of livestock and poultry activities captured by monitoring equipment contaminated by water stains and fly and insect feces stains in a pig farm environment; A decontamination module, used for inputting the stain image into an image generator to obtain a decontamination image corresponding to the stain image output by the image generator; The image generator is obtained by training based on a sample clean image, a sample stain image corresponding to the sample clean image, and the image discriminator on the basis of introducing an attention mechanism. The image generator and the image discriminator constitute a generative adversarial network. The image discriminator is used to distinguish between the decontamination image generated by the image generator and the sample clean image. The attention mechanism is used to learn the occlusion point features or defect point features in the monitoring image under the pig farm environment based on the difference between the original clean image and the corresponding stain image. The image generator includes an attention generation module and an image encoding module; The attention generation module is used to extract image stain attention features from the sample stain image based on an attention mechanism according to a pixel difference between the sample stain-free image and a sample stain image corresponding to the sample stain-free image; The image encoding module is used to encode the sample stain image based on the image stain attention feature to obtain a decontamination image corresponding to the sample stain image; Before inputting the stain image into the image generator, the method further comprises: Acquire multiple groups of sample data sets; wherein each group of the sample data sets includes the sample clean image and a sample stain image corresponding to the sample clean image; the sample clean image in each group of the sample data sets is the same real clean image, the sample stain image includes multiple types of stain images, and each type of stain image includes multiple images containing different stain noises; Performing image calibration on the sample clean images and the sample stain images in each group of the sample data groups, respectively, to obtain calibrated sample clean images and calibrated sample stain images in each group of the sample data groups; Performing pixel difference processing on the calibrated sample clean image and the calibrated sample stain image in each group of the sample data group to obtain a pixel difference point matrix corresponding to each group of the sample data group; The generative adversarial network is trained using each group of the sample data groups and the pixel difference point matrix corresponding to each group of the sample data groups, and when the preset training conditions are met, a trained image generator is obtained; The performing image calibration on the sample clean image and the sample stain image in each group of the sample data group to obtain the calibrated sample clean image and the calibrated sample stain image in each group of the sample data group comprises: The scale-invariant feature transformation method is used to achieve image calibration, that is, the core feature points of the two images in the sample data group are extracted using the scale-invariant feature transformation algorithm, and then the core feature points are obtained and combined with their spatial information to form a corrected image, so as to obtain the calibrated sample clean image and the calibrated sample stain image in each group of sample data.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the method for removing contamination from the stain monitoring image of a pig farm as described in any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method for removing contamination from the stain monitoring images of a pig farm as described in any one of claims 1 to 4 is implemented.

8. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the method for removing contamination from the stain monitoring images of a pig farm as described in any one of claims 1 to 4 is implemented.

Images