Pig cleanliness intelligent evaluation method and system based on hog house inspection robot

Abstract

The invention relates to the technical field of image processing and analysis, in particular to an intelligent pig cleanliness evaluation method and system based on a pig house inspection robot. The method comprises the steps of obtaining historical live pig images of live pigs in a pig house area; performing restoration processing on each historical live pig image to obtain each target live pig image; training the initial cleanliness scoring model based on each target live pig image to obtain a target cleanliness scoring model; inputting the obtained current live pig images into a target cleanliness scoring model to obtain cleanliness scores corresponding to the current live pig images; and obtaining a current pig cleanliness score of the pig house area based on the cleanliness score corresponding to each current pig image. Therefore, the pig cleanliness of the pig house area can be automatically evaluated, manual inspection is replaced, time and labor are saved, and more convenience is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and analysis, and particularly relates to an intelligent evaluation method and system for the cleanliness of pigs based on a pigsty patrol robot. Background Art

[0002] Under the background that the global livestock breeding industry is facing the long-term threat of swine fever epidemic, the pig breeding mode is accelerating its transformation towards high density and airtightness. Although airtight pigsties can effectively block the spread of pathogens, the space limitation has exacerbated the problem of fecal and sewage accumulation. Pigs in a dirty environment for a long time are prone to skin inflammation, respiratory diseases and other problems, and even lead to group health risks. At the same time, the popularization of the concept of welfare-based animal breeding has put forward higher requirements for the cleanliness of pigs.

[0003] At present, the evaluation of the cleanliness of pig body surfaces mainly adopts the traditional visual evaluation method, which is time-consuming and laborious relying on manual observation.

[0004] Traditional technologies adopt single image processing technologies. For example, threshold segmentation in the HSV (Hue, Saturation, Value) color space is used to extract suspected stain areas, and then support vector machine (SVM) and histogram of oriented gradients (HOG) features are combined for cleanliness level classification. Although these methods partially replace manual judgment, there are still significant limitations in practical applications, and their main defects are concentrated in the aspects of environmental adaptability and algorithm robustness. More critically, the traditional color threshold-based segmentation algorithm is highly sensitive to light changes. Interference factors such as shadows and water stains on the pig body surface are easily misjudged as fecal stains, resulting in false positive detections; traditional algorithms are difficult to process image features in complex scenarios. The hair texture of pigs will blur the edge contour of stains, resulting in the failure of threshold segmentation; while manually designed features such as HOG lack discrimination for heterogeneous stains such as mucus and mud, and the classification accuracy drops significantly with the change of stain morphology. These technical bottlenecks severely restrict the implementation and application of the cleanliness evaluation system in real breeding scenarios. Summary of the Invention

[0005] Aiming at the deficiencies of the existing technology, the present invention proposes an intelligent evaluation method and system for the cleanliness of pigs based on a pigsty patrol robot, which improves the convenience of obtaining the cleanliness of pigs.

[0006] In a first aspect, the present invention provides an intelligent evaluation method for the cleanliness of pigs based on a pigsty inspection robot, including: obtaining historical pig images of each live pig in the pigsty area; performing restoration processing on each historical pig image to obtain each target pig image; training an initial cleanliness scoring model based on each target pig image to obtain a target cleanliness scoring model; inputting each obtained current pig image into the target cleanliness scoring model to obtain a cleanliness score corresponding to each current pig image; and obtaining the current pig cleanliness score of the pigsty area based on the cleanliness scores corresponding to each current pig image.

[0007] In one embodiment, the step of obtaining historical pig images of each live pig in the pigsty area includes: collecting a historical pigsty area image based on a pigsty inspection robot, where the historical pigsty area image includes live pigs; performing preprocessing on the historical pigsty area image to obtain a preprocessed historical pigsty area image; and performing segmentation processing on the preprocessed historical pigsty area image to obtain each historical pig image.

[0008] In one embodiment, the step of performing preprocessing on the historical pigsty area image to obtain a preprocessed historical pigsty area image includes: performing geometric transformation processing on the historical pigsty area image to obtain a geometrically transformed historical pigsty area image; performing photometric perturbation processing on the geometrically transformed historical pigsty area image to obtain a photometrically perturbed historical pigsty area image; and performing occlusion simulation processing on the photometrically perturbed historical pigsty area image to obtain the preprocessed historical pigsty area image.

[0009] In one embodiment, the step of performing photometric perturbation processing on the geometrically transformed historical pigsty area image to obtain a photometrically perturbed historical pigsty area image includes: performing offset processing on the RGB channels in the geometrically transformed historical pigsty area image based on a preset offset amount of the RGB channels to obtain the offset RGB channels; and obtaining the photometrically perturbed historical pigsty area image based on the offset RGB channels.

[0010] In one embodiment, the step of performing restoration processing on each historical pig image to obtain each target pig image includes: in response to the existence of a patch area in the historical pig image, performing matte extraction processing on the patch area in the historical pig image to obtain a matte-extracted historical pig image; and performing texture synthesis processing on the matte area of the matte-extracted historical pig image to obtain the target pig image.

[0011] In one embodiment, the step of performing restoration processing on each historical live pig image to obtain each target live pig image further includes: in response to the presence of a pollutant-covered area in the historical live pig image, performing mask covering processing on the pollutant-covered area in the historical live pig image to obtain the historical live pig image after mask covering processing; performing texture reconstruction processing on the mask-covered area of the historical live pig image after mask covering processing to obtain the target live pig image.

[0012] In one embodiment, the step of training the initial cleanliness scoring model based on each target live pig image to obtain the target cleanliness scoring model includes: obtaining the actual value of the cleanliness score of each target live pig image; training the initial cleanliness scoring model based on each target live pig image and the actual value of the cleanliness score corresponding to each target live pig image to obtain the trained initial cleanliness scoring model; obtaining the accuracy of the trained initial cleanliness scoring model; in response to the accuracy of the trained initial cleanliness scoring model being greater than the preset accuracy threshold, determining the trained initial cleanliness scoring model as the target cleanliness scoring model.

[0013] In one embodiment, the step of obtaining the current cleanliness score of the pigs in the pigsty area based on the cleanliness scores corresponding to each current live pig image includes: in response to the absence of a plaque area in the current live pig image, obtaining the total score among the cleanliness scores corresponding to each current live pig image; determining the ratio of the total score to the number of live pigs in each current live pig image as the current cleanliness score of the pigs in the pigsty area.

[0014] In one embodiment, the step of obtaining the current cleanliness score of the pigs in the pigsty area based on the cleanliness scores corresponding to each current live pig image includes: in response to the presence of a plaque area in the current live pig image, determining the plaque category according to the area of the plaque area; obtaining the current cleanliness score of the pigs in the pigsty area according to the plaque category corresponding to each current live pig image and the cleanliness score corresponding to each current live pig image.

[0015] Second aspect, the present invention provides an intelligent evaluation system for pig cleanliness based on a pigsty inspection robot, which is characterized by including: an image acquisition module for acquiring historical pig images of each live pig in the pigsty area; an abnormal image processing module for repairing each historical pig image to obtain each target pig image; a model training module for training an initial cleanliness scoring model based on each target pig image to obtain a target cleanliness scoring model; a cleanliness scoring acquisition module for inputting each acquired current pig image into the target cleanliness scoring model to obtain a cleanliness score corresponding to each current pig image; and a current pig cleanliness scoring acquisition module for obtaining the current pig cleanliness score of the pigsty area based on the cleanliness scores corresponding to each current pig image.

[0016] As can be seen from the above technical solutions, the beneficial technical effects of the present invention are as follows: It can automatically evaluate the cleanliness of pigs in the pigsty area, replacing manual inspections, saving time and effort, and being more convenient.

[0017] By obtaining the cleanliness score through the target cleanliness scoring model, on the one hand, it can be quantified according to the degree of dirt on the pig's body surface through the cleanliness score, providing a classification standard for the cleanliness of the pig's body surface. On the other hand, compared with relying on manual evaluation, the model has higher accuracy and is more stable. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to actual scale.

[0019] Figure 1 is a schematic flowchart of an exemplary embodiment of an intelligent evaluation method for pig cleanliness based on a pigsty inspection robot shown in the present application; Figure 2 is a schematic diagram of an exemplary embodiment of an image of a pigsty area shown in the present application; Figure 3 is a schematic diagram of an exemplary embodiment of adjusting the image size of a target pig image shown in the present application; Figure 4 is a schematic diagram of an exemplary embodiment of a confusion matrix shown in the present application; Figure 5 is a schematic structural diagram of an exemplary embodiment of a CEN module shown in the present application; Figure 6 is a schematic structural diagram of an exemplary embodiment of an External attention shown in the present application; Figure 7 It is a block diagram of an exemplary embodiment of a pig cleanliness intelligent evaluation system based on a pigsty inspection robot shown in the present application; Figure 8 It is a schematic structural diagram of an embodiment of an electronic device shown in the present application; Figure 9 It is a schematic structural diagram of an embodiment of a computer-readable storage medium shown in the present application. Detailed implementation manners

[0020] Hereinafter, embodiments of the technical solutions of the present invention will be described in detail with reference to the accompanying drawings. The following embodiments are only used to illustrate the technical solutions of the present invention more clearly, and thus are only examples and cannot be used to limit the protection scope of the present invention.

[0021] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in the present application should be the ordinary meanings understood by those skilled in the art to which the present invention belongs. The terms "first", "second", etc. in the description and claims of the embodiments of the present disclosure and the above drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances for the implementation of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. Unless otherwise specified, the term "plurality" means two or more. In the embodiments of the present disclosure, the character " / " indicates that the objects before and after are in an "or" relationship. For example, A / B means: A or B. The term "and / or" is a description of the association relationship of an object, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B these three relationships. The term "corresponding" may refer to an association relationship or a binding relationship. A corresponding to B means that there is an association relationship or a binding relationship between A and B.

[0022] First of all, it should be noted that in the context of the global livestock and poultry breeding industry facing the long-term threat of African swine fever epidemic, the pig breeding mode is accelerating towards high density and airtightness. Although airtight pigsties can effectively block the spread of pathogens, the space limitation has exacerbated the problem of fecal pollution accumulation. Pigs in a dirty environment for a long time are prone to skin inflammation, respiratory diseases and other problems, and even cause group health risks. At the same time, the popularization of the concept of welfare-based animal husbandry has put forward higher requirements for the cleanliness of the pigsty environment. At present, the evaluation of pig body surface cleanliness mainly adopts the traditional visual evaluation method, which relies on manual observation and is time-consuming and laborious.

[0023] Based on this, the present application provides a method and system for intelligent evaluation of pig cleanliness based on a pigsty inspection robot, which improves the convenience of obtaining pig cleanliness. The execution subject of a method for intelligent evaluation of pig cleanliness based on a pigsty inspection robot can be a terminal device, a server, or other processing devices. Among them, the terminal device can be a computer, a mobile device, a terminal, a computing device, a vehicle-mounted device, etc. The execution subject of a method for intelligent evaluation of pig cleanliness based on a pigsty inspection robot can also be a device for intelligent evaluation of pig cleanliness based on a pigsty inspection robot. In some possible implementation manners, the method for intelligent evaluation of pig cleanliness based on a pigsty inspection robot can be implemented by a processor calling computer-readable instructions stored in a memory. The execution subject of a method for intelligent evaluation of pig cleanliness based on a pigsty inspection robot can also be a big data cluster. A big data cluster is a computer system architecture formed by connecting multiple computers through a network. The big data cluster can be deployed on a private cloud constructed by K8S (Kubernetes, a container orchestration engine).

[0024] Combined with Figure 1 As shown in the figure, this embodiment provides a method for intelligent evaluation of pig cleanliness based on a pigsty inspection robot, including: Step S110: Obtain historical pig images of each live pig in the pigsty area.

[0025] The pigsty area refers to the area where live pigs are raised.

[0026] The historical pig image refers to an image containing a live pig at a historical moment.

[0027] The device for intelligent evaluation of pig cleanliness obtains historical pig images of each live pig in the pigsty area. Specifically, the device for intelligent evaluation of pig cleanliness collects images of the pigsty area through a photographing device to obtain historical pig images. Among them, the photographing device can be a visible light camera, a camera, etc.

[0028] Step S120: Perform restoration processing on each historical pig image to obtain each target pig image.

[0029] The device for intelligent evaluation of pig cleanliness performs restoration processing on each historical pig image to obtain each target pig image. Specifically, the device for intelligent evaluation of pig cleanliness removes noise in each historical pig image, repairs scratches or damaged parts in each historical pig image, and fills in missing areas in each historical pig image, one or more of them.

[0030] Step S130: Train an initial cleanliness scoring model based on each target pig image to obtain a target cleanliness scoring model.

[0031] The intelligent pig cleanliness evaluation device trains the initial cleanliness scoring model based on each target pig image to obtain the target cleanliness scoring model. Specifically, the intelligent pig cleanliness evaluation device scores each target pig image to obtain the historical cleanliness score; takes the historical cleanliness score corresponding to each target pig image as the label of each target pig image, and inputs each target pig image and the corresponding label into the initial cleanliness scoring model for machine learning to obtain the target cleanliness scoring model.

[0032] Step S140: Input each acquired current pig image into the target cleanliness scoring model to obtain the cleanliness score corresponding to each current pig image.

[0033] The current pig image refers to an image containing the current pig in the current pigsty area.

[0034] The intelligent pig cleanliness evaluation device inputs each acquired current pig image into the target cleanliness scoring model, obtains the result output by the model, and takes the result output by the model as the cleanliness score corresponding to each current pig image.

[0035] The intelligent pig cleanliness evaluation device acquires each current pig image. As an example, the intelligent pig cleanliness evaluation device collects images of each pig in the current pigsty area through a pigsty patrol robot to obtain each current pig image. As another example, the intelligent pig cleanliness evaluation device collects the current pigsty area image through a pigsty patrol robot, and the current pigsty area image includes pigs; preprocesses the current pigsty area image to obtain the preprocessed current pigsty area image; performs segmentation processing on the preprocessed current pigsty area image to obtain each current pig image.

[0036] In one embodiment, a track is installed at the top of the pigsty area, and a pigsty patrol robot is installed on the track. The intelligent pig cleanliness evaluation device intermittently collects the current pigsty area image through the pigsty patrol robot. For example, the pigsty patrol robot takes a frame of image towards the pigs directly above the pigsty area every 10 seconds within 1 minute to obtain the current pigsty area image.

[0037] It should be noted that the specific steps of preprocessing the current pigsty area image to obtain the preprocessed current pigsty area image and performing segmentation processing on the preprocessed current pigsty area image to obtain each current pig image can refer to the specific steps of preprocessing the historical pigsty area image to obtain the preprocessed historical pigsty area image and performing segmentation processing on the preprocessed historical pigsty area image to obtain each historical pig image. Details will not be repeated here.

[0038] Step S150: Obtain the current pig cleanliness score of the pigsty area based on the cleanliness scores corresponding to each current pig image.

[0039] Optionally, the step of obtaining the current pig cleanliness score of the pigsty area based on the cleanliness scores corresponding to each current pig image includes: in response to the absence of plaque areas in the current pig image, obtaining the total score among the cleanliness scores corresponding to each current pig image; and determining the ratio between the total score and the number of pigs in each current pig image as the current pig cleanliness score of the pigsty area.

[0040] For example, the cleanliness scores corresponding to each current pig image and the current pig cleanliness score satisfy the following formula: In the above formula, represents the current pig cleanliness score, n represents the number of pigs in the pigsty area, represents the cleanliness score of the i-th pig.

[0041] It can be seen that by obtaining the historical pig images of each pig in the pigsty area; performing restoration processing on each historical pig image to obtain each target pig image; training the initial cleanliness score model based on each target pig image to obtain the target cleanliness score model; inputting the obtained current pig images into the target cleanliness score model to obtain the cleanliness scores corresponding to each current pig image; and obtaining the current pig cleanliness score of the pigsty area based on the cleanliness scores corresponding to each current pig image. Thus, by using the target cleanliness score model to score the cleanliness of each current pig image, compared with the traditional method of using monitoring equipment for manual inspection of the pigsty hygiene, it does not require manual inspection, saves manpower, can timely detect dirty pigs, improves convenience, and is also beneficial to improving pig welfare.

[0042] Optionally, the step of obtaining the historical pig images of each pig in the pigsty area includes: collecting historical pigsty area images based on a pigsty patrol robot, where the historical pigsty area images include pigs; preprocessing the historical pigsty area images to obtain the preprocessed historical pigsty area images; and performing segmentation processing on the preprocessed historical pigsty area images to obtain each historical pig image.

[0043] In one embodiment, a track is installed at the top of the pigsty area, and a pigsty patrol robot is installed on the track. The pig cleanliness intelligent evaluation device collects historical pigsty area images at intervals through the pigsty patrol robot within a preset time period. For example, the pigsty patrol robot takes one frame of image directly above the pigsty area every 30 minutes from 06:00 to 18:00 every day to obtain historical pigsty area images.

[0044] Optionally, the step of preprocessing the historical pigsty area image to obtain the preprocessed historical pigsty area image includes: performing geometric deformation processing on the historical pigsty area image to obtain the geometrically deformed historical pigsty area image; performing photometric perturbation processing on the geometrically deformed historical pigsty area image to obtain the photometrically perturbed historical pigsty area image; performing occlusion simulation processing on the photometrically perturbed historical pigsty area image to obtain the preprocessed historical pigsty area image.

[0045] In one embodiment, the pig cleanliness intelligent evaluation device realizes the geometric deformation processing of the historical pigsty area image by simulating the muscle tremors of the pig body in the historical pigsty area image through elastic deformation and randomly rotating the live pigs in the historical pigsty area image, so as to obtain the geometrically deformed historical pigsty area image. By performing geometric deformation processing on the historical pigsty area image, it is beneficial to enhance the pose robustness.

[0046] Optionally, the step of performing photometric perturbation processing on the geometrically deformed historical pigsty area image to obtain the photometrically perturbed historical pigsty area image includes: performing offset processing on the RGB channels in the geometrically deformed historical pigsty area image based on the preset offsets of the RGB channels to obtain the offset RGB channels; obtaining the photometrically perturbed historical pigsty area image based on the offset RGB channels. Among them, the preset offsets are determined according to the illumination color temperature. Thus, independent offset of the RGB channels can be realized to simulate different illumination color temperatures and increase the diversity of the image.

[0047] For example, the preset offsets and the offset RGB channels satisfy the following formula: Among them, respectively represent the offset R, G, B channels, , , respectively represent the preset offsets of the R, G, B channels, clamp ( x , a , b ) represents restricting x within the range of a , b .

[0048] Optionally, the step of performing occlusion simulation processing on the historical pigsty area image after photometric perturbation processing to obtain the preprocessed historical pigsty area image includes: adding random elliptical spots to the historical pigsty area image after photometric perturbation processing to obtain the preprocessed historical pigsty area image. The occlusion simulation processing increases the diversity of the image, which is beneficial to improving the accuracy of model training.

[0049] Optionally, the step of performing segmentation processing on the preprocessed historical pigsty area image to obtain each historical live pig image includes: performing resolution unification processing on the preprocessed historical pigsty area image to obtain the historical pigsty area image with unified resolution, and inputting the historical pigsty area image with unified resolution into a preset image segmentation model to obtain each historical live pig image.

[0050] In one embodiment, the intelligent pig cleanliness evaluation device changes the image resolution of the preprocessed historical pigsty area image to 640×640 size, as Figure 2 shown, and then inputs it into a preset image segmentation model to obtain the mask coordinates of each live pig in the historical pigsty area image, and crops each historical live pig image from the historical pigsty area image according to the mask coordinates of each live pig.

[0051] Optionally, the step of performing restoration processing on each historical live pig image to obtain each target live pig image includes: in response to the existence of a patch area in the historical live pig image, performing matte extraction processing on the patch area in the historical live pig image to obtain the historical live pig image after matte extraction; performing texture synthesis processing on the matte extraction area of the historical live pig image after matte extraction to obtain the target live pig image.

[0052] The patch area can be the natural hair color patches on the pig's body surface.

[0053] In one embodiment, the intelligent pig cleanliness evaluation device determines whether there is a patch area in the historical live pig image. If so, it locates the patch area through a preset target detection model to obtain the bounding box coordinates of the patch area, subtracts the patch area from the historical live pig image according to the bounding box coordinates to obtain the historical live pig image after matte extraction, and then performs texture synthesis on the matte extraction area of the historical live pig image after matte extraction based on an image restoration model of a preset generative adversarial network (GAN, Generative Adversarial Network) to generate a pig body background image without patch interference, that is, the target live pig image. By matte extraction and texture synthesis of the patch area, the repair of the patch area is realized, which is beneficial to improving the extraction of effective features under complex interference.

[0054] Optionally, the step of performing repair processing on each historical pig image to obtain each target pig image also includes: in response to the existence of a pollutant-covered area in the historical pig image, performing mask covering processing on the pollutant-covered area in the historical pig image to obtain the historical pig image after mask covering processing; performing texture reconstruction processing on the mask-covered area of the historical pig image after mask covering processing to obtain the target pig image.

[0055] The pollutant-covered area refers to the area where pollutants such as feces and mud are covered by pig hair, which is highly hidden and difficult to identify.

[0056] In one embodiment, an intelligent pig cleanliness assessment device determines whether contaminant-covered areas exist in historical pig images. If so, it locates abnormal areas based on a preset morphological feature-spatial frequency joint detection algorithm to obtain the contaminant-covered areas. For example, the intelligent pig cleanliness assessment device uses a preset convolutional neural network (CNN) to extract the spatial frequency characteristics of pig hair texture in the target pig image. Regions with high-frequency texture abrupt changes in spatial frequency characteristics are identified as areas of hair lift caused by contaminant attachment, thus obtaining the contaminant-covered areas. A preset region growing algorithm is then used to perform connected domain analysis on the contaminant-covered areas to generate a mask of the hair regions covering the contaminants. Finally, a preset Generative Adversarial Network (GAN) model is used to reconstruct the clean pig texture of the masked areas to obtain the target pig image. By masking and reconstructing the contaminant-covered areas, the device eliminates interference caused by hair occlusion on subsequent stain detection. The training data for the preset GAN model consists of sample images of dirty pigs and those after the contaminants have been removed.

[0057] It should be noted that traditional technologies adopt single image processing techniques. For example, the HSV (Hue, Saturation, Value) color space threshold segmentation is used to extract suspected stain areas, and then the support vector machine (SVM) and the histogram of oriented gradients (HOG) features are combined for cleaning level classification. Although these methods partially replace manual judgment, there are still significant limitations in practical applications. Their main defects are concentrated in the aspects of environmental adaptability and algorithm robustness. First, the traditional color threshold-based segmentation algorithm is highly sensitive to light changes. Interference factors such as shadows and water stains on the pig body surface are easily misjudged as fecal stains, resulting in false positive detections. Second, traditional algorithms are difficult to handle image features in complex scenarios: the hair texture of pigs will blur the edge contour of stains, leading to the failure of threshold segmentation; while manually designed features such as HOG lack discrimination for heterogeneous stains such as mucus and mud, and the classification accuracy drops significantly with the change of stain morphology. These technical bottlenecks seriously restrict the implementation of the cleanliness evaluation system in real breeding scenarios.

[0058] In this application, the diversity of images is enhanced by performing geometric deformation processing on historical pigsty area images, independent offset of RGB channels to simulate different light color temperatures, and occlusion simulation processing. By cropping and texture synthesis of the patch area, the repair of the patch area is realized, which is beneficial to improving the extraction of effective features under complex interference. By masking and texture reconstruction of the pollutant coverage area, the interference of hair occlusion on subsequent stain detection is eliminated, thereby improving the accuracy of the model and solving the problem of accurate classification of the pollution degree of pigs in complex environments.

[0059] Optionally, the steps of training the initial cleanliness score model based on each target pig image to obtain the target cleanliness score model include: obtaining the actual cleanliness score value of each target pig image; training the initial cleanliness score model based on each target pig image and the actual cleanliness score value corresponding to each target pig image to obtain the trained initial cleanliness score model; obtaining the accuracy of the trained initial cleanliness score model; and in response to the accuracy of the trained initial cleanliness score model being greater than the preset accuracy threshold, determining the trained initial cleanliness score model as the target cleanliness score model.

[0060] Optionally, the step of obtaining the actual cleanliness score value of each target live pig image includes: in response to the existence of a plaque area in the target live pig image, obtaining the plaque category according to the area of the plaque area; determining the corresponding plaque area score from a preset plaque score mapping table according to the plaque category corresponding to each target live pig image, obtaining the pollution shape and the proportion of the pollution area in each target live pig image, and determining the corresponding alternative cleanliness score from a preset pollution score mapping table according to the pollution shape and the proportion of the pollution area. The preset plaque score mapping table includes the corresponding relationship between the preset pollution shape, the range of the preset pollution area and the preset cleanliness score; obtaining the third product term between the plaque area proportion value and the plaque area score in the target live pig image, and obtaining the fourth product term between the difference between the preset quantity threshold and the plaque area proportion value and the alternative cleanliness score; determining the sum between the third product term and the fourth product term as the actual cleanliness score value corresponding to the target live pig image.

[0061] Optionally, the step of determining the plaque category according to the area of the plaque area includes: obtaining the ratio between the plaque area and the area of the live pig area in the corresponding target live pig image to obtain the plaque area proportion value; determining the corresponding plaque category according to the interval range where the plaque area proportion value is located.

[0062] It should be noted that the specific steps for obtaining the plaque area score of the target live pig image can refer to the steps for obtaining the plaque area score of the current live pig area, which will not be repeated here.

[0063] The proportion of the pollution area is the ratio between the area of the pollution area and the area of the live pig in the target live pig image.

[0064]

[0065] Table 1 is an example table of the preset plaque score mapping table. As shown in Table 1, if the pollution shape is punctiform and the proportion of the punctiform pollution area is in the interval of 10% - 50%, the corresponding category is mild pollution and the corresponding cleanliness score is 4 points; if the pollution shape is punctiform and the proportion of the punctiform pollution area exceeds 50%, the corresponding category is moderate pollution and the corresponding cleanliness score is 3 points; if the pollution shape is contiguous and the proportion of the contiguous pollution area is in the interval of 10% - 50%, the corresponding category is relatively heavy pollution and the corresponding cleanliness score is 2 points.

[0066] Optionally, the step of training the initial cleanliness scoring model based on each target pig image and the actual cleanliness score value corresponding to each target pig image to obtain the trained initial cleanliness scoring model includes: formatting each target pig image to obtain each format-processed target pig image; grouping each format-processed target pig image and the corresponding actual cleanliness score value into a data set; selecting a training set from the data set to train the initial cleanliness scoring model to obtain the trained initial cleanliness scoring model.

[0067] Optionally, the step of performing format processing on each target pig image to obtain each target pig image after format processing includes: normalizing the pixels of each target pig image to obtain each normalized target pig image; performing background processing on each normalized target pig image to obtain each background-processed target pig image; performing expansion processing on each background-processed target pig image to obtain each expanded target pig image; and resizing each expanded target pig image to obtain each format-processed target pig image.

[0068] In one embodiment, the intelligent evaluation device for pig cleanliness normalizes the pixels of each target pig image, and then adjusts the non-pig area in each normalized target pig image to black, that is, the pixel value is set to zero; according to the minimum circumscribed matrix of each target pig image, the boundary is expanded by 10% as the classification input area to obtain each expanded target pig image; finally, based on the long side length of the expanded target pig image, the expanded target pig image is expanded into a square by supplementing a black background in the short side direction, that is, an image with a pixel value set to 0. For example, if the expanded target pig image is a horizontal version, that is, the long side is the width, then the black pixel area is supplemented symmetrically on the upper and lower sides in the height direction; if Figure 3 As shown in the left figure, if it is a vertical version, that is, the long side is the height, then the black pixel area is symmetrically added on the left and right sides in the width direction, and finally the image appears in a square shape.

[0069] Optionally, the step of obtaining the accuracy of the trained initial cleanliness scoring model includes: selecting a validation set from the data set, inputting images in the validation set into the trained initial cleanliness scoring model, and obtaining a cleanliness score prediction value; if the cleanliness score prediction value is the same as the actual cleanliness score value corresponding to the input image, the verification result is correct, otherwise the verification result is incorrect; and calculating the ratio between the number of correct verification results and the total number of verification results to obtain the accuracy of the trained initial cleanliness scoring model.

[0070] In one embodiment, the data set contains 28,115 images, which are divided into a training set and a validation set according to a ratio of 8:2. The accuracy of the initially trained cleanliness scoring model is obtained through the following formula: where represents the accuracy of the initially trained cleanliness scoring model, TP represents that a sample with an actual cleanliness score of A is correctly predicted as A, FP represents that a sample with an actual cleanliness score not being A is incorrectly predicted as A, FN represents that a sample with an actual cleanliness score of A is incorrectly predicted as other, and TN represents that a sample with an actual cleanliness score of other is correctly predicted as other.

[0071] In one embodiment, the initial cleanliness scoring model is trained based on each target live pig image and the actual cleanliness score corresponding to each target live pig image, and the training results are plotted as a confusion matrix. The training accuracy is 91%, and the confusion matrix of the training results is as Figure 4 shown.

[0072] Optionally, the target cleanliness scoring model includes a CNE (Conventional Neural Evolution) module. As shown in Figure 5 , the CNE module includes a convolutional module, an external attention mechanism module, and a third convolutional layer connected in sequence. The convolutional module includes a first convolutional layer with a window size of 7×7 and an input feature map channel number of 96 connected in sequence, and a second convolutional layer with a window size of 1×1 and an input feature map channel number of 384. Externalattention is the external attention mechanism module, and the window size of the third convolutional layer is 1×1 and the input feature map channel number is 96. The target cleanliness scoring model of the present application can focus on the information of different samples during training compared with a general model, and this model can pay attention to the associations between different samples, improving the model accuracy.

[0073] As shown in Figure 6 , External attention includes: feature is the input feature layer, Query is the query vector of the input feature, Norm is the normalization processing layer, and are two memory units, and output is the output. Among them, the external attention mechanism module uses and two memory units to learn the features of the entire data set, enabling the model to take into account the global important information of the data and reducing overfitting in model training.

[0074] For example, for the input feature map , the calculation process of the external attention mechanism module is as follows: In the above formula, characterizes the i th element and the matrix M the j similarity between the M th row, which is a learnable parameter independent of the input and will serve as the memory of the entire training set. A characterizes the attention map derived using the learned prior knowledge at the dataset level. N characterizes the number of pixels in the image. d characterizes the feature dimension.

[0075] Optionally, the step of obtaining the current pig cleanliness score of the pigsty area based on the cleanliness score corresponding to each current pig image includes: in response to the presence of a patch area in the current pig image, determining the patch category according to the area of the patch area; obtaining the current pig cleanliness score of the pigsty area according to the patch category corresponding to each current pig image and the cleanliness score corresponding to each current pig image.

[0076] Optionally, the step of determining the patch category according to the area of the patch area includes: obtaining the ratio between the area of the patch area and the area of the pig area in the corresponding current pig image to obtain a patch area ratio value; determining the corresponding patch category according to the interval range where the patch area ratio value is located.

[0077] Optionally, the step of obtaining the current pig cleanliness score of the pigsty area according to the patch category corresponding to each current pig image and the cleanliness score corresponding to each current pig image includes: determining the corresponding patch area score from a preset patch score mapping table according to the patch category corresponding to each current pig image, where the preset patch score mapping table includes the corresponding relationship between the patch category and the patch area score; obtaining a first product term between the patch area ratio value in the current pig image and the patch area score, and obtaining a second product term between the difference between the preset quantity threshold and the patch area ratio value and the cleanliness score; determining the sum between the first product term and the second product term as the target cleanliness score corresponding to the current pig image; taking the average value between the target cleanliness score corresponding to the current pig image with a patch area and the cleanliness score corresponding to the current pig image without a patch area as the current pig cleanliness score of the pigsty area.

[0078] For example, the patch area ratio value, the patch area score, and the cleanliness score satisfy the following formula: In the above formula, characterizes the patch area score. Characterize the cleanliness score corresponding to the current pig image after removing the plaque part of the mask. Characterize the proportion of the plaque area in the total area of the pig image. Characterize the target cleanliness score corresponding to the current pig image.

[0079]

[0080] Table 2 is an example table of the preset plaque score mapping table. As shown in Table 2, if the pollution area < 20%, the corresponding plaque category is basically clean, and the corresponding plaque area score is 5 points; if the pollution area is between 20% - 50%, the corresponding plaque category is moderately polluted, and the corresponding plaque area score is 3 points; if the proportion of the pollution area exceeds 50%, the corresponding plaque category is severely polluted, and the corresponding plaque area score is 1 point.

[0081] Combined with Figure 7 As shown, this embodiment provides a pig cleanliness intelligent evaluation system based on a pigsty patrol robot, including an image acquisition module 710, an abnormal image processing module 720, a model training module 730, a cleanliness score acquisition module 740, and a current pig cleanliness score acquisition module 750. Specifically: The image acquisition module 710 is used to acquire historical pig images of each pig in the pigsty area. The abnormal image processing module 720 is used to perform restoration processing on each historical pig image to obtain each target pig image. The model training module 730 is used to train the initial cleanliness score model based on each target pig image to obtain a target cleanliness score model. The cleanliness score acquisition module 740 is used to input each acquired current pig image into the target cleanliness score model to obtain the cleanliness score corresponding to each current pig image. The current pig cleanliness score acquisition module 750 is used to obtain the current pig cleanliness score of the pigsty area based on the cleanliness score corresponding to each current pig image.

[0082] This embodiment provides an intelligent pig cleanliness evaluation system based on a pigsty patrol robot. By obtaining historical pig images of each live pig in the pigsty area, repairing each historical pig image to obtain each target pig image, training an initial cleanliness scoring model based on each target pig image to obtain a target cleanliness scoring model, inputting each obtained current pig image into the target cleanliness scoring model to obtain the cleanliness score corresponding to each current pig image, and obtaining the current pig cleanliness score of the pigsty area based on the cleanliness scores corresponding to each current pig image. Thus, by using the target cleanliness scoring model to score the cleanliness of each current pig image, compared with the traditional method of manually checking the pigsty hygiene using monitoring equipment, there is no need for manual inspection, which saves manpower and can timely detect dirty pigs, improving convenience and also being beneficial to enhancing pig welfare.

[0083] To implement the intelligent pig cleanliness evaluation method based on the pigsty patrol robot in the above embodiment, the present application proposes another electronic device. For details, please refer to Figure 8 , Figure 8 which is a schematic structural diagram of an embodiment of the electronic device provided by the present application.

[0084] The electronic device 800 includes a memory 801 and a processor 802, where the memory 801 and the processor 802 are coupled.

[0085] The memory 801 is used to store program data, and the processor 802 is used to execute the program data to implement the intelligent pig cleanliness evaluation method based on the pigsty patrol robot in the above embodiment.

[0086] In this embodiment, the processor 802 can also be referred to as the CPU (Central Processing Unit). The processor 802 may be an integrated circuit chip with signal processing capabilities. The processor 802 can also be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor 802 can also be any conventional processor, etc.

[0087] The present application also provides a computer-readable storage medium. As Figure 9 shown, the computer-readable storage medium 900 is used to store program data 901, and when the program data 901 is executed by the processor, it is used to implement the intelligent pig cleanliness evaluation method based on the pigsty patrol robot in the method embodiment of the present application.

[0088] The method involved in the embodiment of the intelligent evaluation method for the cleanliness of pigs based on the pigsty inspection robot, when implemented and existing in the form of software functional units and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0089] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention, and they should all be covered within the scope of the claims and the description of the present invention.

Claims

1. An intelligent evaluation method for the cleanliness of pigs based on a pigsty inspection robot, characterized in that, Including: Obtaining historical pig images of each live pig in the pigsty area; Performing restoration processing on each historical pig image to obtain each target pig image; Training an initial cleanliness scoring model based on each target pig image to obtain a target cleanliness scoring model; Inputting each obtained current pig image into the target cleanliness scoring model to obtain the cleanliness score corresponding to each current pig image; Obtaining the current pig cleanliness score of the pigsty area based on the cleanliness scores corresponding to each current pig image; The step of obtaining the current pig cleanliness score of the pigsty area based on the cleanliness scores corresponding to each current pig image includes: in response to the existence of a patch area in the current pig image, determining the patch category according to the area of the patch area; obtaining the current pig cleanliness score of the pigsty area according to the patch category corresponding to each current pig image and the cleanliness score corresponding to each current pig image; The step of obtaining the current pig cleanliness score of the pigsty area according to the patch category corresponding to each current pig image and the cleanliness score corresponding to each current pig image includes: determining the corresponding patch area score from a preset patch score mapping table according to the patch category corresponding to each current pig image, where the preset patch score mapping table includes the correspondence between the patch category and the patch area score; obtaining a first product term between the patch area proportion value in the current pig image and the patch area score, and obtaining a second product term between the difference between the preset quantity threshold and the patch area proportion value and the cleanliness score; determining the sum between the first product term and the second product term as the target cleanliness score corresponding to the current pig image; taking the average value between the target cleanliness score corresponding to the current pig image with a patch area and the cleanliness score corresponding to the current pig image without a patch area as the current pig cleanliness score of the pigsty area.

2. The method according to claim 1, characterized in that, The step of obtaining historical pig images of each live pig in the pigsty area includes: Collecting historical pigsty area images based on a pigsty inspection robot, where the historical pigsty area images include live pigs; Performing preprocessing on the historical pigsty area images to obtain preprocessed historical pigsty area images; Performing segmentation processing on the preprocessed historical pigsty area images to obtain each historical pig image.

3. The method according to claim 2, wherein The step of performing preprocessing on the historical pigsty area images to obtain preprocessed historical pigsty area images includes: Performing geometric transformation processing on the historical pigsty area images to obtain geometrically transformed historical pigsty area images; Performing photometric perturbation processing on the geometrically transformed historical pigsty area images to obtain photometrically perturbed historical pigsty area images; Performing occlusion simulation processing on the photometrically perturbed historical pigsty area images to obtain the preprocessed historical pigsty area images.

4. The method according to claim 3, wherein The step of performing photometric perturbation processing on the geometrically transformed historical pigsty area images to obtain photometrically perturbed historical pigsty area images includes: Offset the RGB channels in the historical pigsty area image after geometric deformation based on the preset offsets of the RGB channels to obtain the offset RGB channels; Obtain the historical pigsty area image after photometric perturbation processing based on the offset RGB channels.

5. The method according to claim 1, characterized in that, The step of performing restoration processing on each historical live pig image to obtain each target live pig image includes: In response to the existence of a patch area in the historical live pig image, perform matte extraction processing on the patch area in the historical live pig image to obtain the historical live pig image after matte extraction; Perform texture synthesis processing on the matte extraction area of the historical live pig image after matte extraction to obtain the target live pig image.

6. The method according to claim 1, characterized in that, The step of performing restoration processing on each historical live pig image to obtain each target live pig image further includes: In response to the existence of a pollutant coverage area in the historical live pig image, perform mask coverage processing on the pollutant coverage area in the historical live pig image to obtain the historical live pig image after mask coverage processing; Perform texture reconstruction processing on the mask coverage area of the historical live pig image after mask coverage processing to obtain the target live pig image.

7. The method according to claim 1, wherein The step of training the initial cleanliness scoring model based on each target live pig image to obtain the target cleanliness scoring model includes: Obtain the actual cleanliness scoring values of each target live pig image; Train the initial cleanliness scoring model based on each target live pig image and the actual cleanliness scoring values corresponding to each target live pig image to obtain the trained initial cleanliness scoring model; Obtain the accuracy rate of the trained initial cleanliness scoring model; In response to the accuracy rate of the trained initial cleanliness scoring model being greater than the preset accuracy threshold, determine the trained initial cleanliness scoring model as the target cleanliness scoring model.

8. The method according to claim 1, wherein The step of obtaining the current pig cleanliness score of the pigsty area based on the cleanliness scores corresponding to each current live pig image includes: In response to the non-existence of a patch area in the current live pig image, obtain the total score among the cleanliness scores corresponding to each current live pig image; Determine the ratio between the total score and the number of live pigs in each current live pig image as the current pig cleanliness score of the pigsty area.

9. An intelligent evaluation system for the cleanliness of pigs based on a pigsty inspection robot, characterized in that, Includes: An image acquisition module for acquiring historical live pig images of each live pig in the pigsty area; An abnormal image processing module for performing restoration processing on each historical live pig image to obtain each target live pig image; A model training module for training the initial cleanliness scoring model based on each target live pig image to obtain the target cleanliness scoring model; A cleanliness score acquisition module for inputting the acquired current live pig images into the target cleanliness scoring model to obtain the cleanliness scores corresponding to each current live pig image; A current pig cleanliness score acquisition module for obtaining the current pig cleanliness score of the pigsty area based on the cleanliness scores corresponding to each current live pig image; The step of obtaining the current pig cleanliness score of the pigsty area based on the cleanliness scores corresponding to each current pig image includes: in response to the presence of a patch area in the current pig image, determining the patch category according to the area of the patch area; obtaining the current pig cleanliness score of the pigsty area according to the patch category corresponding to each current pig image and the cleanliness score corresponding to each current pig image. The step of obtaining the current pig cleanliness score of the pigsty area according to the patch category corresponding to each current pig image and the cleanliness score corresponding to each current pig image includes: determining the corresponding patch area score from a preset patch score mapping table according to the patch category corresponding to each current pig image, where the preset patch score mapping table includes the corresponding relationship between the patch category and the patch area score; obtaining a first product term between the patch area proportion value in the current pig image and the patch area score, and obtaining a second product term between the difference between the preset quantity threshold and the patch area proportion value and the cleanliness score; determining the sum between the first product term and the second product term as the target cleanliness score corresponding to the current pig image; taking the average value between the target cleanliness score corresponding to the current pig image with a patch area and the cleanliness score corresponding to the current pig image without a patch area as the current pig cleanliness score of the pigsty area.

Images