A method, apparatus, equipment, and medium for identifying living beings with abnormal health conditions.

Abstract

This application discloses a method, apparatus, device, and medium for identifying living organisms with abnormal health conditions, relating to the field of detection. The method learns unobstructed or partially obstructed areas (target areas) from visible light images using an attention mechanism classification network model. Then, it detects the health condition of the living organism based on the temperature data of the target area, improving the accuracy of health condition detection. Secondly, the attention mechanism classification network model learns the target area of ​​the living organism in visible light, which does not require manual calibration, improving the efficiency of health condition detection. Furthermore, compared to methods using single-modal data for health condition detection, this application's method uses both visible light and temperature modal data for joint identification, avoiding missed detections and low accuracy associated with single-modal methods.

Description

Technical Field

[0001] This application relates to the field of detection, and in particular to a method, apparatus, equipment and medium for identifying living beings with abnormal health conditions. Background Technology

[0002] While livestock and poultry farming has largely automated processes such as feeding, watering, manure removal, and temperature control, the detection of livestock illness or death still relies primarily on manual methods. Some proposed intelligent methods, such as using robots to tap on pens and observe livestock movement, suffer from the risk of startling the animals, and the algorithms are not always effective. Wearing leg bands on each animal to record its walking routes is too expensive for most farms. Alternatively, detecting key points and measuring temperature are impractical due to obstructions and the potential for data loss.

[0003] It is difficult to distinguish sick and dead livestock and poultry from their appearance alone. Body temperature is an important indicator of their health, illness, and death. Infrared thermometers can be used to obtain the body temperature of livestock and poultry, helping to identify those that have died from disease. However, some livestock and poultry that die early have body temperatures consistent with room temperature and cannot be seen in infrared images. Therefore, neither visible light nor infrared thermometry alone is sufficient for accurate detection of livestock and poultry deaths. To achieve more accurate identification and location, a combination of visible light and infrared thermometry is needed for livestock and poultry death detection.

[0004] The current main method for combined visible light and infrared detection involves first identifying the target using visible light, and then calculating a threshold in the corresponding infrared region. This method requires the target to be unobstructed, but caged livestock and poultry are very densely packed and often obstructed by objects such as water troughs and feed troughs, especially as the livestock and poultry grow larger, the obstruction worsens. Other methods first detect key points in the livestock and poultry, and then use the temperature at these key points (such as the head and feet) to calculate whether the livestock or poultry has died. This method can avoid false alarms caused by water trough obstruction, but key point calibration is done manually, which is time-consuming and labor-intensive. Furthermore, due to the high density of caged livestock and poultry, key points are easily lost, leading to missed reports of sick or dead livestock and poultry.

[0005] Therefore, in situations where livestock and poultry are densely packed and heavily obstructed, improving the accuracy and efficiency of detecting diseased and dead livestock and poultry is a technical problem that urgently needs to be solved by researchers in this field. Summary of the Invention

[0006] The purpose of this application is to provide a method, apparatus, device, and medium for identifying living beings with abnormal health conditions, which can improve the accuracy and efficiency of detecting the health status of living beings in situations where living beings are densely packed or severely obscured.

[0007] To address the aforementioned technical problems, this application provides a method for identifying living beings with abnormal health conditions, applied to an AI server, the method comprising:

[0008] Acquire visible light images of the organism to be detected by the image acquisition device, and temperature data images that match the visible light images of the organism to be detected;

[0009] The visible light image and temperature data image of the organism to be detected are input into an attention mechanism classification network model; wherein, the attention mechanism classification network model is trained using the visible light image of the sample organism, the temperature data of the target area, and the health status of the sample organism as sample data; the attention mechanism classification network model learns the target area for detecting the health status of the sample organism from the visible light image of the sample organism, wherein the target area is an unobstructed area or a partially obstructed area in the visible light image of the sample organism;

[0010] The health status of the organism to be tested is detected using the attention mechanism classification network model.

[0011] Preferably, establishing the attention mechanism classification network model includes:

[0012] The image acquisition device acquires data containing each of the sample organisms; wherein the data includes at least a visible light image of the sample organism and a temperature data image matching the visible light image of the sample organism;

[0013] Data on each of the sample life forms with abnormal health conditions are obtained from the collected data of the sample life forms;

[0014] The visible light images of each of the sample life forms with abnormal health conditions are input into the first convolutional network model, and the probability of the target region of the sample life form with abnormal health conditions is output.

[0015] The temperature data images of each of the sample organisms with abnormal health conditions are input into the second convolutional network model, and the temperature data of the target region of the sample organisms with abnormal health conditions are output.

[0016] The probability of the target region of each of the sample organisms with abnormal health status is multiplied by the temperature data of the target region to obtain weighted feature data;

[0017] The weighted feature data is then input into the neural network model;

[0018] The neural network model outputs the classification results of the sample organisms with abnormal health conditions.

[0019] Preferably, before inputting the visible light images of each of the sample organisms with abnormal health conditions into the first convolutional network model, the method further includes:

[0020] The visible light images of each of the sample organisms with abnormal health conditions are normalized and scaled to obtain scaled visible light images;

[0021] Before inputting the temperature data images of each of the sample organisms with abnormal health conditions into the second convolutional network model, the method further includes:

[0022] The temperature data images of each of the sample organisms with abnormal health conditions are normalized and scaled to obtain scaled temperature data images; wherein the resolution of the scaled visible image is the same as the resolution of the scaled temperature data image.

[0023] Preferably, before normalizing and scaling the visible light images of each of the sample organisms with abnormal health conditions, and obtaining the scaled visible light images, the method further includes:

[0024] The location of each of the said sample life forms in the visible light image containing each of the said sample life forms is identified by a target recognition algorithm;

[0025] The visible light image containing each of the sample organisms and the temperature data image containing each of the sample organisms are registered to determine the temperature data of each of the sample organisms.

[0026] Preferably, identifying the location of each of the sample organisms in the visible light image containing each of the sample organisms using a target recognition algorithm includes:

[0027] For the visible light images containing each of the aforementioned sample organisms, the labelImg tool is used to annotate them and obtain the positions of each of the aforementioned sample organisms after manual selection.

[0028] Using target recognition algorithms, the visible light images of each labeled living organism are trained to obtain target recognition algorithm models and the accuracy of each target recognition algorithm model;

[0029] Using the target recognition algorithm model that achieves a preset accuracy, the location of each of the sample life forms in the visible light image containing each of the sample life forms is identified.

[0030] Preferably, the registration of the visible light image containing each of the sample organisms and the temperature data image containing each of the sample organisms to determine the temperature data of each of the sample organisms includes:

[0031] The black and white checkerboard calibration board is photographed using an image acquisition device at a preset distance from the calibration board.

[0032] Acquire visible light and infrared images of the black and white checkerboard calibration plate;

[0033] The homography matrix is ​​obtained through a registration algorithm;

[0034] The visible light image containing each of the sample organisms and the temperature data image containing each of the sample organisms are registered according to the homography matrix to determine the temperature data of each of the sample organisms.

[0035] Preferably, before acquiring visible light images containing each of the sample organisms using an industrial camera in the image acquisition device, the method further includes:

[0036] The industrial camera in the image acquisition device takes pictures of a chessboard of a preset area from multiple angles to obtain images of the chessboard.

[0037] The distortion parameters of the industrial camera are generated by calibrating the chessboard image using a distortion correction algorithm.

[0038] Correspondingly, after acquiring visible light images of each of the sample organisms using an industrial camera in the image acquisition device, the method further includes:

[0039] The distortion parameters are used to correct the visible light images of each of the sample organisms captured by the industrial camera.

[0040] To address the aforementioned technical issues, this application also provides a device for identifying living beings with abnormal health conditions, comprising: an image acquisition device and an AI server;

[0041] The image acquisition device is used to acquire a visible light image of the organism to be detected and a temperature data image that matches the visible light image of the organism to be detected.

[0042] The AI ​​server is connected to the image acquisition device and is used to acquire the visible light image of the organism to be detected and the temperature data image matching the visible light image of the organism to be detected, which are acquired by the image acquisition device. The visible light image and the temperature data image of the organism to be detected are input into an attention mechanism classification network model. The attention mechanism classification network model is trained using the visible light image of the sample organism, the temperature data of the target area, and the health status of the sample organism as sample data. The attention mechanism classification network model learns the target area for detecting the health status of the sample organism from the visible light image of the sample organism. The target area is an unobstructed area or a partially obstructed area in the visible light image of the sample organism. The health status of the organism to be detected is detected by the attention mechanism classification network model.

[0043] Preferably, the image acquisition device includes an industrial camera and an infrared temperature measurement module;

[0044] The field of view of the industrial camera is greater than that of the infrared temperature measurement module.

[0045] Preferably, there are multiple AI servers.

[0046] To address the aforementioned technical problems, this application also provides a device for identifying living beings with abnormal health conditions, applied to an AI server, the device comprising:

[0047] The acquisition module is used to acquire the visible light image of the organism to be detected acquired by the image acquisition device and the temperature data image that matches the visible light image of the organism to be detected;

[0048] An input module is used to input the visible light image and temperature data image of the organism to be detected into an attention mechanism classification network model; wherein, the attention mechanism classification network model is trained using the visible light image of the sample organism, the temperature data of the target area, and the health status of the sample organism as sample data; the attention mechanism classification network model learns the target area for detecting the health status of the sample organism from the visible light image of the sample organism, wherein the target area is an unobstructed area or a partially obstructed area in the visible light image of the sample organism;

[0049] The detection module is used to detect the health status of the organism to be detected through the attention mechanism classification network model.

[0050] To address the aforementioned technical problems, this application also provides a device for identifying living beings with abnormal health conditions, comprising:

[0051] Memory, used to store computer programs;

[0052] A processor, used to execute the computer program to implement the steps of the above-described method for identifying life forms with abnormal health conditions.

[0053] To address the aforementioned technical problems, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method for identifying life forms with abnormal health conditions described above.

[0054] The method for identifying living beings with abnormal health conditions provided in this application is applied to an AI server. The method includes: acquiring a visible light image of the living being to be detected captured by an image acquisition device, and a temperature data image matching the visible light image of the living being to be detected; inputting the visible light image and temperature data image of the living being to be detected into an attention mechanism classification network model; wherein the attention mechanism classification network model is trained using the visible light image of the sample living being, the temperature data of the target area, and the health condition of the sample living being as sample data; the attention mechanism classification network model learns a target area from the visible light image of the sample living being used to detect the health condition of the sample living being, the target area being an unobstructed area or a partially obstructed area in the visible light image of the sample living being; and detecting the health condition of the living being to be detected using the attention mechanism classification network model. Compared to previous methods that manually calibrate key points of living organisms to determine their health status, the method provided in this application learns unobstructed or partially obstructed areas (i.e., target areas) from visible light images using an attention mechanism classification network model. Then, it detects the health status of the living organism based on the temperature data of the target area, improving the accuracy of health status detection. Secondly, the target area of ​​the living organism in visible light is learned through the attention mechanism classification network model, eliminating the need for manual calibration and improving the efficiency of health status detection. Furthermore, compared to methods using single-modality data for health status detection, this application's method uses both visible light and temperature modalities for joint identification, avoiding the missed detections and low accuracy associated with single-modality methods.

[0055] In addition, this application also provides a device, equipment, and computer-readable storage medium for identifying living beings with abnormal health conditions, which have the same or corresponding technical features as the aforementioned method for identifying living beings with abnormal health conditions, and have the same effect. Attached Figure Description

[0056] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating a method for identifying abnormally healthy living beings in an AI server, provided as an embodiment of this application;

[0058] Figure 2 A flowchart illustrating the establishment of an attention mechanism classification network model is provided in this application embodiment;

[0059] Figure 3 A schematic diagram of a module box for housing an industrial camera and an infrared temperature measurement module, provided for an embodiment of this application;

[0060] Figure 4a A visible light image provided in an embodiment of this application;

[0061] Figure 4b An infrared image that matches a visible light image, provided as an embodiment of this application;

[0062] Figure 5 A structural diagram of a device for identifying abnormal living beings in an AI server, provided as an embodiment of this application;

[0063] Figure 6 A structural diagram of a device for identifying living beings with abnormal health conditions is provided for another embodiment of this application;

[0064] Figure 7 A flowchart illustrating an attention-based method for detecting livestock and poultry mortality, as provided in this application embodiment.

[0065] Figure 8a Visible light image of a chicken captured by an industrial camera in an image acquisition device, as provided in the embodiment of the application;

[0066] Figure 8b The infrared image of a chicken acquired by an infrared thermometer in an image acquisition device, as provided in this embodiment of the application;

[0067] Figure 8c Visible light image of a chicken identified by YOLOv5, provided in an embodiment of this application;

[0068] Figure 8d Visible light image of a chicken with abnormal health status identified by the abnormal health status life form identification method of this application;

[0069] Figure 8e Infrared image of a chicken with abnormal health status identified by the abnormal health status identification method of this application. Detailed Implementation

[0070] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0071] The core of this application is to provide a method, apparatus, device, and medium for identifying living beings with abnormal health conditions, which can improve the accuracy and efficiency of detecting the health status of living beings in situations where living beings are densely packed or severely obscured.

[0072] In livestock and poultry farms, it is necessary to monitor the health status of livestock and poultry. Currently, this is mainly done by manually marking key points, such as the head and feet of the animals. However, in farms, the density of livestock and poultry is high, and obstructions are easily created, leading to a decrease in the detection rate and a high false alarm rate for livestock and poultry health monitoring. Therefore, this application proposes a livestock and poultry health status detection method based on an attention mechanism. This method learns the key areas of livestock and poultry (i.e., the target areas mentioned in the embodiments of this application), thereby achieving accurate identification and location of organisms with abnormal health status based on the temperature data of the key areas of livestock and poultry.

[0073] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Figure 1 A flowchart illustrating a method for identifying abnormally healthy living beings in an AI server, as provided in this application embodiment, is shown below. Figure 1 As shown, the method includes:

[0074] S10: Acquire the visible light image of the organism to be detected acquired by the image acquisition device, as well as the temperature data image that matches the visible light image of the organism to be detected;

[0075] S11: Input the visible light image and temperature data image of the organism to be detected into the attention mechanism classification network model;

[0076] S12: The health status of the organism to be tested is detected by classifying the network model using an attention mechanism.

[0077] In this embodiment, an attention mechanism classification model is used to detect the health status of the organism to be tested. The attention mechanism classification network model is trained using the visible light image of the sample organism, the temperature data of the target area, and the health status as sample data. The attention mechanism classification network model learns the target area for detecting the health status of the sample organism from the visible light image of the sample organism. The target area is the unobstructed area or the area that is not completely obstructed in the visible light image of the sample organism.

[0078] When establishing an attention mechanism classification network model, a preferred implementation method is to acquire data containing each sample organism through an image acquisition device; wherein, the data includes at least a visible light image of the sample organism and a temperature data image that matches the visible light image of the sample organism.

[0079] Data on individuals with abnormal health conditions were obtained from the collected sample organism data.

[0080] The visible light images of each life sample with abnormal health status are input into the first convolutional network model, and the probability of the target region of the life sample with abnormal health status is output.

[0081] The temperature data images of each sample life form with abnormal health status are input into the second convolutional network model, and the temperature data of the target area of ​​the sample life form with abnormal health status is output.

[0082] The probability of the target region for each sample of life form with abnormal health status is multiplied by the temperature data of the target region to obtain weighted feature data;

[0083] The weighted feature data is then input into the neural network model;

[0084] The neural network model outputs the classification results of the sample organisms with abnormal health conditions.

[0085] The sample organism must be of the same type as the organism to be detected. For example, if the organism to be detected is a chicken, then the sample organism selected when building the attention mechanism classification model is also a chicken. First, visible light images and temperature data images of the sample organism are acquired using an image acquisition device. To improve detection accuracy, the image acquisition device in this embodiment includes an industrial camera and an infrared temperature measurement module, used to acquire visible light images and temperature data images of the organism, respectively. The industrial camera and infrared temperature measurement module can be placed in the same module or used as a separate module. It should be noted that the visible light images and temperature data images must be matched; that is, the visible light images and temperature data images must be captured simultaneously. There are no limitations on the frequency and area of ​​image acquisition; they are determined according to the actual situation. When training the attention mechanism classification model on the sample organisms, there may be an imbalance in the ratio of normal organisms to sick organisms and dead organisms. Therefore, this embodiment uses random sampling when training the sample data: after randomly selecting one live organism, one sick or dead organism is randomly selected for model training.

[0086] The attention mechanism classification model includes multimodal input, a first convolutional network, a second convolutional network, and neural network model training. The multimodal input refers to visible light images and temperature data images. In this embodiment, the ResNet18 network model is used for the first convolutional network, the second convolutional network, and the neural network model. Preferably, the first convolutional network model uses 4 convolutional layers, and the second convolutional network model uses 2 convolutional layers. Since the first convolutional network in this embodiment is used to convolve the visible light image, it can also be called the visible light network; the second convolutional network convolves the temperature data image, so it can also be called the temperature network. After acquiring the visible light image and temperature data image of the sample organism, these are used as input to the attention mechanism classification model, and the health status of the sample organism is used as the output of the attention mechanism classification model. The classification of the health status of the sample organism is not limited and is determined according to the actual situation. For example, the health status of the sample organism can be divided into three categories: normal, sick, and dead.

[0087] Taking a farm as an example, this paper explains the process of establishing a classification model for attention mechanisms.

[0088] (1) Collect data at different growth stages in the farm, collecting multiple sets of data, such as 100,000 sets. Each set of data includes a visible light image and a full-frame infrared temperature data. Extract the visible light image and temperature data image for each livestock and poultry.

[0089] (2) Screen out data on sick and dead livestock and poultry;

[0090] (3) The visible light network employs four convolutional layers, with the last layer using the sigmoid activation function. The output is the probability of key regions in livestock and poultry, thus implementing an attention mechanism. The first layer design of both the visible light and temperature networks uses a 7x7 convolutional kernel with a stride of 2, as shown in the formula:

[0091] input_col_2=Relu(Conv2d(input_col,3,32,kernel_size=7,stride=2))

[0092] The implementation formula for the attention mechanism layer is as follows:

[0093] ratio_weight=Sigmoid(Conv2d(input_col_end,32,1,kernel_size=3,stride=1))

[0094] The temperature measurement uses a 2-layer convolutional network design, with the following formula:

[0095] input_tmp_2=Relu(Conv2d(input_tmp,1,32,kernel_size=7,stride=2))

[0096] input_tmp_end=Relu(Conv2d(input_tmp_2,32,32,kernel_size=5,stride=1))

[0097] The attention mechanism result is multiplied by the temperature feature result to obtain the weighted feature data:

[0098] input_for_resnet=input_tmp_end*repeat(ratio_weight, 32)

[0099] The ResNet18 network needs to adjust the first layer convolutional kernel according to the output dimension of step (3). In this embodiment, livestock and poultry are categorized into three types: normal, sick, and dead. Therefore, the output category of the ResNet18 network is adjusted to 3.

[0100] After establishing the attention mechanism classification model, visible light images and temperature data images of the organism to be detected are acquired through image acquisition equipment and then input into the attention mechanism classification network model, which then outputs the health detection result of the organism to be detected, which is one of the following: normal, sick, or dead.

[0101] The method for identifying living beings with abnormal health conditions provided in this embodiment is applied to an Artificial Intelligence (AI) server. The method includes: acquiring a visible light image of the living being to be detected and a temperature data image matching the visible light image of the living being to be detected, captured by an image acquisition device; and inputting the visible light image and temperature data image of the living being to be detected into an attention mechanism classification network model. Compared to previous methods that manually calibrate key points of the living being to determine its health condition, the method provided in this embodiment learns unobstructed or partially obstructed areas from the visible light image, i.e., target areas, through the attention mechanism classification network model. Then, based on the temperature data of the living being in the target area, the health condition of the living being is detected, improving the accuracy of the health condition detection. Secondly, the target area of ​​the living being in visible light is learned by the attention mechanism classification network model, and this target area does not require manual calibration, improving the efficiency of the health condition detection. Furthermore, compared to methods that use single-modality data for health condition detection, the method in this embodiment uses both visible light and temperature modal data for joint identification, avoiding the missed detections and low accuracy of single-modality methods.

[0102] After acquiring the visible light image and temperature data image of the sample organism, there may be differences between the two modalities. Therefore, in order to facilitate the input of data from the two different modalities into the attention mechanism classification model, a preferred implementation method is that, before inputting the visible light image of each sample organism with abnormal health status into the first convolutional network model, the method for identifying organisms with abnormal health status further includes:

[0103] The visible light images of each sample organism with abnormal health conditions were normalized and scaled, and the scaled visible light images were obtained.

[0104] Before inputting the temperature data images of each living organism with abnormal health conditions into the second convolutional network model, the methods for identifying living organisms with abnormal health conditions also include:

[0105] Temperature data images of each sample organism with abnormal health conditions are normalized and scaled to obtain scaled temperature data images; the resolution of the scaled visible image is the same as the resolution of the scaled temperature data image.

[0106] The steps in this embodiment can be understood as preprocessing. For example, preprocessing visible light involves dividing by 255 and scaling to a 128*128 resolution, as shown in the formula:

[0107] input_col=resize(pic / 255,(128,128))

[0108] Temperature data was obtained based on the body temperature of livestock and poultry and the minimum room temperature in the farm. After testing, high values ​​were normalized to 45 and low values ​​to 25, and then scaled to a resolution of 128*128. The formula is as follows:

[0109] input_tmp=resize((tmp-25) / (45-25),(128,128))

[0110] Figure 2 This is a flowchart illustrating the establishment of an attention mechanism classification network model, as provided in an embodiment of this application. Figure 2 As shown, visible light and temperature are preprocessed to obtain images with a resolution of 128*128, which are then fed into the attention mechanism classification model.

[0111] In the method provided in this embodiment, the visible light image and temperature data image are normalized and scaled to facilitate the subsequent processing of the two modalities by the attention mechanism classification model.

[0112] To locate the position of the same living organism in the visible light image and its corresponding temperature data in the temperature data image from visible light images and temperature data images containing various living organism samples, a preferred embodiment includes, before normalizing and scaling the visible light images of each living organism with abnormal health conditions and obtaining the scaled visible light images, the method for identifying living organisms with abnormal health conditions further includes:

[0113] The location of each life form in a visible light image containing each life form was identified using a target recognition algorithm.

[0114] The visible light images containing each sample organism and the temperature data images containing each sample organism were registered to determine the temperature data of each sample organism.

[0115] Specifically, identifying the location of each life form in a visible light image containing each life form using a target recognition algorithm includes:

[0116] For visible light images containing each sample organism, the labelImg tool was used to annotate and obtain the positions of each sample organism after manual selection;

[0117] Using target recognition algorithms, visible light images of each labeled living organism are trained to obtain target recognition algorithm models and the accuracy of each target recognition algorithm model.

[0118] Using a target recognition algorithm model that achieves a preset accuracy, the location of each life form in a visible light image containing each life form is identified.

[0119] The image acquisition device uses an industrial camera to capture visible light images of each sample organism and an infrared thermography module to capture temperature data images of each sample organism. The industrial camera in the image acquisition device is a 4-megapixel starlight-level zoom camera, suitable for the relatively low-light environment of the farm. The infrared thermography module uses a 256*192 resolution uncooled vanadium oxide thermography module, which can acquire full-frame temperature and infrared images via the SDK. For convenient image acquisition, the industrial camera and infrared thermography module are assembled in a... Figure 3 Inside the module box, the field of view of the industrial camera was adjusted to be larger than that of the infrared temperature measurement module. Figure 3 This is a schematic diagram of a module box for housing an industrial camera and an infrared temperature measurement module, provided as an embodiment of this application.

[0120] After acquiring visible light images and temperature data images using an image acquisition device, the visible light images are labeled using the labelImg tool, and livestock and poultry are manually selected. The labeled data is trained using YOLOv5, and the model with the highest accuracy is saved. In this embodiment, the visible light image resolution used for training and inference is 1024*768.

[0121] In the method for collecting data of various living organisms using an image acquisition device provided in this embodiment, a target recognition algorithm with high accuracy is used to accurately determine the target living organism, and the temperature data of the target living organism is obtained by registering visible light images and temperature data.

[0122] Based on the above embodiments, in order to determine the temperature data of each sample organism, a preferred embodiment is to register a visible light image containing each sample organism and a temperature data image containing each sample organism to determine the temperature data of each sample organism, including:

[0123] The black and white checkerboard calibration board is photographed using an image acquisition device at a preset distance from the calibration board.

[0124] Acquire visible light and infrared images of the black and white checkerboard calibration plate;

[0125] The homography matrix is ​​obtained through a registration algorithm;

[0126] The visible light images and temperature data images of each life sample are registered using the homography matrix to determine the temperature data of each life sample.

[0127] First, use an infrared black and white checkerboard calibration board. Take an image from a preset distance (e.g., 2 meters) away from the calibration board using the image acquisition device, and obtain the image... Figure 4a , Figure 4b A set of visible light and infrared images, comparing the captured visible light and infrared images. Figure 4a A visible light image provided for an embodiment of this application. Figure 4b An infrared image matching a visible light image is provided as an embodiment of this application. A registration algorithm is used to generate a homography matrix. Using the homography matrix, the visible light image containing each living organism sample and the temperature data image containing each living organism sample are registered to determine the temperature data of each living organism sample.

[0128] In practice, industrial cameras in image acquisition devices may experience distortion, leading to image deformation and potentially causing errors in the identification of living organisms. Therefore, in a preferred embodiment, before acquiring visible light images containing each living organism through the industrial camera in the image acquisition device, the method for identifying living organisms with abnormal health conditions further includes:

[0129] The image acquisition device uses an industrial camera to take pictures of a chessboard of a preset area from multiple angles to obtain images of the chessboard.

[0130] The distortion parameters of an industrial camera are generated by calibrating a checkerboard image using a distortion correction algorithm.

[0131] Correspondingly, after acquiring visible light images of each sample organism using an industrial camera in an image acquisition device, the method for identifying organisms with abnormal health conditions also includes:

[0132] The visible light images of various living organisms captured by industrial cameras are corrected using distortion parameters.

[0133] There are no limitations on the preset area or the number of images captured. For example, using an 11*9 checkerboard pattern, 20 images can be captured from different angles. A distortion correction algorithm is then used to generate distortion parameters for the industrial camera. These distortion parameters are then used to correct the visible light images containing the various biological samples captured by the industrial camera.

[0134] The above describes a method for identifying living beings with abnormal health conditions. This embodiment also provides a device for identifying living beings with abnormal health conditions, which includes: an image acquisition device and an AI server.

[0135] Image acquisition equipment is used to acquire visible light images of the organism to be detected, as well as temperature data images that match the visible light images of the organism to be detected.

[0136] The AI ​​server connects to an image acquisition device to obtain a visible light image of the organism to be detected, along with temperature data matching the visible light image. The visible light image and temperature data of the organism are then input into an attention mechanism classification network model. This model is trained using the visible light image of the organism, temperature data of the target area, and the organism's health status as sample data. The model learns target areas from the visible light image of the organism for detecting its health status; these target areas are either unobstructed or partially obstructed regions. The health status of the organism is then detected using the attention mechanism classification network model. The image acquisition device includes an industrial camera and an infrared temperature measurement module; the industrial camera has a larger field of view than the infrared temperature measurement module.

[0137] The device for identifying living beings with abnormal health conditions provided in this embodiment has corresponding technical features to the method for identifying living beings with abnormal health conditions described above. The embodiments of the method for identifying living beings with abnormal health conditions have been described in detail above, and the embodiments of the device for identifying living beings with abnormal health conditions will not be repeated here, but the effects are the same as above.

[0138] In implementation, to improve detection efficiency, a preferred method uses multiple AI servers. These AI servers utilize Graphics Processing Units (GPUs) supporting Computer Unified Device Architecture (CUDA) as AI computing power and are deployed in the cloud. Increasing the number of servers provides higher concurrency capabilities to support more image acquisition devices. A single server can achieve a recognition rate of 50 frames per second, and through the accumulation of livestock and poultry data, the method for identifying abnormal health conditions in this application can be continuously optimized to improve recognition effectiveness.

[0139] In the above embodiments, the method for identifying living beings with abnormal health conditions has been described in detail. This application also provides embodiments of devices and equipment for identifying living beings with abnormal health conditions. It should be noted that this application describes the embodiments of the device part from two perspectives: one is based on functional modules, and the other is based on hardware.

[0140] Figure 5 This is a structural diagram of a device for identifying abnormal health conditions of living beings applied to an AI server, provided as an embodiment of this application. This embodiment, based on functional modules, includes:

[0141] The acquisition module 10 is used to acquire the visible light image of the living organism to be detected acquired by the image acquisition device and the temperature data image that matches the visible light image of the living organism to be detected;

[0142] Input module 11 is used to input the visible light image and temperature data image of the organism to be detected into the attention mechanism classification network model; wherein, the attention mechanism classification network model is trained by using the visible light image of the sample organism, the temperature data of the target area, and the health status of the sample organism as sample data; the attention mechanism classification network model learns the target area for detecting the health status of the sample organism from the visible light image of the sample organism, and the target area is the unoccluded area or the area that is not completely occluded in the visible light image of the sample organism;

[0143] The detection module 12 is used to detect the health status of the organism to be detected through an attention mechanism classification network model.

[0144] Since the embodiments of the apparatus and the embodiments of the method correspond to each other, please refer to the description of the embodiments of the method for the embodiments of the apparatus. They will not be repeated here, and the effect is the same as above.

[0145] Figure 6 This is a structural diagram of a device for identifying living beings with abnormal health conditions, as provided in another embodiment of this application. This embodiment is based on a hardware perspective, such as... Figure 6 As shown, the device for identifying living beings with abnormal health conditions includes:

[0146] Memory 20 is used to store computer programs;

[0147] The processor 21 is configured to implement the steps of the method for identifying living beings with abnormal health conditions as described in the above embodiments when executing a computer program.

[0148] The device for identifying living beings with abnormal health conditions provided in this embodiment may include, but is not limited to, smartphones, tablets, laptops, or desktop computers.

[0149] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU, which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0150] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 20 is used to store at least the following computer program 201, which, after being loaded and executed by the processor 21, is capable of implementing the relevant steps of the method for identifying life forms with abnormal health conditions disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, and the storage method may be temporary or permanent storage. The operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, the data involved in the aforementioned method for identifying life forms with abnormal health conditions.

[0151] In some embodiments, the device for identifying living beings with abnormal health conditions may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0152] Those skilled in the art will understand that Figure 6 The structure shown does not constitute a limitation on the device for identifying living beings with abnormal health conditions and may include more or fewer components than illustrated.

[0153] The device for identifying living beings with abnormal health conditions provided in this application includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the following method: the method for identifying living beings with abnormal health conditions, with the same effect as above.

[0154] This application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiments.

[0155] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0156] The computer-readable storage medium provided in this application includes the aforementioned method for identifying living beings with abnormal health conditions, and has the same effect.

[0157] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Figure 7 This is an overall flowchart of an attention-based method for detecting livestock and poultry mortality, provided in an embodiment of this application. Figure 7 As shown, the data collected by the image acquisition device is sent to the AI ​​server via the network. The AI ​​server uses a target recognition algorithm and an attention mechanism classification network to identify the results of livestock and poultry disease and death detection.

[0158] This section demonstrates the process and effectiveness of detecting livestock and poultry disease using the method described in this application. Taking a chicken farm as an example... Figure 8a The visible light image of a chicken, captured by an industrial camera in an image acquisition device, is provided in the embodiment of the application. Points A, B, C, D, and E represent the positions of the occluded chickens, respectively. Figure 8b The infrared image of a chicken acquired by the infrared temperature measurement module in the image acquisition device is provided in the embodiment of this application, wherein the selected area represents the infrared image of a chicken with abnormal health condition; Figure 8cThe visible light image of a chicken identified by YOLOv5 is provided in the embodiments of this application, wherein the selected area represents all individual chickens; Figure 8d This is a visible light image of a chicken with abnormal health status identified by the abnormal health status identification method of this application. Figure 8e Infrared image of a chicken with abnormal health status identified by the abnormal health status identification method of this application. Figure 8d and Figure 8e In the diagram, dashed boxes represent locations where traditional algorithms fail to identify or misidentify chickens with abnormal health conditions in partially occluded areas, while the proposed method for identifying abnormal health conditions in this application identifies these chickens correctly. Solid boxes represent locations of chickens with abnormal health conditions in unoccluded areas that can be identified by both traditional algorithms and the proposed method. Figure 8a It is difficult to distinguish diseased or dead livestock and poultry using only visible light; through... Figure 8b It can be observed that dead livestock and poultry are difficult to detect in infrared images; through Figure 8a Points A, D, and E show that livestock and poultry can easily be blocked by water troughs, fences, or other livestock and poultry. Through... Figure 8a Points C, D, and E reveal that key points of some livestock and poultry, such as the head, legs, and rump, are difficult to locate and mark. Through... Figure 8c It can be observed that YOLOv5 can accurately identify the location of each animal, including those that are occluded. Therefore, an attention-based classification network is needed to ensure proper identification under slight occlusion and to prevent false alarms under severe occlusion. Figure 8d , 8e It can be observed that the method for identifying living beings with abnormal health conditions in this application identified dead livestock and poultry in obscured areas, and there were no false alarms at locations such as water troughs and fences. Based on the method and equipment for identifying living beings with abnormal health conditions in this application, the detection rate of dead livestock and poultry in livestock and poultry farms is approximately 85%, and the false alarm rate is approximately 3%. When combined with inspection robots and other methods, it can save labor costs and realize automated detection of livestock and poultry deaths.

[0159] In summary, this application uses both visible light and temperature modal data for joint identification, avoiding missed detections and low accuracy associated with single-modal data. It employs an attention mechanism to learn key areas of livestock and poultry from visible light images, eliminating the need for manual calibration. The method for identifying abnormally healthy organisms dynamically adjusts the probability of these key areas based on input, exhibiting good adaptability in dense and occluded areas, resulting in a high detection rate and low false alarm rate for diseased and dead livestock and poultry. The use of industrial cameras and low-resolution infrared thermometers reduces costs and facilitates acceptance by farms. It does not require changes to the farm layout, generates no sound or strong light, and does not frighten livestock and poultry. Cloud deployment allows for continuous optimization of the method through the accumulation of livestock and poultry data, leading to continuous improvement in identification performance.

[0160] The foregoing has provided a detailed description of the method, apparatus, device, and medium for identifying living beings with abnormal health conditions provided in this application. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0161] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for identifying living beings with abnormal health conditions, applied to an AI server, characterized in that, The method includes: Acquire visible light images of the organism to be detected by the image acquisition device, and temperature data images that match the visible light images of the organism to be detected; The visible light image and temperature data image of the organism to be detected are input in parallel into an attention mechanism classification network model; wherein, the attention mechanism classification network model is trained using the visible light image of the sample organism, the temperature data of the target area, and the health status of the sample organism as sample data; the attention mechanism classification network model learns the target area for detecting the health status of the sample organism from the visible light image of the sample organism, wherein the target area is an unobstructed area or a partially obstructed area in the visible light image of the sample organism; The attention mechanism classification network model includes a first convolutional network model, a second convolutional network model, and a neural network model: the visible light image is input into the first convolutional network model, which outputs the probability of the target region of the sample organism; the temperature data image is input into the second convolutional network model, which outputs the temperature data of the target region of the sample organism; the probability of the target region of the sample organism is multiplied and weighted by the temperature data of the target region of the sample organism and then input into the neural network model. The health status of the organism to be detected is detected by the attention mechanism classification network model, and the health status includes at least normal and abnormal conditions.

2. The method for identifying living beings with abnormal health conditions according to claim 1, characterized in that, The establishment of the attention mechanism classification network model includes: The image acquisition device acquires data containing each of the sample organisms; wherein the data includes at least a visible light image of the sample organism and a temperature data image matching the visible light image of the sample organism; Data on each of the sample life forms with abnormal health conditions are obtained from the collected data of the sample life forms; The visible light images of each of the sample life forms with abnormal health conditions are input into the first convolutional network model, and the probability of the target region of the sample life form with abnormal health conditions is output. The temperature data images of each of the sample organisms with abnormal health conditions are input into the second convolutional network model, and the temperature data of the target region of the sample organisms with abnormal health conditions are output. The probability of the target region of each of the sample organisms with abnormal health status is multiplied by the temperature data of the target region to obtain weighted feature data; The weighted feature data is then input into the neural network model; The neural network model outputs the classification results of the sample organisms with abnormal health conditions.

3. The method for identifying living beings with abnormal health conditions according to claim 2, characterized in that, Before inputting the visible light images of each of the sample organisms with abnormal health conditions into the first convolutional network model, the method further includes: The visible light images of each of the sample organisms with abnormal health conditions are normalized and scaled to obtain scaled visible light images; Before inputting the temperature data images of each of the sample organisms with abnormal health conditions into the second convolutional network model, the method further includes: The temperature data images of each of the sample organisms with abnormal health conditions are normalized and scaled to obtain scaled temperature data images; wherein the resolution of the scaled visible image is the same as the resolution of the scaled temperature data image.

4. The method for identifying living beings with abnormal health conditions according to claim 3, characterized in that, Before normalizing and scaling the visible light images of each of the sample organisms with abnormal health conditions, and obtaining the scaled visible light images, the method further includes: The location of each of the said sample life forms in the visible light image containing each of the said sample life forms is identified by a target recognition algorithm; The visible light image containing each of the sample organisms and the temperature data image containing each of the sample organisms are registered to determine the temperature data of each of the sample organisms.

5. The method for identifying living beings with abnormal health conditions according to claim 4, characterized in that, The step of identifying the location of each of the sample life forms in the visible light image containing each of the sample life forms using a target recognition algorithm includes: For the visible light images containing each of the aforementioned sample organisms, the labelImg tool is used to annotate them and obtain the positions of each of the aforementioned sample organisms after manual selection. Using target recognition algorithms, the visible light images of each labeled living organism are trained to obtain target recognition algorithm models and the accuracy of each target recognition algorithm model; Using the target recognition algorithm model that achieves a preset accuracy, the location of each of the sample life forms in the visible light image containing each of the sample life forms is identified.

6. The method for identifying living beings with abnormal health conditions according to claim 4, characterized in that, The process of registering the visible light image containing each of the said sample organisms and the temperature data image containing each of the said sample organisms to determine the temperature data of each of the said sample organisms includes: The black and white checkerboard calibration board is photographed using an image acquisition device at a preset distance from the calibration board. Acquire visible light and infrared images of the black and white checkerboard calibration plate; The homography matrix is ​​obtained through a registration algorithm; The visible light image containing each of the sample organisms and the temperature data image containing each of the sample organisms are registered according to the homography matrix to determine the temperature data of each of the sample organisms.

7. The method for identifying living beings with abnormal health conditions according to any one of claims 4 to 6, characterized in that, Before acquiring visible light images of each of the sample organisms using an industrial camera in the image acquisition device, the method further includes: The industrial camera in the image acquisition device takes pictures of a chessboard of a preset area from multiple angles to obtain images of the chessboard. The image of the checkerboard pattern is calibrated using a distortion correction algorithm to generate the distortion parameters of the industrial camera. Correspondingly, after acquiring visible light images of each of the sample organisms using an industrial camera in the image acquisition device, the method further includes: The distortion parameters are used to correct the visible light images of each of the sample organisms captured by the industrial camera.

8. A device for identifying living beings with abnormal health conditions, characterized in that, include: Image acquisition equipment, AI server; The image acquisition device is used to acquire a visible light image of the organism to be detected and a temperature data image that matches the visible light image of the organism to be detected. The AI ​​server is connected to the image acquisition device and is used to acquire the visible light image of the organism to be detected and the temperature data image matching the visible light image of the organism to be detected, acquired by the image acquisition device. The visible light image and the temperature data image of the organism to be detected are input in parallel into an attention mechanism classification network model. The attention mechanism classification network model is trained using the visible light image of the sample organism, the temperature data of the target area, and the health status of the sample organism as sample data. The attention mechanism classification network model learns the target area for detecting the health status of the sample organism from the visible light image of the sample organism. The region in the visible light image of the sample organism is either unobstructed or partially obstructed. The attention mechanism classification network model includes a first convolutional network model, a second convolutional network model, and a neural network model: the visible light image is input into the first convolutional network model, which outputs the probability of the target region of the sample organism; the temperature data image is input into the second convolutional network model, which outputs the temperature data of the target region of the sample organism; the probability of the target region of the sample organism is multiplied and weighted by the temperature data of the target region of the sample organism and then input into the neural network model; the health status of the organism to be detected is detected through the attention mechanism classification network model, and the health status includes at least normal and abnormal conditions.

9. The device for identifying living beings with abnormal health conditions according to claim 8, characterized in that, The image acquisition device includes an industrial camera and an infrared temperature measurement module; The field of view of the industrial camera is greater than that of the infrared temperature measurement module.

10. The device for identifying living beings with abnormal health conditions according to claim 8 or 9, characterized in that, There are multiple AI servers.

11. A device for identifying living beings with abnormal health conditions, applied to an AI server, characterized in that, The device includes: The acquisition module is used to acquire a visible light image of the organism to be detected acquired by the image acquisition device and a temperature data image that matches the visible light image of the organism to be detected. An input module is used to input the visible light image and temperature data image of the organism to be detected into an attention mechanism classification network model in parallel. The attention mechanism classification network model is trained using the visible light image of the organism, the temperature data of the target region, and the health status of the organism as sample data. The attention mechanism classification network model learns the target region from the visible light image of the organism to detect the health status of the organism. The target region is an unobstructed or partially obstructed area in the visible light image of the organism. The attention mechanism classification network model includes a first convolutional network model, a second convolutional network model, and a neural network model: the visible light image is input into the first convolutional network model, which outputs the probability of the target region of the organism; the temperature data image is input into the second convolutional network model, which outputs the temperature data of the target region of the organism; the probability of the target region of the organism is multiplied and weighted by the temperature data of the target region of the organism and then input into the neural network model. The detection module is used to detect the health status of the organism to be detected through the attention mechanism classification network model, wherein the health status includes at least normal and abnormal conditions.

12. A device for identifying living beings with abnormal health conditions, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the method for identifying a living being with an abnormal health condition as described in any one of claims 1 to 7 when executing the computer program.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method for identifying living beings with abnormal health conditions as described in any one of claims 1 to 7.

Images