Lion head goose monitoring method, device and equipment and storage medium

Abstract

The application discloses a lion-headed goose monitoring method, device and equipment and a storage medium, wherein multi-dimensional data of a breeding farm, including pictures, videos, sounds, body temperatures and the like of lion-headed geese, are acquired, the multi-dimensional data are preprocessed, and then are sent into a target detection model, a tracking algorithm and an abnormality judgment logic for calculation, lion-headed geese with a high probability of disease or abnormal lion-headed geese are detected, and then the above early warning data and information are pushed to users in time through a client, so that accurate identification of individual lion-headed geese in a large number of scenes is realized, and identification based on multi-dimensional factors of monitoring videos is realized, automatic monitoring and identification are realized, and the situation that a large-scale disease infection is caused due to the fact that an abnormality cannot be found in time in a lion-headed goose flock is avoided.

Description

Lionhead Goose Monitoring Methods, Devices, Equipment and Storage Media Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a method, apparatus, device, and storage medium for monitoring lion-headed geese. Background Technology

[0002] With the development of science and technology, artificial intelligence has been gradually applied to various industries, especially in scenarios of automatic monitoring and identification, such as lionhead goose farming. Currently, in lionhead goose or other farming scenarios, there are already monitoring methods installed for care, but this care is limited to video monitoring and does not identify the health status of lionhead geese in real time based on video. Even if health status can be monitored, it is only based on data within the scenario, rather than on individual identification. Therefore, we continue to provide a monitoring method based on individual lionhead geese to free up human resources, promptly detect abnormal individual lionhead geese, and improve the quality of farming. Summary of the Invention

[0003] The purpose of this invention is to provide a method, device, equipment, and storage medium for monitoring Lionhead geese, in order to solve the problem of low accuracy in monitoring the health status of individuals when the number of geese raised is too large.

[0004] To achieve the above objectives, the first aspect of the present invention provides a method for monitoring lion-headed geese based on multimodal fusion sensing, comprising:

[0005] Video data of the Lionhead Goose flock was collected using monitoring equipment installed in the Lionhead Goose farm.

[0006] The lion-headed goose in the video data is identified and marked using a preset target detection model;

[0007] Based on the markers, each lion-headed goose in the video data is tracked and identified to determine whether the body shape of each lion-headed goose is abnormal.

[0008] If an abnormality is detected, the body temperature and sound information of the abnormal Lionhead Goose are obtained, and the corresponding Lionhead Goose is identified as a suspected sick goose based on the body temperature and sound information to obtain the monitoring results.

[0009] Based on the monitoring results, a notification message is generated to notify the farmers.

[0010] Optionally, the step of tracking and identifying each lion-headed goose in the video data based on the markers, and determining whether the body posture of each lion-headed goose is abnormal, includes:

[0011] The tracking algorithm is used to extract images of lion-headed geese containing the markers from each frame of the video data;

[0012] The extracted lion-headed goose images with the same label are sorted in chronological order to obtain a lion-headed goose image sequence.

[0013] The lion-head goose image sequence is input into a preset body posture analysis model. The body posture of the lion-head goose in each lion-head goose image in the lion-head goose image sequence is detected and identified. The body posture identification result of each lion-head goose image is then classified into normal and abnormal categories to obtain a score.

[0014] If the calculated score is greater than a preset anomaly threshold, then the marked Lionhead Goose is determined to be suspected of being abnormal.

[0015] Optionally, the step of extracting the lion-headed goose image containing the marker from each frame of the video data using a tracking algorithm includes:

[0016] Based on the markers, the current position of the corresponding lion-headed goose in the monitoring screen is determined; using the Kalman filter algorithm, the moving position of the lion-headed goose in the next frame of the video is predicted based on the current position; from the markers located at the moving position in the next frame of the video, and comparing them with the markers in the previous frame of the video, the image of the moving position is captured to obtain an image of the lion-headed goose.

[0017] Optionally, the step of inputting the lion-headed goose image sequence into a preset body posture analysis model, detecting and recognizing the body posture of the lion-headed goose in each image of the lion-headed goose image sequence, and performing a binary classification calculation of normal and abnormal for the body posture recognition result of each lion-headed goose image to obtain a calculated score, including:

[0018] The sequence of lion-headed geese images is input into a preset body posture analysis model to detect and identify the beak breathing pattern, posture, and gait of the lion-headed geese in each image of the sequence.

[0019] Using a pre-defined binary classification neural network, scores are calculated for the beak breathing pattern, posture, and gait of the lion-headed goose in each image.

[0020] Based on the scores, the normal and abnormal characteristics of each lion-headed goose in each of the lion-headed goose images are determined.

[0021] Optionally, the step of acquiring the abnormal body temperature and sound information of Lionhead geese, and identifying suspected sick geese based on the body temperature and sound information to obtain monitoring results includes:

[0022] Abnormal calls of Lionhead geese were collected using microphones within the Lionhead Goose Farm, and abnormal body temperatures of the geese were collected using temperature sensors within the Lionhead Goose Farm.

[0023] The sound of the lion-headed goose was matched with the sound of a goose in normal condition to determine whether it was abnormal.

[0024] Based on the goose's body temperature, determine whether the Lionhead Goose's body temperature is abnormal;

[0025] If both the call of the Lionhead Goose and its physical condition are abnormal, then the Lionhead Goose is determined to be in a disease state.

[0026] If either the call of the lion-headed goose or the goose's body exhibits an abnormality, then the lion-headed goose is determined to be in a severely abnormal state.

[0027] Optionally, generating a notification message to the farmer based on the monitoring results includes:

[0028] Capture the frame of the abnormal lion-headed goose from the video data, mark the abnormal lion-headed goose's number and location information on the frame, and generate an image notification message;

[0029] The image notification message is sent to the farmer's terminal via a mini-program on the monitoring platform.

[0030] Optionally, after generating a notification message to notify the farmers based on the monitoring results, the method further includes:

[0031] The number of abnormal Lionhead geese in the image notification message is counted by deploying an optimized deep learning object detection algorithm on the device, and then pushed to the farmer.

[0032] A second aspect of the present invention provides a Lionhead Goose monitoring device based on multimodal fusion sensing, comprising:

[0033] The acquisition module is used to collect video data of the Lionhead Goose flock through monitoring equipment installed in the Lionhead Goose farm;

[0034] The tagging module is used to identify and tag the lion-headed goose in the video data using a preset target detection model;

[0035] The tracking module is used to track and identify each lion-headed goose in the video data based on the markers, and to determine whether the body posture of each lion-headed goose is abnormal.

[0036] The monitoring module is used to obtain the body temperature and sound information of the abnormal Lionhead geese when the abnormal body posture is judged, and to identify the corresponding Lionhead geese as suspected sick geese based on the body temperature and sound information, and obtain the monitoring results.

[0037] The notification module is used to generate notification messages to farmers based on the monitoring results.

[0038] A third aspect of the present invention provides a computer device, comprising: a memory and at least one processor, wherein the memory stores instructions, and the memory and the at least one processor are interconnected via a circuit; the at least one processor invokes the instructions in the memory to cause the computer device to perform the various steps of the above-described multimodal fusion perception-based lion-headed goose monitoring method.

[0039] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the steps of the above-described multimodal fusion perception-based lion-headed goose monitoring method.

[0040] Beneficial effects:

[0041] This invention proposes a method, device, equipment, and storage medium for monitoring Lionhead geese based on multimodal fusion perception. The method involves collecting video data of the Lionhead goose flock using monitoring equipment installed in the goose farm; identifying and marking the Lionhead geese in the video data using a preset target detection model; tracking and identifying each goose based on the markings to determine if its physical condition is abnormal; if abnormal, acquiring the body temperature and vocal information of the abnormal goose, and identifying it as a suspected sick goose based on this information, thus obtaining the monitoring result; and generating a notification message to the farmer based on the monitoring result. By utilizing a target detection model and target tracking, the method achieves multi-dimensional fusion perception of the Lionhead goose's physical condition, vocalization, and body temperature to monitor the health status of individual geese. This solves the problem of existing monitoring schemes failing to accurately identify large numbers of geese. Furthermore, by identifying multiple factors from the monitoring video, it not only achieves automatic monitoring and identification but also avoids large-scale disease outbreaks in the Lionhead goose flock due to the failure to detect abnormalities in a timely manner. Attached Figure Description

[0042] Figure 1 is a flowchart illustrating a lion-headed goose monitoring method based on multimodal fusion perception provided in an embodiment of the present invention.

[0043] Figure 2 is another flowchart of the lion-headed goose monitoring method based on multimodal fusion perception provided in an embodiment of the present invention;

[0044] Figure 3 is a schematic diagram of a lion-headed goose monitoring device based on multimodal fusion perception provided in an embodiment of the present invention;

[0045] Figure 4 is a schematic diagram of a lion-headed goose monitoring device based on multimodal fusion perception provided in an embodiment of the present invention;

[0046] Figure 5 is a schematic diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] As shown in Figure 1, Figure 1 is a flowchart of a lion-headed goose monitoring method based on multimodal fusion perception according to an embodiment of the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0049] To achieve the above objectives, embodiments of the present invention provide a method for monitoring lion-headed geese based on multimodal fusion perception, comprising:

[0050] 110. Collect video data of the Lionhead Goose flock using monitoring equipment installed in the Lionhead Goose farm.

[0051] In this step, video data of the Lionhead Goose flock is collected using cameras installed within the Lionhead Goose farm. This video data is a frame of the farm's monitoring footage at a specific real-time point, and the monitoring footage includes the Lionhead geese, feeding equipment, and other farm equipment. In another embodiment, it also includes body temperature data and identification information.

[0052] In practical applications, the monitoring screen in this video data can display the identification information and body temperature data of each Lionhead Goose. The identification information is collected by tags with positioning functions, such as Bluetooth positioning tags, RFID tags, GPS positioning tags, etc. The body temperature data is obtained based on thermal imaging technology. In this case, the monitoring screen is not only a high-definition picture, but also includes a heat map. The body temperature data of each Lionhead Goose is displayed through the heat map, and the identification information, such as the Lionhead Goose number, is displayed in the detection box of each Lionhead Goose.

[0053] 120. Use a preset target detection model to identify and mark the lion-headed goose in the video data.

[0054] In this step, the target detection model is a target detection algorithm built based on deep learning neural network training. By inputting the video data collected above into the target detection model, the model extracts the monitoring screen containing the lion-headed goose, then identifies the identification information in the monitoring screen, extracts the content of the identification information to generate a tag, generates a tracking box for the lion-headed goose corresponding to the identification information, adds the tag to the monitoring screen, and obtains the target profile.

[0055] 130. Track and identify each lion-headed goose in the video data based on the tags, and determine whether the body posture of each lion-headed goose is abnormal.

[0056] In this step, the marker in step 120 is used as the target, and the video data of the next frame is obtained. The video data of the next frame is also processed based on the target detection model mentioned above. Then, the marker in the video data of the previous frame is used as the retrieval condition to find the target with the same marker in the video data of the next frame. After the target is found, the image of the marker is extracted and compared. When all the information is the same, it is determined to be the same target and associated to achieve tracking.

[0057] Furthermore, when determining whether the body posture of each lion-headed goose is abnormal, the image of the target lion-headed goose is extracted and input into the preset lion-headed goose body posture recognition model to identify the body posture of the lion-headed goose. Then, the identified body posture is compared with the body posture of the lion-headed goose in a normal state to determine whether it is abnormal.

[0058] In practical applications, determining whether a lion-headed goose's posture is abnormal involves comparing its posture in multiple consecutive frames of surveillance footage with that of a normal goose. The number of frames with abnormal postures is extracted and calculated as a percentage of the total number of frames captured. If this percentage exceeds a preset threshold, the goose is considered abnormal; otherwise, it is not. Furthermore, multiple abnormality levels can be set based on the preset threshold to categorize them according to severity.

[0059] 140. If an abnormality is detected, obtain the body temperature and sound information of the abnormal Lionhead Goose, and identify the suspected sick goose based on the body temperature and sound information to obtain the monitoring results.

[0060] In this embodiment, when it is determined that there are abnormal lion-headed geese in the monitoring screen, the abnormal lion-headed geese are extracted from the monitoring screen, and the area where each abnormal lion-headed goose is located is obtained. Then, the sound acquisition device in the area is scheduled to collect the sound of each abnormal lion-headed goose, and at the same time, the body temperature information is obtained from the heat map of the monitoring screen.

[0061] Then, the collected sound information is divided into tracks to obtain multiple sounds, and then each sound is matched one by one. If there is an abnormal sound, the body temperature data is judged to obtain the monitoring results of suspected sick geese.

[0062] 150. Based on the monitoring results, generate notification messages to notify farmers.

[0063] In this embodiment, the monitoring result is that there are abnormal Lionhead geese in the current monitoring screen. Then, a screenshot tool is used to take a screenshot of the current screen and save it. Based on the monitoring and identification results, the abnormal Lionhead geese in the screenshot are annotated, for example, by using a box to select them. The annotated screenshot is then sent to the breeding user as a prompt message.

[0064] This implementation demonstrates that by acquiring multi-dimensional data from the breeding farm, including images, videos, sounds, and body temperatures of Lionhead geese, and preprocessing this data, the information is fed into a target detection model, tracking algorithm, and anomaly detection logic for calculation. This process identifies Lionhead geese that are highly likely to be diseased or abnormal. The resulting warning data and information are then promptly pushed to users via the client, enabling accurate identification of individual Lionhead geese in large-scale scenarios. Furthermore, by identifying multiple factors based on monitoring videos, this approach not only achieves automatic monitoring and identification but also prevents large-scale disease outbreaks in Lionhead goose flocks due to the failure to detect anomalies in a timely manner.

[0065] The present invention will be further described below with reference to a specific example. The example is a 5G smart aquaculture platform, which uses a hardware framework that collects data by setting up a surround camera and transmits the monitoring data through 5G technology. The method provided in this embodiment is shown in Figure 2.

[0066] 210. Collect video data of the Lionhead Goose flock using monitoring equipment installed in the Lionhead Goose farm.

[0067] In this step, real-time data of each Lionhead Goose in the flock is collected by setting up cameras around the farm. The data is then transmitted back to the 5G smart farming platform via a 5G communication network or local area network. The 5G smart farming platform is set up in the monitoring room of the farm. The platform preprocesses the transmitted data to obtain standard video data, such as removing the background and marking the location of the Lionhead Goose.

[0068] 220. Use a preset target detection model to identify and mark the lion-headed goose in the video data.

[0069] In this step, the preprocessed video data is input into a preset target detection model for lion-headed goose identification. This target detection model is trained using manually labeled image data of lion-headed geese. Specifically, the model is trained offline using manually labeled image data of lion-headed geese. Based on this model, images captured by cameras in the farm are detected, and the lion-headed geese are identified. Finally, the coordinates of the lion-headed geese are obtained, enabling large-scale goose counting. Since the lighting conditions in the goose farm or the shooting angle of subsequently installed cameras can significantly affect the accuracy of the counting, this invention addresses this by collecting data under more conditions and using data augmentation methods to make the results more robust and improve the accuracy of goose counting in different scenarios. Regarding real-time performance, the counting algorithm is optimized in real-time by using a deep learning-based streaming media framework (DeepStream) when acquiring the input source, and a real-time deep learning inference framework (TensorRT) is used to accelerate the execution speed of the counting algorithm, thereby improving real-time performance.

[0070] In this embodiment, the target detection algorithm only outlines all the lion-headed geese in each frame of the image. To continuously track and store the state information of each goose, it is necessary to continuously track each lion-headed goose and label it. The purpose of target tracking is to link the same goose in previous and subsequent images and assign each goose an ID, thereby enabling long-term tracking of each goose.

[0071] 230. Use a tracking algorithm to extract images of tagged lion-headed geese from each frame of video data.

[0072] In this step, the current position of the corresponding lion-headed goose in the monitoring screen is determined based on the marker; the Kalman filter algorithm is used to predict the moving position of the lion-headed goose in the next frame of video based on the current position; the marker located at the moving position in the next frame of video is compared with the marker in the previous frame of video, and if they are the same, the image of the moving position is captured to obtain the image of the lion-headed goose.

[0073] In this embodiment, video data labeled with IDs is input into the tracking algorithm. The tracking algorithm identifies the IDs in the video data to extract the lion-headed goose image for each ID in each frame of the video data. The tracking algorithm includes the Kalman filter algorithm and the Hungarian algorithm.

[0074] In practical applications, this tracking operation is based on a tracking-by-detection strategy, meaning it tracks targets based on the results of deep learning object detection. The tracking operation comprises two core algorithms: the Kalman filter and the Hungarian algorithm. The Kalman filter predicts the current position based on the target's position in the previous moment, and it estimates the target's position more accurately than sensors. The Hungarian algorithm determines whether a target in the current frame is the same as a target in the previous frame. When target detection is complete, the Kalman filter updates the tracking data, while the Hungarian algorithm performs optimal matching between the targets detected in the previous and current frames, linking the same goose in adjacent images to complete the tracking. Users can see all the geese in the camera's view framed and labeled, allowing them to locate and track specific geese.

[0075] 240. Sort the extracted Lion Head Goose images with the same label in chronological order to obtain the Lion Head Goose image sequence.

[0076] 250. Input the sequence of Lion Head Goose images into the preset body posture analysis model, detect and identify the body posture of the Lion Head Goose in each Lion Head Goose image in the sequence, and perform binary classification calculation of normal and abnormal body posture recognition results for each Lion Head Goose image to obtain a calculated score.

[0077] In this step, the sequence is input into a preset body posture analysis model to identify the body posture of the lion-headed goose in each image. Based on the identification results, a body posture score is calculated, and the scores of lion-headed goose images in consecutive frames are compared to obtain the comparison results.

[0078] In this embodiment, identifying the Lionhead Goose's physical characteristics includes beak shape, posture, and gait. Specifically,

[0079] The sequence is input into a preset body posture analysis model to detect and identify the beak breathing pattern, posture, and gait of the Lionhead Goose;

[0080] Determine whether the breathing posture is abnormal based on the described mouth breathing pattern;

[0081] Determine whether the posture and gait are abnormal.

[0082] Specifically, the sequence of lion-headed geese images is input into a preset body posture analysis model to detect and identify the beak breathing pattern, posture, and gait of the lion-headed geese in each image of the sequence.

[0083] Using a pre-defined binary classification neural network, the scores of the mouth breathing pattern, posture, and gait of the lion-headed goose in each image are calculated. These scores can be understood as the scores of the rationality of the mouth breathing pattern, posture, and gait of the lion-headed goose calculated by the binary classification neural network, that is, the rationality of the bone deformation corresponding to the mouth breathing pattern, posture, and gait.

[0084] Based on the scores, the normal and abnormal characteristics of each lion-headed goose in each of the lion-headed goose images are determined.

[0085] In practical applications, Lionhead geese suffer from complex diseases with a variety of symptoms, including inactivity (standing still for extended periods), anorexia, abnormal breathing and demeanor, abnormal posture, abnormal gait, abnormal vocalizations, and abnormal body temperature. The multimodal perception algorithm used in this invention first detects geese in videos, tracks them for a certain period, and acquires video footage of a segment of the goose to determine if it is standing still and inactive. Combined with a user-defined feeding schedule, it determines if the goose is anorexic. Simultaneously, images are extracted from the acquired goose videos, and the goose's beak breathing pattern is detected to determine if its breathing and demeanor are abnormal. Further, the algorithm detects the goose's posture and gait in each frame of the video and determines if these are abnormal. Next, goose vocalizations are collected via microphone; the multimodal perception algorithm receives this vocalization data and identifies sick geese to determine if they are abnormal. Body temperature data collected by a temperature sensor is used to determine if the goose's body temperature is abnormal. Finally, the results of all the above steps are combined to determine if the goose is sick and in a severely abnormal state.

[0086] 260. If the calculated score is greater than the preset abnormal threshold, the marked Lionhead Goose is determined to be suspected of being abnormal.

[0087] 270. Obtain the body temperature and sound information of abnormal Lionhead geese, and identify the corresponding Lionhead geese as suspected sick geese based on the body temperature and sound information to obtain the monitoring results.

[0088] In this step, abnormal calls of Lionhead geese are collected through microphones in the Lionhead Goose Farm, and abnormal body temperatures of Lionhead geese are collected through temperature sensors in the Lionhead Goose Farm.

[0089] The sound of the lion-headed goose was matched with the sound of a goose in normal condition to determine whether it was abnormal.

[0090] Based on the goose's body temperature, determine whether the Lionhead Goose's body temperature is abnormal;

[0091] If both the call of the Lionhead Goose and its physical condition are abnormal, then the Lionhead Goose is determined to be in a disease state.

[0092] If either the call of the lion-headed goose or the goose's body exhibits an abnormality, then the lion-headed goose is determined to be in a severely abnormal state.

[0093] In this embodiment, the tracking algorithm obtains the trajectory of each goose and its status information. It collects the goose's vocal data through a microphone and its body temperature data through a temperature sensor. These multi-dimensional data are input into a multimodal perception algorithm. The algorithm detects whether the goose is inactive (standing still for a long time), has anorexia or loss of appetite, abnormal breathing or expression, abnormal posture, abnormal gait, abnormal vocalization, abnormal body temperature, and other major typical symptoms. The algorithm further integrates these detection results to calculate the probability that the goose is sick or in an abnormal state.

[0094] In practical applications, this also includes obtaining abnormal weight data of Lionhead geese for judgment. Specifically, an assessment is performed based on the weight data to obtain the weight assessment result of the Lionhead geese; then the body temperature data is assessed to obtain the body temperature assessment result of the Lionhead geese; further, the duration of the Lionhead geese's active movement is assessed through multi-frame video data to obtain the activity assessment result of the Lionhead geese.

[0095] The assessment process involves determining whether the final stable weight data is below the first weight threshold; if so, the development is considered slow; otherwise, it involves determining whether the weight data is below the second weight threshold; if so, the development is considered normal; otherwise, it involves determining whether the weight data is below the third weight threshold; if so, the development is considered excellent; otherwise, the development is considered excessive.

[0096] Furthermore, the evaluation of body temperature data includes the following:

[0097] Based on the body temperature characteristics and underwing body temperature patterns of Lionhead geese, the first, second, and third body temperature thresholds were determined.

[0098] Determine whether the corrected body temperature data is higher than the first body temperature threshold;

[0099] If so, it is assessed as overheating; otherwise, it is determined whether the corrected body temperature data is higher than the second body temperature threshold.

[0100] If so, it is assessed as normal; otherwise, it is determined whether the corrected body temperature data is higher than the third body temperature threshold.

[0101] If so, it is assessed as low temperature; otherwise, it is assessed as supercooled.

[0102] Furthermore, the assessment of the duration of active movement in Lionhead geese includes:

[0103] Based on the revised active exercise duration data, determine the active exercise duration data group for the same hour period;

[0104] Normalize the data set of active activity duration within the same hour, and represent it as: where: is the normalized data set of active activity duration within the same hour, is the kth historical active activity duration of the Lion Head Goose from the previous x days, and d is the total number of days;

[0105] Based on the normalized data set of active duration of movement in the same hour, the similarity comparison result is calculated and expressed as: where: is the similarity comparison result between the current k-th lion-headed goose and the k-th lion-headed goose from x days ago in the same hour, and is the active duration data of the lion-headed goose with identity information k in one hour on that day;

[0106] The similarity comparison results are fused using an adaptive Gaussian convolution fusion function and combined with nonlinear compression to obtain the activity score of the Lionhead Goose within the same time period, expressed as: where: is the activity score of the Lionhead Goose within the same time period, e is the natural constant, is the probability density function of a normal distribution with a mean of 0 and a standard deviation of 1, and is the adaptive parameter;

[0107] Based on the historical activity rating data of Lionhead geese, the first activity threshold, the second activity threshold, and the third activity threshold are determined;

[0108] Determine whether the activity score of the Lionhead Goose within the same time period is less than the first activity threshold. If so, it is assessed as inactive. Otherwise, determine whether the activity score of the Lionhead Goose within the same time period is less than the second activity threshold. If so, it is assessed as low-activity. Otherwise, determine whether the activity score of the Lionhead Goose within the same time period is less than the third activity threshold. If so, it is assessed as moderately active. Otherwise, it is assessed as highly active.

[0109] 280. Based on the monitoring results, generate notification messages to notify farmers.

[0110] In this embodiment, when a goose meeting the criteria for abnormality or disease is found, an image of the goose's location is captured and its coordinates are recorded. This information is then sent to the user via a mini-program, allowing the customer to immediately learn about sick geese or geese in abnormal condition, along with related information.

[0111] In this embodiment, after generating a notification message to notify the farmers based on the monitoring results, the method further includes:

[0112] The number of abnormal Lionhead geese in the image notification message is counted by deploying an optimized deep learning object detection algorithm on the device, and then pushed to the farmer.

[0113] In practical applications, the optimized and adjusted deep learning object detection algorithm YOLO (YouOnly Look Once: Unified, Real-Time Object Detection) is deployed on the device to calculate large-scale goose flock counts. The robustness and accuracy of the algorithm are improved by optimizing the training strategy, while DeepStream and TensorRt are used to improve the real-time performance of the algorithm.

[0114] In summary, this invention collects multi-dimensional data on Lionhead geese, including images, videos, body temperature, and vocalizations, by deploying front-end devices (cameras, temperature sensors, and microphones). This allows farmers to accurately count the number of Lionhead geese at their farms online via a client application. Simultaneously, tracking technology is used to obtain real-time status information of the geese. This real-time status information, along with the multi-dimensional data such as body temperature and vocalizations, is combined with a multi-modal perception algorithm to accurately and in real-time identify sick or abnormally sized Lionhead geese. This statistical and early warning information is then transmitted to farmers via a mini-program, helping them understand the real-time status and health of their geese. This provides valuable reference and feedback, enabling early disease warnings, early detection, and early treatment. This reduces the difficulty of inspections, improves the working environment, increases the survival rate of geese in the farm, reduces losses caused by disease, and allows farmers to manage their flocks more efficiently and scientifically, ultimately creating greater economic benefits for them.

[0115] The above describes the lion-headed goose monitoring method based on multimodal fusion perception in the embodiments of the present invention. The following describes the lion-headed goose monitoring device based on multimodal fusion perception in the embodiments of the present invention. Referring to Figure 3, one embodiment of the lion-headed goose monitoring device based on multimodal fusion perception in the embodiments of the present invention includes:

[0116] The acquisition module 310 is used to acquire video data of the Lionhead Goose flock through monitoring equipment installed in the Lionhead Goose farm.

[0117] The tagging module 320 is used to identify and tag the lion-headed goose in the video data using a preset target detection model;

[0118] The tracking module 330 is used to track and identify each lion-headed goose in the video data based on the marker, and to determine whether the body posture of each lion-headed goose is abnormal.

[0119] The monitoring module 340 is used to obtain the body temperature and sound information of the abnormal Lionhead geese when it is determined that the body posture of each Lionhead goose is abnormal, and to identify the corresponding Lionhead goose as a suspected sick goose based on the body temperature and sound information, and to obtain the monitoring results.

[0120] The notification module 350 is used to generate a notification message to notify the farmers based on the monitoring results.

[0121] This embodiment collects video data of the lion-head goose flock using monitoring equipment installed in the lion-head goose farm; it identifies and marks the lion-head geese in the video data using a preset target detection model; based on the markings, it tracks and identifies each lion-head goose in the video data to determine if its physical condition is abnormal; if abnormal, it obtains the body temperature and sound information of the abnormal lion-head goose, and identifies the corresponding lion-head goose as a suspected sick goose based on the body temperature and sound information, obtaining the monitoring results; based on the monitoring results, it generates a notification message to notify the farmer. By using a target detection model and target tracking, it achieves multi-dimensional fusion perception of the lion-head goose's physical condition, sound, and body temperature to monitor the health status of individual lion-head geese, solving the problem of existing monitoring solutions being unable to accurately identify geese in scenarios with a large number of geese.

[0122] Please refer to Figure 4. A second embodiment of the Lion Head Goose monitoring device based on multimodal fusion sensing in this invention includes:

[0123] The acquisition module 310 is used to acquire video data of the Lionhead Goose flock through monitoring equipment installed in the Lionhead Goose farm.

[0124] The tagging module 320 is used to identify and tag the lion-headed goose in the video data using a preset target detection model;

[0125] The tracking module 330 is used to track and identify each lion-headed goose in the video data based on the marker, and to determine whether the body posture of each lion-headed goose is abnormal.

[0126] The monitoring module 340 is used to obtain the body temperature and sound information of the abnormal Lionhead geese when it is determined that the body posture of each Lionhead goose is abnormal, and to identify the corresponding Lionhead goose as a suspected sick goose based on the body temperature and sound information, and to obtain the monitoring results.

[0127] The notification module 350 is used to generate a notification message to notify the farmers based on the monitoring results.

[0128] In this embodiment, the tracking module 330 includes:

[0129] Extraction unit 331 is used to extract the lion-headed goose image containing the marker from each frame of the video data using a tracking algorithm;

[0130] The sorting unit 332 is used to sort the extracted lion-headed goose images with the same label in chronological order to obtain a lion-headed goose image sequence.

[0131] The comparison unit 334 is used to input the lion-head goose image sequence into a preset body posture analysis model, detect and identify the body posture of the lion-head goose in each lion-head goose image in the lion-head goose image sequence, and perform normal and abnormal binary classification calculation on the body posture recognition result of each lion-head goose image to obtain a calculated score.

[0132] The determining unit 335 is used to determine that the marked Lionhead Goose is suspected of being abnormal when the calculated score is greater than a preset abnormal threshold.

[0133] In this embodiment, the extraction unit 331 is specifically used for:

[0134] Based on the markers, the current position of the corresponding lion-headed goose in the monitoring screen is determined; using the Kalman filter algorithm, the moving position of the lion-headed goose in the next frame of the video is predicted based on the current position; from the markers located at the moving position in the next frame of the video, and comparing them with the markers in the previous frame of the video, the image of the moving position is captured to obtain an image of the lion-headed goose.

[0135] In this embodiment, the comparison unit 334 is specifically used for:

[0136] The sequence of lion-headed geese images is input into a preset body posture analysis model to detect and identify the beak breathing pattern, posture, and gait of the lion-headed geese in each image of the sequence.

[0137] Using a pre-defined binary classification neural network, scores are calculated for the beak breathing pattern, posture, and gait of the lion-headed goose in each image.

[0138] Based on the scores, the normal and abnormal characteristics of each lion-headed goose in each of the lion-headed goose images are determined.

[0139] In this embodiment, the monitoring model 340 includes:

[0140] The acquisition unit 341 is used to collect abnormal calls of Lionhead geese through microphones in the Lionhead Goose Farm and to collect abnormal body temperatures of Lionhead geese through temperature sensors in the Lionhead Goose Farm.

[0141] Matching unit 342 is used to perform audio frequency matching between the lion-headed goose call and the normal goose call to determine whether it is abnormal;

[0142] The judgment unit 343 is used to determine whether the body temperature of the Lionhead Goose is abnormal based on the body temperature of the goose; if both the call of the Lionhead Goose and the body of the goose are abnormal, then the Lionhead Goose is determined to be in a disease state; if either the call of the Lionhead Goose or the body of the goose is abnormal, then the Lionhead Goose is determined to be in a seriously abnormal state.

[0143] In this embodiment, the notification module 350 includes:

[0144] The interception unit 351 is used to intercept the scene of the abnormal lion-headed goose in the video data, and mark the number and location information of the abnormal lion-headed goose in the scene, and generate an image notification message.

[0145] The notification unit 352 is used to send the image notification message to the farmer's terminal via the monitoring platform's mini-program.

[0146] In this embodiment, the Lion Head Goose monitoring device based on multimodal fusion perception further includes: a computing module 360, which is specifically used for:

[0147] The number of abnormal Lionhead geese in the image notification message is counted by deploying an optimized deep learning object detection algorithm on the device, and then pushed to the farmer.

[0148] In summary, by receiving multi-dimensional data from the farm, including images, videos, sounds, and body temperatures of Lionhead geese, and after preprocessing these multi-dimensional data, the data is fed into a multi-modal perception algorithm for calculation. This algorithm detects Lionhead geese that are likely to be diseased or abnormal, and then promptly pushes the above warning data and information to users through the client.

[0149] Figures 3-4 above describe the Lion Head Goose monitoring device based on multimodal fusion perception in this embodiment of the invention from the perspective of modular functional entities. The following describes the computer equipment in this embodiment of the invention from the perspective of hardware processing.

[0150] Figure 5 is a schematic diagram of a computer device 700 provided in an embodiment of the present invention. The computer device 700 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 710 (e.g., one or more processors) and a memory 720, and one or more storage media 730 (e.g., one or more mass storage devices) for storing application programs 733 or data 732. The memory 720 and storage media 730 may be temporary or persistent storage. The program stored in the storage media 730 may include one or more modules (not shown in the figure), each module including a series of instruction operations on the computer device 700. Furthermore, the processor 710 may be configured to communicate with the storage media 730 and execute the series of instruction operations in the storage media 730 on the computer device 700 to implement the steps of the aforementioned redundant power balancing control method.

[0151] The computer device 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input / output interfaces 760, and / or one or more operating systems 731, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that the computer device structure shown in FIG5 does not constitute a limitation on the computer device provided by the present invention, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0152] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the various steps of the Lion Head Goose Monitoring Method based on Multimodal Fusion Perception.

[0153] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0154] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0155] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring lionhead geese based on multimodal fusion sensing, characterized in that, The multimodal fusion perception-based lionhead goose monitoring method includes: collecting video data of the lionhead goose flock through monitoring equipment installed in the lionhead goose farm; identifying and marking the lionhead geese in the video data using a preset target detection model; tracking and identifying the body posture of each lionhead goose in the video data based on the markings; calculating the scores of the beak breathing pattern, posture, and gait of each lionhead goose using a preset classification neural network; if the calculated score is greater than a preset abnormality threshold, the marked lionhead goose is identified as potentially abnormal; acquiring the body temperature and sound information of the abnormal lionhead goose, and identifying the corresponding lionhead goose as a suspected sick goose based on the body temperature and sound information to obtain the monitoring results; and generating a notification message to notify the farmers based on the monitoring results.

2. The lion-headed goose monitoring method based on multimodal fusion perception according to claim 1, characterized in that, The step of tracking and identifying each lion-headed goose in the video data based on the marker includes: extracting lion-headed goose images containing the marker from each frame of the video data using a tracking algorithm; sorting the extracted lion-headed goose images with the same marker in chronological order to obtain a lion-headed goose image sequence; and inputting the lion-headed goose image sequence into a preset body posture analysis model to detect and identify the body posture of the lion-headed goose in each lion-headed goose image in the lion-headed goose image sequence.

3. The lion-headed goose monitoring method based on multimodal fusion perception according to claim 2, characterized in that, The step of extracting the lion-headed goose image containing the marker from each frame of the video data using a tracking algorithm includes: determining the current position of the corresponding lion-headed goose in the monitoring screen based on the marker; predicting the movement position of the lion-headed goose in the next frame of the video based on the current position using a Kalman filter algorithm; and capturing the image of the lion-headed goose from the marker at the movement position in the next frame of the video, comparing it with the marker in the previous frame of the video, to obtain the lion-headed goose image.

4. The lion-headed goose monitoring method based on multimodal fusion perception according to claim 2, characterized in that, The step of inputting the lion-head goose image sequence into a preset posture analysis model to detect and identify the posture of the lion-head goose in each image of the lion-head goose image sequence includes: inputting the lion-head goose image sequence into the preset posture analysis model to detect and identify the beak breathing pattern, posture, and gait of the lion-head goose in each image of the lion-head goose image sequence; using a preset binary classification neural network to calculate the score of the beak breathing pattern, posture, and gait of the lion-head goose in each image of the lion-head goose; and determining whether each lion-head goose in each image of the lion-head goose is normal or abnormal based on the score.

5. The lion-headed goose monitoring method based on multimodal fusion sensing according to any one of claims 1-4, characterized in that, The process of acquiring abnormal body temperature and sound information of Lionhead geese, and identifying suspected sick geese based on this information to obtain monitoring results, includes: collecting abnormal calls from Lionhead geese using microphones within the farm, and collecting abnormal body temperatures from temperature sensors within the farm; matching the calls with normal goose calls to determine if they are abnormal; determining if the goose's body temperature is abnormal based on the body temperature; if both the calls and body temperature are abnormal, the goose is determined to be in a diseased state; if either the calls or the body temperature are abnormal, the goose is determined to be in a severely abnormal state.

6. The lion-headed goose monitoring method based on multimodal fusion perception according to claim 5, characterized in that, The step of generating a notification message to farmers based on the monitoring results includes: capturing a frame of an abnormal Lionhead goose from the video data, marking the abnormal Lionhead goose's number and location information on the frame, and generating an image notification message; and sending the image notification message to the farmer's terminal via a mini-program on the monitoring platform.

7. The lion-headed goose monitoring method based on multimodal fusion perception according to claim 6, characterized in that, After generating a notification message to the farmers based on the monitoring results, the process further includes: deploying an optimized and adjusted deep learning object detection algorithm on the device to count the number of abnormal Lionhead geese in the image notification message and pushing the results to the farmers.

8. A monitoring device for lion-headed geese based on multimodal fusion sensing, characterized in that, The multimodal fusion perception-based lionhead goose monitoring device includes: a data acquisition module for acquiring video data of the lionhead goose flock through monitoring equipment installed in the lionhead goose farm; a tagging module for identifying and tagging the lionhead geese in the video data using a preset target detection model; a tracking module for tracking and identifying the body posture of each lionhead goose in the video data based on the tags; calculating scores for the mouth breathing pattern, posture, and gait of each lionhead goose using a preset classification neural network; if the calculated score is greater than a preset abnormality threshold, the tagged lionhead goose is identified as potentially abnormal; a monitoring module for acquiring the body temperature and sound information of abnormal lionhead geese, and identifying the corresponding lionhead geese as potentially sick geese based on the body temperature and sound information to obtain monitoring results; and a notification module for generating a notification message to notify the farmers based on the monitoring results.

9. A computer device, characterized in that, The computer device includes: a memory and at least one processor, the memory storing instructions, the memory and the at least one processor being interconnected via a circuit; the at least one processor invokes the instructions in the memory to cause the computer device to perform the various steps of the Lion Head Goose Monitoring Method based on Multimodal Fusion Perception as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the Lionhead Goose Monitoring Method based on Multimodal Fusion Perception as described in any one of claims 1-7.

Images