Intelligent livestock poultry house inspection robot monitoring and early warning method and system

Abstract

The invention discloses an intelligent livestock poultry house inspection robot monitoring and early warning method and system, and the method comprises the steps: synchronously collecting poultry house image data through a multispectral and infrared thermal imaging module of an inspection robot, carrying out the alignment of multi-source data through a registration algorithm, and carrying out the separation and marking of livestock individuals, environments and interference regions through a semantic and instance segmentation technology; extracting livestock and poultry texture, color and temperature features and environment temperature and humidity and illumination features, and combining threshold judgment and trend analysis to identify abnormity; and an early warning signal is generated and transmitted to a remote terminal and the robot is controlled to perform secondary acquisition in case of abnormality. The system comprises a multi-source image acquisition and transmission unit, a registration and segmentation unit, a feature extraction and quantification unit, an anomaly recognition and analysis unit, an early warning generation and integration unit and a remote interaction control unit, can replace manual inspection, solves the problems of low efficiency, single data and misjudgment and missed judgment of a traditional mode, realizes all-weather accurate monitoring and early warning of the poultry house, and ensures safe production of breeding.

Description

Technical Field

[0001] This invention relates to the field of livestock and poultry house inspection robots, and in particular to a monitoring and early warning method and system for intelligent livestock and poultry house inspection robots. Background Technology

[0002] With the development of livestock farming towards large-scale and intensive operations, the number of livestock and poultry in poultry houses has increased significantly. Traditional manual inspection methods face problems such as low efficiency, limited coverage, and poor real-time performance. Manual inspection requires staff to frequently enter the poultry house, which not only consumes a lot of manpower and resources but also may lead to missed or misjudged abnormal conditions of livestock and poultry due to human error. At the same time, frequent entry and exit can easily disrupt the stable temperature and humidity environment inside the poultry house, affecting the growth of livestock and poultry. In addition, the complex environmental conditions inside the poultry house, such as high dust levels and large changes in light, further increase the difficulty of manual inspection, making it difficult to achieve all-weather, accurate monitoring of the growth status of livestock and poultry and the environment inside the house. There is an urgent need to use intelligent technologies to improve the efficiency and accuracy of inspection and early warning systems, ensuring safe production and efficient management in livestock farming.

[0003] Existing technologies for monitoring livestock and poultry houses have two significant drawbacks. Firstly, existing monitoring systems mostly employ single-type image acquisition devices, only able to obtain images of localized areas within the house. They cannot achieve simultaneous acquisition and fusion analysis of multi-source data, resulting in insufficient extraction of individual livestock and poultry characteristics and environmental features. This makes it difficult to accurately identify abnormal conditions in livestock and poultry; for example, relying solely on visible light images cannot effectively determine abnormal body temperature, easily leading to delayed or false alarms. Secondly, existing early warning methods largely rely on simple threshold comparisons, lacking dynamic analysis of characteristic parameter trends and failing to establish a complete multi-level early warning mechanism. When minor abnormalities occur in the house, timely warning signals cannot be issued, and when the abnormality worsens, it is difficult to generate targeted response instructions based on the severity of the abnormality. This results in insufficient timeliness and practicality of early warnings, failing to provide accurate decision support for livestock managers. Summary of the Invention

[0004] In order to overcome the shortcomings and deficiencies of existing technologies, this invention provides a monitoring and early warning method and system for intelligent livestock and poultry house inspection robots.

[0005] The technical solution adopted in this invention is a monitoring and early warning method for an intelligent livestock and poultry house inspection robot, characterized by the following steps: S1, using a multispectral imaging module and an infrared thermal imaging module mounted on the inspection robot, synchronous image acquisition is performed on livestock and poultry groups in different areas of the poultry house. The multispectral imaging module acquires image data of multiple bands at preset intervals, and the infrared thermal imaging module acquires image data of the temperature distribution on the surface of the livestock and poultry at a preset frame rate; S2, the acquired multispectral image data and infrared thermal imaging image data are transmitted to a data processing module, where an image registration algorithm is used to spatially align the multi-source image data, ensuring that the pixel positions of the same individual livestock or poultry in different types of images correspond one-to-one; S3, target segmentation is performed on the registered image data, and a semantic segmentation algorithm is used to separate the individual livestock / poultry area and the house environment area. S4. The system extracts the feature parameters of each marked livestock and poultry individual area, including texture features and color features based on multispectral images and temperature features based on infrared thermal imaging images. At the same time, it extracts the temperature and humidity features and light intensity features of the indoor environment. S5. The extracted feature parameters are input into the early warning analysis module. Through preset feature threshold judgment and feature trend analysis, the abnormal state of the livestock and poultry individuals and the abnormal situation of the indoor environment are identified. S6. When an abnormal state or abnormal situation is identified, the early warning analysis module generates a corresponding early warning signal and transmits the early warning signal, along with the image data and feature parameter data of the abnormal area, to the remote monitoring terminal. At the same time, it controls the inspection robot to locate the abnormal area and perform secondary image acquisition and feature parameter extraction.

[0006] Furthermore, the image registration algorithm in S2 adopts a feature point-based registration method, and the registration error calculation model is as follows: ,in, For registration error, The number of feature points, For the first in the multispectral image The coordinates of the feature points The first in the infrared thermal imaging image The coordinates of the feature points For the image rotation angle, for Directional translation amount for Directional translation amount. Further, in S4, texture feature extraction employs the gray-level co-occurrence matrix method, and the texture contrast calculation model is: Where Con represents texture contrast. For image gray levels, Coordinates in the gray-level co-occurrence matrix The element value at that position, The step size of the gray-level co-occurrence matrix. The direction angle of the gray-level co-occurrence matrix.

[0007] Furthermore, the characteristic trend analysis in S5 adopts the linear regression analysis method, and the trend slope calculation model is as follows: , The trend slope The number of times the feature parameters were collected. For collection sequence number, For the first The feature parameter values ​​collected this time.

[0008] Furthermore, the abnormal state judgment in S5 adopts a multi-feature fusion judgment method, and the calculation model of the fusion judgment value is as follows: , To merge the judgment values, The number of feature parameters participating in the fusion. For the first The weight coefficients of each feature parameter, For the first The actual values ​​of each feature parameter For the first The minimum value of each feature parameter. For the first The maximum value of each feature parameter.

[0009] Furthermore, the frame rate adjustment for the secondary image acquisition in S6 adopts an adaptive adjustment method, and the adjusted frame rate calculation model is as follows: , The adjusted capture frame rate. This is the initial acquisition frame rate. This is the frame rate adjustment factor. This represents the current level of abnormality. The range of abnormality levels. This represents the maximum degree of abnormality.

[0010] Further, step S3 includes the following sub-steps: S31, inputting the registered multi-source image data into a pre-trained semantic segmentation network, extracting features from the image data through convolutional layers in the network to generate multi-scale feature maps, and then downsampling the feature maps through pooling layers to retain the labeled feature information; S32, performing upsampling operations on the downsampled feature maps to restore the spatial resolution of the image, and simultaneously fusing feature maps of different scales to enhance the segmentation ability for targets of different sizes; S33, generating semantic segmentation results of the image through the output layer of the semantic segmentation network, dividing the image pixels into livestock and poultry individual categories, barn environment categories, and interference categories to obtain a preliminary segmentation image; S34, performing instance segmentation on the livestock and poultry individual category regions in the preliminary segmentation image, determining the bounding box of each livestock and poultry individual through an object detection algorithm, and then, in conjunction with the semantic segmentation results, labeling each livestock and poultry individual region individually.

[0011] Further, S4 includes the following sub-steps: S41, for each marked livestock individual region, extract texture features from the multispectral image, traverse the region through a sliding window, calculate the gray-level co-occurrence matrix within each window, and then extract contrast, correlation, energy, and homogeneity texture feature parameters from the gray-level co-occurrence matrix; S42, extract color features from the livestock individual region of the multispectral image, convert the image from the RGB color space to the HSV color space, and calculate the mean, variance, maximum, and minimum values ​​of the H channel, S channel, and V channel as color feature parameters; S43, extract temperature features from the livestock individual region of the infrared thermal imaging image, traverse the temperature values ​​of all pixels in the region, and calculate the average temperature, maximum temperature, minimum temperature, and temperature standard deviation in the region as temperature feature parameters; S44, for the indoor environment region, extract temperature and humidity features and light intensity features through image analysis algorithms, and calculate the average temperature and humidity, average light intensity, and environmental feature parameters of the environment region by combining them with a preset reference region in the image.

[0012] Further, S5 includes the following sub-steps: S51, comparing the individual livestock and poultry characteristic parameters extracted in S4 with the preset normal characteristic parameter threshold range, and determining whether each characteristic parameter exceeds the threshold range. If it exceeds, it is marked as the abnormal type corresponding to that characteristic parameter; S52, performing trend analysis on multiple sets of continuously collected individual livestock and poultry characteristic parameters, and determining whether the characteristic parameters show an abnormal trend by calculating the rate of change and trend of change of the characteristic parameters. If it does, it is marked as a trend abnormality type; S53, comparing the indoor environmental characteristic parameters with the preset normal environmental threshold range, and determining whether the environmental characteristic parameters exceed the threshold. If it exceeds, it is marked as an environmental abnormality type; S54, combining the abnormal markings of individual livestock and poultry characteristic parameters, trend abnormality markings, and indoor environmental abnormality markings, and determining whether there is an abnormal state of individual livestock and poultry or an abnormal situation of indoor environment through logical judgment rules, forming a preliminary abnormality identification result.

[0013] A monitoring and early warning system for intelligent livestock and poultry house inspection robots is provided. This system, applied to the monitoring and early warning methods of intelligent livestock and poultry house inspection robots, includes: a multi-source image acquisition and synchronous transmission unit, mounted on the inspection robot body, consisting of a multispectral imaging component, an infrared thermal imaging component, and a data transmission subunit. The multispectral imaging component acquires multi-band image data of the poultry house at preset band intervals and resolutions, while the infrared thermal imaging component acquires image data of the livestock and poultry body surface temperature distribution at a preset frame rate. The data transmission subunit synchronously transmits the image data to the data processing unit; a multi-source image registration and target segmentation unit, connected to the multi-source image acquisition and synchronous transmission unit, includes built-in image registration algorithm modules and target segmentation algorithm modules, which perform spatial alignment processing on the multi-source image data, semantic segmentation and instance segmentation on the registered image data, and separate and label individual livestock and poultry, the house environment, and interference areas; and a multi-dimensional feature extraction and quantization unit, connected to the multi-source image registration and target segmentation unit, including livestock and poultry individual feature extraction submodules and house environment feature extraction submodules, which extract features from the labeled livestock and poultry... The system extracts texture, color, and temperature features from individual poultry areas, and extracts temperature, humidity, and light intensity features from the indoor environment. All features are then quantified. An abnormal state identification and trend analysis unit, connected to the multi-dimensional feature extraction and quantification unit, includes a feature threshold judgment module, a feature trend calculation module, and an abnormal comprehensive judgment module. These modules compare feature parameters with preset thresholds, analyze feature parameter trends, and combine the results to identify abnormal states and situations. A multi-level early warning signal generation and data integration unit, connected to the abnormal state identification and trend analysis unit, generates multi-level early warning signals based on the type and severity of the abnormality. It also integrates image data of the abnormal area, feature parameter data, and early warning signals to form an early warning information package. A remote terminal interaction and robot control unit, connected to both the multi-level early warning signal generation and data integration unit and the inspection robot itself, receives the early warning information package and transmits it to the remote monitoring terminal. Simultaneously, it generates control commands based on the early warning information to control the inspection robot to locate the abnormal area and perform secondary image acquisition and feature parameter extraction operations.

[0014] Compared with the prior art, this application has at least one of the following beneficial effects: This invention proposes a monitoring and early warning method and system for intelligent livestock and poultry house inspection robots. By equipping the inspection robot with multispectral and infrared thermal imaging modules, it can achieve simultaneous acquisition of multi-source images, replacing traditional manual inspections, greatly improving inspection efficiency and coverage, and avoiding the disruption of environmental stability caused by frequent entry and exit of staff into poultry houses. At the same time, by using image registration and target segmentation technology, it can accurately separate individual livestock and poultry, the environment, and interference areas, solving the problem of incomplete data collection by a single device. By extracting the texture, color, and temperature features of livestock and poultry, as well as the environmental temperature, humidity, and light features, and combining multi-feature fusion judgment and trend analysis, we can accurately identify abnormal conditions of livestock and poultry and environmental anomalies, thus overcoming the shortcomings of traditional single threshold comparison which cannot perform dynamic analysis. The multi-level early warning signal generation and data integration unit can generate corresponding early warnings based on the degree of anomaly. Combined with remote terminal interaction and robot secondary data acquisition and control, it not only solves the problem of misjudgment and omission by manual inspection, but also forms a complete early warning response mechanism. It can issue early warnings for minor anomalies in a timely manner and generate targeted instructions, providing comprehensive and accurate decision support for livestock and poultry management personnel, and ensuring safe production and efficient management of livestock and poultry farming. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method steps of the present invention; Figure 2 This is a diagram showing the system unit composition of the present invention. Detailed Implementation

[0016] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] like Figure 1 As shown, the intelligent livestock and poultry house inspection robot monitoring and early warning method includes the following steps: S1, through the multispectral imaging module and infrared thermal imaging module carried by the inspection robot, synchronously acquires images of livestock and poultry groups in different areas of the poultry house. The multispectral imaging module acquires image data of multiple bands at preset band intervals, and the infrared thermal imaging module acquires image data of the temperature distribution of the livestock and poultry body surface at a preset frame rate. Specifically, step S1 is the data acquisition foundation of the entire monitoring and early warning method. It utilizes the multispectral imaging module and infrared thermal imaging module mounted on the inspection robot to simultaneously acquire image data within the poultry house. The multispectral imaging module acquires data at preset band intervals, specifically 10 to 50 nanometers, covering the visible and near-infrared bands from 400 to 1000 nanometers. Each acquisition captures image data from 8 to 12 bands, with an image resolution of 1920×1080 pixels to ensure the capture of subtle physical features of individual livestock and poultry. The infrared thermal imaging module acquires data at a preset frame rate of 15 to 30 frames per second, providing real-time image data of the temperature distribution on the surface of livestock and poultry. The temperature detection range covers -20℃ to 150℃, with a temperature resolution of 0.05℃, accurately reflecting changes in the surface temperature of livestock and poultry. During implementation, the inspection robot moves along a preset inspection path, covering all breeding areas in the poultry house, including feeding areas, drinking areas, and rest areas. The time spent collecting data in each area is set to 30 to 60 seconds to ensure that the collected data can fully cover the livestock and poultry groups in different locations in the poultry house, providing complete and accurate raw data support for subsequent data processing and avoiding subsequent analysis errors due to incomplete collection areas or insufficient data accuracy.

[0018] S2 transmits the collected multispectral image data and infrared thermal imaging image data to the data processing module. The multi-source image data is spatially aligned using an image registration algorithm, so that the pixel positions of the same livestock individual correspond one-to-one in different types of images. Specifically, step S2 is the preprocessing stage for multi-source image data. Its purpose is to achieve spatial alignment between multispectral image data and infrared thermal imaging image data, laying the foundation for subsequent target segmentation and feature extraction. In implementation, the two types of image data acquired in S1 are first transmitted to the data processing module via wired or wireless transmission at a rate of no less than 100Mbps to ensure real-time data transmission and avoid data backlog. The data processing module incorporates an image registration algorithm. The algorithm first performs feature point detection on the two types of images, using the Harris corner detection algorithm to extract feature points. At least 500 feature points are extracted from each image, and these feature points must be evenly distributed across the image to avoid concentration in localized areas that could lead to registration errors. Subsequently, a feature point matching algorithm is used to pair the feature points of the two types of images, achieving a matching accuracy of over 85%. Incorrectly matched feature points are removed using the RANSAC algorithm to ensure matching accuracy. Finally, the image spatial transformation parameters are calculated based on the matched feature points to achieve spatial alignment of multi-source images. After alignment, the pixel position deviation of the same livestock individual in the two types of images does not exceed 2 pixels, ensuring that the subsequent feature extraction of the same livestock individual can be based on a consistent spatial position, avoiding errors in feature parameter extraction due to image misalignment, and improving the reliability of subsequent analysis results.

[0019] S3. Target segmentation is performed on the registered image data. Separate the individual livestock and poultry region, the indoor environment region, and the interference region through the semantic segmentation algorithm. Then, each individual livestock and poultry region is individually labeled through the instance segmentation algorithm. Specifically, step S3 is the target segmentation stage, which separates and labels the effective target regions from the registered multi-source image data, providing a clear analysis object for accurate feature parameter extraction. In practice, the registered image data is first preprocessed, including image denoising using a Gaussian filtering algorithm with a kernel size of 3×3 or 5×5 to remove salt-and-pepper noise and Gaussian noise, achieving a noise removal rate of at least 90% while preserving target edge information. Then, a semantic segmentation algorithm is initiated, employing semantic segmentation networks such as U-Net or MaskR-CNN. The network input is the registered multi-source image fusion data, and the output consists of three types of pixel labels, corresponding to the individual livestock / poultry region, the indoor environment region, and the interference region, respectively. The intersection-over-union (IoU) ratio of the semantic segmentation needs to reach over 80% to ensure accurate separation of the three types of regions. For the separated livestock and poultry individual regions, the instance segmentation algorithm is then launched. The algorithm first determines the bounding box of each livestock and poultry individual through the object detection network. The bounding box positioning error does not exceed 5 pixels. Then, combined with the semantic segmentation results, the livestock and poultry individual regions within each bounding box are individually labeled, and a unique identifier is assigned to each livestock and poultry individual. The labeling accuracy needs to reach more than 90% to ensure that feature parameters can be extracted separately for each livestock and poultry individual in the future, avoiding confusion of feature parameters of different livestock and poultry individuals and improving the targeting and accuracy of feature extraction.

[0020] S4, extract the feature parameters of each marked livestock and poultry individual area, including texture features and color features based on multispectral images and temperature features based on infrared thermal imaging images, and extract the temperature and humidity features and light intensity features of the indoor environment area. Specifically, step S4 is the feature parameter extraction stage, which requires extracting multi-dimensional feature parameters from the livestock and poultry individual areas and the indoor environment areas marked in S3 to provide data for subsequent anomaly identification. For the livestock and poultry individual areas, texture features are first extracted using the gray-level co-occurrence matrix method. A 16×16 pixel sliding window is divided within the marked area, with a window step size of 8 pixels. A gray-level co-occurrence matrix is ​​calculated for each window, with 256 gray levels, a step size of 1 or 2, and four orientation angles of 0°, 45°, 90°, and 135°. Four texture parameters—contrast, correlation, energy, and homogeneity—are extracted from the matrix, with each parameter retained to two decimal places to ensure the detail of the texture features. Next, color features are extracted by converting the multispectral image from the RGB color space to the HSV color space. The mean, variance, maximum, and minimum values ​​of the H, S, and V channels within the marked area are calculated, with four parameters extracted for each channel, for a total of 12 color feature parameters. The parameter calculation accuracy is retained to three decimal places to accurately reflect the color changes of individual livestock and poultry. Finally, temperature features are extracted by iterating through all pixels in the marked area of ​​the infrared thermal imaging image, calculating the average temperature, maximum temperature, minimum temperature, and temperature standard deviation within the area. Temperature values ​​are retained to one decimal place to reflect the temperature distribution on the surface of livestock and poultry. For the indoor environment area, temperature and humidity features and light intensity features are extracted using image analysis algorithms. Combined with a preset reference area in the image (the area of ​​the reference area is not less than 5% of the total image area), the mean temperature and humidity and mean light intensity of the environmental area are calculated. Temperature and humidity accuracy is retained to one decimal place, and light intensity accuracy is retained to an integer to ensure that the environmental feature parameters can accurately reflect the indoor environmental conditions.

[0021] S5. Input the extracted feature parameters into the early warning analysis module. Through preset feature threshold judgment and feature trend analysis, identify the abnormal state of individual livestock and poultry and the abnormal conditions of the indoor environment. Specifically, step S5 is the anomaly identification stage. By analyzing the feature parameters extracted in S4, it determines whether there are any abnormalities in the individual livestock and poultry or the environment inside the shed, providing a basis for subsequent early warning. In implementation, the feature parameters of the individual livestock and poultry are first compared with preset normal feature parameter threshold ranges. These normal threshold ranges are based on statistical analysis of a large number of healthy livestock and poultry. For example, the normal range for the average temperature of individual livestock and poultry is set to 38.5℃ to 40.5℃, the normal range for texture contrast is set to 100 to 300, and the normal range for the average H channel value in color features is set to 0.3 to 0.6. If a feature parameter exceeds the corresponding threshold range, it is marked as the corresponding abnormality type. For example, a temperature exceeding the range is marked as "abnormal body temperature," and a texture contrast exceeding the range is marked as "abnormal body surface texture." Secondly, trend analysis is performed on multiple sets (no less than 5 sets) of continuously collected individual livestock and poultry feature parameters to calculate the rate of change and trend of the feature parameters. For example, if the temperature shows an upward trend in three consecutive collections, and each increase exceeds 0.2℃, it is marked as "abnormal body temperature trend," avoiding misjudgment due to single data fluctuations. Simultaneously, the environmental characteristic parameters inside the shed are compared with preset normal environmental threshold ranges. For example, the normal range for ambient temperature is set to 20℃ to 28℃, the normal range for humidity is set to 50% to 70%, and the normal range for light intensity is set to 2000 lux to 5000 lux. If the range is exceeded, it is marked as "abnormal ambient temperature," "abnormal ambient humidity," or "abnormal light intensity." Finally, by combining all abnormal markers, logical judgment rules are used to determine whether there is an abnormal state (e.g., if "abnormal body temperature" and "abnormal body surface texture" are present simultaneously, it is determined to be an individual livestock or poultry abnormality; if "abnormal ambient temperature" is present and lasts for more than 5 minutes, it is determined to be an abnormal environment inside the shed). The anomaly identification accuracy rate needs to reach more than 85% to ensure accurate identification of real abnormal situations and reduce misjudgments and omissions. S6, when an abnormal state or abnormal situation is identified, the early warning analysis module generates a corresponding early warning signal and transmits the early warning signal, along with the image data and feature parameter data of the abnormal area, to the remote monitoring terminal. At the same time, the inspection robot is controlled to locate the abnormal area and perform secondary image acquisition and feature parameter extraction.

[0022] Specifically, step S6 is the early warning and secondary verification stage. After an anomaly is identified in S5, the early warning mechanism is activated, and the robot is controlled to perform secondary data collection and verification to ensure the accuracy of the anomaly judgment and the timeliness of subsequent processing. During implementation, if S5 identifies an abnormal state or situation, the early warning analysis module immediately generates a corresponding early warning signal. The early warning signals are divided into three levels according to the severity of the anomaly: Level 1 is a minor anomaly (e.g., ambient humidity slightly exceeding the normal range but not affecting the livestock's condition); Level 2 is a moderate anomaly (e.g., the body temperature of individual livestock slightly exceeding the range); and Level 3 is a severe anomaly (e.g., the body temperature of multiple livestock significantly exceeding the range or the ambient temperature severely deviating from the normal range). Different levels of early warning signals correspond to different early warning prompts. For example, a Level 1 early warning only displays a text prompt on the remote monitoring terminal; a Level 2 early warning displays a text prompt accompanied by an audible alarm; and a Level 3 early warning, in addition to text and audible prompts, also sends an early warning SMS to the manager's mobile phone. Simultaneously, the early warning signal, along with image data and feature parameter data of the abnormal area, is transmitted to the remote monitoring terminal with a transmission delay of no more than 10 seconds to ensure that managers can obtain anomaly details in a timely manner. In addition, the early warning analysis module generates control commands and transmits them to the inspection robot. The robot adjusts its inspection path and locates the abnormal area with a positioning accuracy error of no more than 1 meter. After reaching the abnormal area, it initiates secondary image acquisition. The parameters such as band interval, frame rate, and resolution of the secondary acquisition are consistent with those of S1, but the acquisition time is extended to 60 to 90 seconds, and the number of acquisitions is increased to 2. At the same time, feature parameters are re-extracted, and the feature parameters of the secondary acquisition are compared with the data of the first acquisition. If both sets of data show an anomaly, the anomaly is confirmed to exist. If the secondary data is normal, the initial data is judged to be a misjudgment, avoiding false warnings caused by accidental factors, improving the reliability of the early warning, and providing an accurate basis for subsequent aquaculture management decisions.

[0023] Preferably, the image registration algorithm in S2 adopts a feature point-based registration method, and the registration error calculation model is as follows: ,in, For registration error, The number of feature points, For the first in the multispectral image The coordinates of the feature points The first in the infrared thermal imaging image The coordinates of the feature points For the image rotation angle, for Directional translation amount for Directional translation. Specifically, the image registration algorithm in step S2 explicitly adopts a feature-point-based registration method and calculates the registration error through a specific model. The purpose is to quantitatively evaluate the accuracy of spatial alignment of multi-source images and ensure the accuracy of subsequent target segmentation and feature extraction. During implementation, the number of feature points involved in registration is first determined. The number of feature points extracted from each image needs to be controlled between 500 and 800, and they need to be evenly distributed in the livestock / poultry individual area, environmental area, and edge area of ​​the image to avoid local registration deviations due to feature point concentration. After obtaining the coordinates of the corresponding feature points in the multispectral image and infrared thermal imaging image, the coordinate data needs to be filtered to remove invalid coordinates caused by image noise or occlusion. The effective coordinate retention rate needs to reach over 90%. Then, the coordinates are substituted into the registration error calculation model, where the image rotation angle is set to a range of -10° to 10°, and the translation values ​​in both the X and Y directions are set to a range of -50 pixels to 50 pixels. The registration error value is then calculated through the model. In practical applications, the registration error needs to be controlled within the range of 1 to 3 pixels. If the error exceeds this range, the feature point matching results need to be readjusted or the spatial transformation parameters optimized until the error meets the requirements. This model intuitively reflects the registration effect through quantitative indicators, avoiding the subjectivity of traditional qualitative evaluation, and ensuring that the pixel positions of the same livestock individual correspond accurately in different images, providing a reliable spatial basis for the subsequent extraction of feature parameters of a unified target region.

[0024] Preferably, the texture feature extraction in S4 adopts the gray-level co-occurrence matrix method, and the texture contrast calculation model is as follows: Where Con represents texture contrast. For image gray levels, Coordinates in the gray-level co-occurrence matrix The element value at that position, The step size of the gray-level co-occurrence matrix. The orientation angle is the gray-level co-occurrence matrix. Specifically, the gray-level co-occurrence matrix method for texture feature extraction in step S4 calculates texture contrast through a specific model, accurately quantifying the differences in surface texture of individual livestock and poultry, and providing a basis for judging texture dimension for identifying abnormal health status of livestock and poultry. In implementation, the image gray levels are first determined. Based on the resolution and gray-level distribution characteristics of the multispectral image, the gray levels are set to 256 levels to ensure detailed reflection of image gray-level changes. Then, the step size of the gray-level co-occurrence matrix is ​​set, with a step size of 1 or 2 pixels. Too large a step size will lead to loss of texture information, while too small a step size will increase the computational load; a reasonable selection needs to be made based on the texture density of individual livestock and poultry in the image. The orientation angle is set to four directions: 0°, 45°, 90°, and 135°, to comprehensively capture texture features in different directions. After calculating the gray-level co-occurrence matrix, it is substituted into the texture contrast calculation model. The model obtains the texture contrast value by accumulating the product of the squared gray-level difference between pixel pairs with different gray-level values ​​and the corresponding matrix element value. In practical applications, the texture contrast of healthy livestock and poultry individuals is typically between 100 and 300. If the texture contrast of an individual livestock or poultry exceeds this range, such as being below 80 or above 350, it may indicate abnormalities such as feather loss or skin lesions. This model transforms abstract texture features into quantifiable values, facilitating subsequent comparison with normal thresholds and improving the objectivity and accuracy of identifying abnormal conditions in livestock and poultry.

[0025] Preferably, the feature trend analysis in S5 adopts the linear regression analysis method, and the trend slope calculation model is as follows: , The trend slope The number of times the feature parameters were collected. For collection sequence number, For the first The feature parameter values ​​collected in step S5 are as follows. Specifically, the feature trend analysis in step S5 uses linear regression analysis and a specific model to calculate the trend slope. This dynamically captures the changing patterns of individual livestock and poultry feature parameters, avoiding misjudgments caused by relying solely on single parameter values ​​and improving the timeliness of anomaly identification. During implementation, the number of feature parameter collections is first determined, requiring 5 to 10 consecutive data sets collected at 5 to 10-minute intervals to ensure the trend of feature parameters over a certain period is reflected. Too few collections will lead to inaccurate trend analysis, while too many will increase data processing pressure. The collected feature parameter values ​​are then sorted by collection sequence number, starting from 1 and increasing sequentially. Each sequence number corresponds to a set of feature parameter values. The sequence number and corresponding feature parameter value are substituted into the trend slope calculation model. The model calculates the covariance correlation term between the collection sequence number and the feature parameter value in the numerator and the variance correlation term of the collection sequence number in the denominator, ultimately yielding the trend slope. In practical applications, if the slope of the trend is positive and the absolute value is greater than 0.5, it indicates that the characteristic parameter is showing a significant upward trend, such as a continuous rise in the body temperature of livestock and poultry; if the slope is negative and the absolute value is greater than 0.5, it indicates that the characteristic parameter is showing a significant downward trend. This model quantifies the rate and direction of change of the characteristic parameter through the slope, and can detect abnormal trends in advance before the characteristic parameter exceeds the static threshold. For example, when the body temperature rises slowly but does not reach the abnormal threshold, the positive slope can be used to predict possible abnormal body temperature, providing support for early warning.

[0026] Preferably, the abnormal state judgment in S5 adopts a multi-feature fusion judgment method, and the calculation model of the fusion judgment value is as follows: , To merge the judgment values, The number of feature parameters participating in the fusion. For the first The weight coefficients of each feature parameter, For the first The actual values ​​of each feature parameter For the first The minimum value of each feature parameter. For the first The maximum value of each feature parameter. Specifically, in step S5, the abnormal state judgment adopts a multi-feature fusion judgment method and calculates the fusion judgment value through a specific model. This integrates feature parameters from multiple dimensions, avoiding the limitations of single feature parameter judgment and improving the comprehensiveness and reliability of abnormal identification. During implementation, the number of feature parameters involved in the fusion is first determined. Based on the physiological characteristics and abnormal manifestations of individual livestock and poultry, 5 to 8 key feature parameters are typically selected, such as average body surface temperature, texture contrast, color H channel mean, color S channel variance, temperature standard deviation, etc., to ensure coverage of multiple dimensions such as temperature, texture, and color. Then, a weight coefficient is set for each feature parameter, with a total weight coefficient of 1. The weight is assigned according to the degree of influence of each feature parameter on the abnormal state. For example, the average body surface temperature has a greater impact on health status, so its weight is set to 0.3 to 0.4; the texture contrast weight is set to 0.2 to 0.3; and the weights of other parameters are adjusted sequentially according to their importance. After normalizing each feature parameter, it is substituted into the fusion judgment value calculation model. The model obtains the fusion judgment value by accumulating the product of the normalized value of each feature parameter and its corresponding weight coefficient. In practical applications, the fusion judgment value is typically set to a threshold of 0.6; values ​​greater than 0.6 are considered abnormal, while values ​​less than 0.6 are considered normal. This model integrates multi-dimensional information to avoid misjudgments caused by a single parameter being abnormal while other parameters are normal. Furthermore, by assigning weights, it highlights the role of key parameters, thereby improving the scientific rigor of anomaly detection.

[0027] Preferably, the frame rate adjustment for the secondary image acquisition in S6 adopts an adaptive adjustment method, and the adjusted frame rate calculation model is as follows: , The adjusted capture frame rate. This is the initial acquisition frame rate. This is the frame rate adjustment factor. This represents the current level of abnormality. The range of abnormality levels. This represents the maximum degree of abnormality.

[0028] Specifically, the secondary image acquisition frame rate adjustment in step S6 employs an adaptive adjustment method and calculates the adjusted frame rate using a specific model. The acquisition frequency is dynamically optimized based on the degree of anomaly, ensuring sufficient abnormal data acquisition while avoiding unnecessary resource waste. During implementation, the initial acquisition frame rate is first determined, maintaining consistency with the frame rate in step S1, and set to 15 to 30 frames per second to ensure the stability of the basic acquisition frequency. Then, a frame rate adjustment coefficient is set. Based on the inspection robot's image processing capabilities and data transmission bandwidth, the adjustment coefficient is set to 0.5 to 1.0. A coefficient that is too large may cause the frame rate to exceed the device's processing capacity, while a coefficient that is too small will not effectively increase the acquisition frequency. Simultaneously, the range of anomaly severity values ​​is determined, dividing the anomaly severity into a range of 0 to 10, where 0 represents no anomaly and 10 represents the most severe anomaly. The specific anomaly severity value is determined based on the degree to which the characteristic parameters of the anomaly region deviate from the normal threshold. The anomaly severity threshold is set to 3 to 5; values ​​below this threshold indicate minor anomalies, while values ​​above indicate moderate or severe anomalies. The maximum anomaly severity value is fixed at 10. Substituting the above parameters into the frame rate adjustment model, the model obtains the adjusted frame rate by multiplying the initial frame rate by (1 plus the product of the adjustment coefficient and the percentage of the anomaly deviation). In practical applications, if the anomaly level is 8 (higher than the threshold of 5), the adjustment coefficient is 0.8, and the initial frame rate is 20 frames / second, then the adjusted frame rate is 20 × (1 + 0.8 × (8-5) / (10-5)) = 29.6 frames / second, which is close to 30 frames / second, allowing for more intensive collection of data from anomaly areas. This model achieves adaptive frame rate optimization, matching appropriate collection frequencies for different anomaly levels, improving the effectiveness of secondary data collection and the utilization rate of equipment resources.

[0029] Preferably, step S3 includes the following sub-steps: S31, inputting the registered multi-source image data into a pre-trained semantic segmentation network, extracting features from the image data through convolutional layers in the network to generate multi-scale feature maps, and then downsampling the feature maps through pooling layers to retain the labeled feature information; S32, performing upsampling operations on the downsampled feature maps to restore the spatial resolution of the image, and simultaneously fusing feature maps of different scales to enhance the segmentation ability for targets of different sizes; S33, generating semantic segmentation results of the image through the output layer of the semantic segmentation network, dividing the image pixels into livestock and poultry individual categories, barn environment categories, and interference categories to obtain a preliminary segmentation image; S34, performing instance segmentation on the livestock and poultry individual category regions in the preliminary segmentation image, determining the bounding box of each livestock and poultry individual through an object detection algorithm, and then, in conjunction with the semantic segmentation results, labeling each livestock and poultry individual region separately.

[0030] Specifically, step S3, the target segmentation process, includes four sub-steps. It achieves accurate separation and labeling of individual livestock, the indoor environment, and disturbance areas through multi-level processing, clarifying the target range for subsequent feature extraction. In implementation, S31 first inputs the registered multi-source image data into a pre-trained semantic segmentation network. The network's convolutional layers use 3×3 or 5×5 kernels with a stride value of 1 and the same padding method. Multiple rounds of convolution extract features of different dimensions from the image, generating 4 to 6 layers of multi-scale feature maps. Then, max pooling or average pooling layers are used to downsample the feature maps, with a pooling kernel size of 2×2 and a stride value of 2, preserving key features while reducing data volume. S32 upsamples the downsampled feature maps using bilinear interpolation, restoring the feature map resolution to 1 / 3 of the original image. The process involves a 2-1x multiplication, simultaneously fusing feature maps of different scales through skip connections to enhance the segmentation ability for small-sized livestock and poultry individuals. In step S33, the softmax output layer of the semantic segmentation network generates pixel category probability maps, classifying pixels into three categories with a probability threshold of 0.6. Pixels higher than the threshold are assigned to their corresponding categories, resulting in a preliminary segmented image. In step S34, YOLO or Faster R-CNN object detection algorithms are used to determine the bounding boxes of livestock and poultry individuals, with an IoU threshold of 0.5. Pixels not belonging to the livestock and poultry category within the bounding boxes are removed based on the semantic segmentation results. A unique ID is assigned to each livestock and poultry individual, with a labeling accuracy exceeding 90%. This step, through phased processing, addresses the issue of insufficient accuracy from a single segmentation algorithm, ensuring that subsequent feature extraction targets only the effective target regions.

[0031] Preferably, step S4 includes the following sub-steps: S41, for each marked livestock individual region, extract texture features from the multispectral image, traverse the region through a sliding window, calculate the gray-level co-occurrence matrix within each window, and then extract contrast, correlation, energy, and homogeneity texture feature parameters from the gray-level co-occurrence matrix; S42, extract color features from the livestock individual region of the multispectral image, convert the image from the RGB color space to the HSV color space, and calculate the mean, variance, maximum, and minimum values ​​of the H channel, S channel, and V channel as color feature parameters; S43, extract temperature features from the livestock individual region of the infrared thermal imaging image, traverse the temperature values ​​of all pixels in the region, and calculate the average temperature, maximum temperature, minimum temperature, and temperature standard deviation in the region as temperature feature parameters; S44, for the indoor environment region, extract temperature and humidity features and light intensity features through image analysis algorithms, and calculate the average temperature and humidity, average light intensity, and environmental feature parameters of the environment region by combining them with a preset reference region in the image.

[0032] Specifically, the feature extraction process in step S4 includes four sub-steps. The core is to comprehensively extract multi-dimensional features of livestock individuals and the environment from the marked region, providing rich data support for anomaly identification. In implementation, S41 uses a 16×16 pixel sliding window to traverse the livestock individual region, with a window step size of 8 pixels. For each window, a gray-level co-occurrence matrix is ​​calculated, with 256 gray levels, a step size of 1, and orientation angles of 0°, 45°, 90°, and 135°. Four to six texture features, such as contrast and correlation, are extracted from this matrix. S42 converts the multispectral image from RGB space to HSV space, calculating the mean, variance, maximum, and minimum values ​​of the H, S, and V channels using formulas. Four parameters are extracted for each channel, resulting in a total of 12 color features. During calculation, pixels are sampled... The sampling interval is set to 1 to ensure all pixels within the coverage area are covered. In step S43, the individual livestock areas in the infrared thermal imaging image are traversed, and the temperature values ​​of all pixels within the area are statistically analyzed. The average temperature, maximum temperature, minimum temperature, and temperature standard deviation are calculated, with temperature statistics retained to one decimal place. In step S44, for the indoor environment area, 3 to 5 preset reference areas are selected from the image. The area of ​​each reference area is set to 5% to 8% of the total image area. The mean environmental temperature and humidity, and mean light intensity are calculated based on the correspondence between the image grayscale values ​​and the preset calibration curve. The temperature and humidity accuracy is retained to one decimal place, and the light intensity accuracy is retained as an integer. This step, through multi-dimensional feature extraction, avoids recognition bias caused by single features and improves the comprehensiveness of anomaly identification.

[0033] Preferably, step S5 includes the following sub-steps: S51, comparing the individual livestock and poultry characteristic parameters extracted in S4 with a preset normal characteristic parameter threshold range to determine whether each characteristic parameter exceeds the threshold range; if it does, it is marked as the abnormal type corresponding to that characteristic parameter; S52, performing trend analysis on multiple sets of continuously collected individual livestock and poultry characteristic parameters, and determining whether the characteristic parameters show an abnormal trend by calculating the rate of change and trend of change of the characteristic parameters; if so, it is marked as a trend abnormality type; S53, comparing the indoor environmental characteristic parameters with a preset normal environmental threshold range to determine whether the environmental characteristic parameters exceed the threshold; if so, it is marked as an environmental abnormality type; S54, combining the abnormality markings of individual livestock and poultry characteristic parameters, trend abnormality markings, and indoor environmental abnormality markings, and determining whether there is an abnormal state of individual livestock and poultry or an abnormal situation in the indoor environment through logical judgment rules, forming a preliminary abnormality identification result.

[0034] Specifically, the anomaly identification process in step S5 includes four sub-steps. Through multi-dimensional judgment and comprehensive analysis, it accurately identifies abnormalities in individual livestock and poultry and the environment, reducing misjudgments and omissions. During implementation, S51 compares the characteristic parameters of individual livestock and poultry with preset normal thresholds. These normal thresholds are based on statistical analysis of over 1000 sets of healthy livestock and poultry data. For example, the average body surface temperature threshold is set to 38.5℃ to 40.5℃, and the texture contrast threshold is set to 100 to 300. If the values ​​exceed these thresholds, the corresponding abnormality type is marked, with a marking coverage rate of over 95%. S52 sorts 5 to 8 consecutive sets of characteristic parameters by collection time and calculates the rate of change between adjacent sets. The rate of change threshold is set to 10%. If the rate of change for 3 consecutive sets exceeds the threshold or shows a certain abnormality, the abnormality is considered cleared. If a monotonous trend is observed, it is marked as an abnormal trend. S53 compares environmental characteristic parameters with preset thresholds, such as an ambient temperature threshold of 20℃ to 28℃, a humidity threshold of 50% to 70%, and a light intensity threshold of 2000 lux to 5000 lux. If these thresholds are exceeded, an environmental abnormality is marked. S54 uses logical AND / OR rules to synthesize all markers. If two or more individual characteristic abnormalities or one trend abnormality are present simultaneously, it is determined to be an individual livestock or poultry abnormality. If environmental parameters exceed the thresholds and the duration exceeds 5 minutes, it is determined to be an environmental abnormality. This step, through phased verification and comprehensive judgment, overcomes the limitations of single-threshold judgment and improves the accuracy of abnormality identification.

[0035] like Figure 2As shown, the intelligent livestock and poultry house inspection robot monitoring and early warning system is applied to the intelligent livestock and poultry house inspection robot monitoring and early warning method. It includes: a multi-source image acquisition and synchronous transmission unit, mounted on the inspection robot body, consisting of a multispectral imaging component, an infrared thermal imaging component, and a data transmission subunit. The multispectral imaging component acquires multi-band image data of the poultry house at preset band intervals and resolutions, the infrared thermal imaging component acquires image data of the livestock and poultry body surface temperature distribution at a preset frame rate, and the data transmission subunit synchronously transmits the image data to the data processing unit; a multi-source image registration and target segmentation unit, connected to the multi-source image acquisition and synchronous transmission unit, with built-in image registration algorithm module and target segmentation algorithm module, respectively performing spatial alignment processing on the multi-source image data, semantic segmentation and instance segmentation on the registered image data, separating and marking individual livestock and poultry, the house environment, and interference areas; and a multi-dimensional feature extraction and quantization unit, connected to the multi-source image registration and target segmentation unit, including a livestock and poultry individual feature extraction submodule and a house environment feature extraction submodule, respectively extracting and quantizing features from the marked images. The system extracts texture, color, and temperature features from individual livestock and poultry areas, and extracts temperature, humidity, and light intensity features from the indoor environment. All features are then quantified. An abnormal state identification and trend analysis unit, connected to the multi-dimensional feature extraction and quantification unit, includes a feature threshold judgment module, a feature trend calculation module, and an abnormal comprehensive judgment module. These modules compare feature parameters with preset thresholds, analyze feature parameter trends, and combine the results to identify abnormal states and situations. A multi-level early warning signal generation and data integration unit, connected to the abnormal state identification and trend analysis unit, generates multi-level early warning signals based on the type and severity of the abnormality. It also integrates abnormal area image data, feature parameter data, and early warning signals to form an early warning information package. A remote terminal interaction and robot control unit, connected to both the multi-level early warning signal generation and data integration unit and the inspection robot itself, receives the early warning information package and transmits it to the remote monitoring terminal. Simultaneously, it generates control commands based on the early warning information to control the inspection robot to locate the abnormal area and perform secondary image acquisition and feature parameter extraction operations.

[0036] A monitoring and early warning method and system for intelligent livestock and poultry house inspection robots is proposed. Equipped with multispectral imaging and infrared thermal imaging modules, the robot enables simultaneous acquisition of multi-source images, replacing traditional manual inspection methods. This significantly improves inspection efficiency and coverage, reduces manpower and material costs, and avoids the problem of frequent staff entry and exit from the poultry house disrupting the stable temperature and humidity environment. Simultaneously, image registration algorithms are used to spatially align multi-source data, and semantic segmentation and instance segmentation techniques are employed to accurately separate individual livestock and poultry, the house environment, and interference areas. This effectively addresses the shortcomings of existing technologies, such as incomplete data acquisition from single devices and low target recognition accuracy, laying a reliable data foundation for subsequent feature extraction and anomaly detection.

[0037] In terms of anomaly identification and early warning control, this method and system extracts the texture, color, and temperature features of individual livestock and poultry, as well as the temperature, humidity, and light intensity features of the indoor environment. Combined with multi-feature fusion judgment and feature trend analysis, it achieves accurate identification of abnormal livestock and poultry states and environmental anomalies, overcoming the problems of traditional single-threshold comparisons which cannot dynamically analyze and are prone to misjudgment and missed judgment. Furthermore, the multi-level early warning signal generation unit can generate corresponding early warnings based on the degree of anomaly. Combined with remote terminal interaction and robot secondary data acquisition control, a complete early warning response mechanism is formed. This not only solves the shortcomings of manual inspections in terms of poor real-time performance and inability to promptly capture minor anomalies, but also provides livestock managers with comprehensive early warning information and control commands including images and parameters, assisting in efficient decision-making and ensuring safe livestock and poultry production.

[0038] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," "link," and "fix" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0039] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various equivalent changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A monitoring and early warning method for intelligent livestock and poultry house inspection robots, characterized in that, Includes the following steps: S1, through the multispectral imaging module and infrared thermal imaging module carried by the inspection robot, synchronously acquires images of livestock and poultry groups in different areas of the poultry house. The multispectral imaging module acquires image data of multiple bands at preset band intervals, and the infrared thermal imaging module acquires image data of the temperature distribution of the livestock and poultry body surface at a preset frame rate. S2 transmits the collected multispectral image data and infrared thermal imaging image data to the data processing module. The multi-source image data is spatially aligned using an image registration algorithm, so that the pixel positions of the same livestock individual correspond one-to-one in different types of images. S3. Target segmentation is performed on the registered image data. Separate the individual livestock and poultry region, the indoor environment region, and the interference region through the semantic segmentation algorithm. Then, each individual livestock and poultry region is individually labeled through the instance segmentation algorithm. S4, extract the feature parameters of each marked livestock and poultry individual area, including texture features and color features based on multispectral images and temperature features based on infrared thermal imaging images, and extract the temperature and humidity features and light intensity features of the indoor environment area. S5. Input the extracted feature parameters into the early warning analysis module. Through preset feature threshold judgment and feature trend analysis, identify the abnormal state of individual livestock and poultry and the abnormal conditions of the indoor environment. S6. When an abnormal state or situation is identified, the early warning analysis module generates a corresponding early warning signal and transmits the early warning signal, along with the image data and feature parameter data of the abnormal area, to the remote monitoring terminal. At the same time, it controls the inspection robot to locate the abnormal area and perform secondary image acquisition and feature parameter extraction.

2. The monitoring and early warning method for intelligent livestock and poultry house inspection robots according to claim 1, characterized in that, The image registration algorithm in S2 adopts a feature point-based registration method, and the registration error calculation model is as follows: ,in, For registration error, The number of feature points, For the first in the multispectral image The coordinates of the nth feature point are the nth feature point in the infrared thermal imaging image. The coordinates of the feature points For the image rotation angle, for Directional translation amount for Directional translation amount.

3. The monitoring and early warning method for intelligent livestock and poultry house inspection robots according to claim 1, characterized in that, In S4, texture feature extraction uses the gray-level co-occurrence matrix method, and the texture contrast calculation model is as follows: Where Con represents texture contrast. For image gray levels, Coordinates in the gray-level co-occurrence matrix The element value at that position, The step size of the gray-level co-occurrence matrix. The direction angle of the gray-level co-occurrence matrix.

4. The monitoring and early warning method for intelligent livestock and poultry house inspection robots according to claim 1, characterized in that, The characteristic trend analysis in S5 uses linear regression analysis, and the trend slope calculation model is as follows: , The trend slope The number of times the feature parameters were collected. For collection sequence number, For the first The feature parameter values ​​collected this time.

5. The monitoring and early warning method for the intelligent livestock and poultry house inspection robot according to claim 1, characterized in that, The abnormal state judgment in S5 adopts a multi-feature fusion judgment method, and the calculation model of the fusion judgment value is as follows: , To merge the judgment values, The number of feature parameters participating in the fusion. For the first The weight coefficients of each feature parameter, For the first The actual values ​​of each feature parameter For the first The minimum value of each feature parameter. For the first The maximum value of each feature parameter.

6. The monitoring and early warning method for the intelligent livestock and poultry house inspection robot according to claim 1, characterized in that, The frame rate adjustment in the secondary image acquisition in S6 adopts an adaptive adjustment method, and the adjusted frame rate calculation model is as follows: , The adjusted capture frame rate. This is the initial acquisition frame rate. This is the frame rate adjustment factor. This represents the current level of abnormality. The range of abnormality levels. This represents the maximum degree of abnormality.

7. The monitoring and early warning method for the intelligent livestock and poultry house inspection robot according to claim 1, characterized in that, S3 includes the following steps: S31, the registered multi-source image data is input into the pre-trained semantic segmentation network, the convolutional layer in the network extracts features from the image data to generate multi-scale feature maps, and then the pooling layer downsamples the feature maps to retain the labeled feature information; S32 performs an upsampling operation on the downsampled feature map to restore the spatial resolution of the image, and at the same time fuses feature maps of different scales to enhance the segmentation ability of targets of different sizes. S33, the semantic segmentation result of the image is generated through the output layer of the semantic segmentation network, and the image pixels are divided into livestock and poultry individual categories, indoor environment categories and interference categories to obtain a preliminary segmented image; S34. Perform instance segmentation on the livestock and poultry individual category regions in the preliminary segmentation image, determine the bounding box of each livestock and poultry individual through the object detection algorithm, and then, combined with the semantic segmentation results, label each livestock and poultry individual region separately.

8. The monitoring and early warning method for the intelligent livestock and poultry house inspection robot according to claim 1, characterized in that, S4 includes the following sub-steps: S41, for each marked livestock and poultry individual region, extract texture features from the multispectral image, traverse the region through a sliding window, calculate the gray-level co-occurrence matrix in each window, and then extract contrast, correlation, energy and homogeneity texture feature parameters from the gray-level co-occurrence matrix. S42, extract color features from the individual livestock and poultry regions of the multispectral image, convert the image from RGB color space to HSV color space, and calculate the mean, variance, maximum and minimum values ​​of the H channel, S channel and V channel as color feature parameters respectively. S43, extract temperature features from the individual livestock and poultry regions of the infrared thermal imaging image, traverse the temperature values ​​of all pixels in the region, and calculate the average temperature, maximum temperature, minimum temperature and temperature standard deviation in the region as temperature feature parameters. S44. For the indoor environment area, temperature and humidity features and light intensity features are extracted through image analysis algorithms. Combined with the preset reference area in the image, the average temperature and humidity and average light intensity of the environmental area are calculated as environmental feature parameters.

9. The monitoring and early warning method for the intelligent livestock and poultry house inspection robot according to claim 1, characterized in that, S5 includes the following steps: S51, compare the individual livestock and poultry feature parameters extracted in S4 with the preset normal feature parameter threshold range, and determine whether each feature parameter exceeds the threshold range. If it exceeds the threshold range, mark it as the abnormal type corresponding to that feature parameter. S52, perform trend analysis on multiple sets of livestock and poultry individual characteristic parameters collected continuously, and determine whether the characteristic parameters show abnormal change trends by calculating the rate of change and the trend of change of the characteristic parameters. If so, mark it as a trend abnormality type. S53, compare the environmental characteristic parameters inside the building with the preset normal environmental threshold range, and determine whether the environmental characteristic parameters exceed the threshold. If they do, mark them as an abnormal environmental type. S54 integrates the abnormal markers of individual livestock and poultry characteristics, abnormal markers of trends, and abnormal markers of the indoor environment. Through logical judgment rules, it determines whether there are abnormal conditions of individual livestock and poultry or abnormal conditions of the indoor environment, and forms a preliminary abnormality identification result.

10. A monitoring and early warning system for intelligent livestock and poultry house inspection robots, characterized in that: The system is applied to the intelligent livestock and poultry house inspection robot monitoring and early warning method described in claim 1, comprising: The multi-source image acquisition and synchronous transmission unit is mounted on the inspection robot body and consists of a multispectral imaging component, an infrared thermal imaging component, and a data transmission subunit. The multispectral imaging component acquires multi-band image data of the poultry house according to a preset band interval and resolution. The infrared thermal imaging component acquires image data of the temperature distribution of the livestock and poultry body surface according to a preset frame rate. The data transmission subunit synchronously transmits the image data to the data processing unit. The multi-source image registration and target segmentation unit is connected to the multi-source image acquisition and synchronous transmission unit. It has built-in image registration algorithm module and target segmentation algorithm module, which respectively perform spatial alignment processing on multi-source image data, semantic segmentation and instance segmentation on registered image data, and separate and mark individual livestock and poultry, indoor environment and interference areas. The multi-dimensional feature extraction and quantization unit is connected to the multi-source image registration and target segmentation unit. It includes a livestock and poultry individual feature extraction submodule and a barn environment feature extraction submodule. It extracts texture, color and temperature feature parameters from the marked livestock and poultry individual area, and extracts temperature, humidity and light intensity feature parameters from the barn environment area, and quantizes all feature parameters. The abnormal state identification and trend analysis unit is connected to the multi-dimensional feature extraction and quantification unit. It has a built-in feature threshold judgment module, feature trend calculation module and abnormal comprehensive judgment module. It compares the feature parameters with the preset threshold, analyzes the trend of feature parameter changes, and combines the results of the two to identify abnormal states and situations. The multi-level early warning signal generation and data integration unit is connected to the abnormal state identification and trend analysis unit. It generates multi-level early warning signals according to the type and degree of abnormality, and integrates abnormal area image data, feature parameter data and early warning signals to form an early warning information package. The remote terminal interaction and robot control unit are connected to the multi-level early warning signal generation and data integration unit and the inspection robot body, respectively. It receives early warning information packets and transmits them to the remote monitoring terminal. At the same time, it generates control commands based on the early warning information, controls the inspection robot to locate the abnormal area, and performs secondary image acquisition and feature parameter extraction operations.

Images