A Reliability-Driven Method and System for Detecting Empty Pig Pens
By constructing a pixel-level depth confidence and lightweight neural network based on a confidence-driven method based on depth images, the stability problem of detecting empty pig pens under all-weather lighting conditions is solved, and accurate detection is achieved at night or in the absence of light, while reducing computational complexity.
Patent Information
- Application Number
- CN202611225164.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-13
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2046-08-13
AI Technical Summary
Existing methods for detecting empty pig pens cannot operate stably under all-weather lighting conditions, especially at night or in the absence of light, and cannot effectively identify whether there are pigs in the pen. Furthermore, existing deep learning models have high computational complexity, making it difficult to achieve low-latency real-time response on edge devices in farms.
A depth image-based credibility-driven method is adopted to generate a credibility region by constructing pixel-level depth credibility, extract the effective contour information of the target column, and use a lightweight neural network for occupancy status classification. Combined with temporal consistency analysis and stability parameters, empty column detection is achieved.
It operates stably under all-weather lighting conditions, and can accurately detect the status of pig pens, especially at night or in the absence of light. It reduces computational complexity, adapts to changes in ambient light, and improves the stability and accuracy of detection.
Smart Images

Figure CN122737640B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer vision, and in particular to a credibility-driven method and system for detecting empty pig pens. Background Technology
[0002] In the refined management of large-scale pig farms, timely and accurate detection of the occupancy status of each pen in the gestation and sow rearing pens is a fundamental step in achieving automated decision-making for pig herd turnover, precision feeding, and health monitoring. Existing technologies employ automated computer vision detection schemes based on ordinary RGB cameras. These schemes typically extract features and detect targets from the acquired visible light images to determine the presence of pigs in the pens. However, to ensure sows rest, lighting in the gestation and sow rearing pens is usually turned off at night, making it impossible for RGB cameras to acquire identifiable images under no-light or low-light conditions. Even during the day, the lighting in the pens is often unevenly distributed, with numerous shadows and alternating areas of light and dark, leading to a sharp decline in accuracy. Furthermore, some schemes use complex deep learning models for detection. While this improves accuracy to some extent, the large number of model parameters and high computational complexity make it difficult to achieve low-latency real-time response on edge devices with limited computing resources commonly found in farms. Moreover, their over-reliance on visible light images makes them unable to fundamentally overcome sensitivity to ambient light conditions. Therefore, a method is urgently needed to solve the technical problem of existing methods for detecting empty pig pens not operating stably under all-weather lighting conditions. Summary of the Invention
[0003] This application provides a credibility-driven method and system for detecting empty pig pens, which solves the technical problem that existing methods for detecting empty pig pens cannot operate stably under all-weather lighting conditions.
[0004] To achieve the above objectives, this application adopts the following technical solution: Firstly, a confidence-driven method for detecting empty pig pens is provided. The method includes: acquiring a depth image of the pig pen to be detected; constructing pixel-level depth confidence based on the depth image, evaluating the confidence of each pixel in the depth image, and generating a confidence region based on the confidence of each pixel; extracting effective contour information of the target pen based on the confidence region, and adaptively correcting a preset target detection region based on the effective contour information to obtain a target detection region corresponding to the target pen; encoding spatial structure features of the depth information within the target detection region to generate feature description information characterizing the three-dimensional spatial structure of the target pen, and inputting the feature description information into a lightweight neural network for occupancy status classification to obtain the classification result and classification confidence of the current frame; performing temporal consistency analysis based on the classification results and classification confidence of multiple consecutive frames to determine the current empty pen detection state of the target pen, and calculating a stability parameter to characterize the detection stability; and performing empty pen detection according to preset rules based on the stability parameter to obtain the detection result.
[0005] In conjunction with the first aspect mentioned above, in one possible implementation, pixel-level depth confidence is constructed based on the depth image. Confidence is evaluated for each pixel in the depth image, and a reliable region is generated based on the confidence of each pixel. This includes: calculating the depth gradient continuity, instantaneous temporal stability, depth missing compensation degree, and historical temporal stability of each pixel based on the depth information corresponding to each pixel in the depth image; constructing the depth confidence corresponding to each pixel based on the depth gradient continuity, instantaneous temporal stability, depth missing compensation degree, and historical temporal stability; comparing the depth confidence with a preset confidence threshold to determine target pixels that meet the confidence conditions, and generating a reliable region based on the target pixels; and completing the depth holes in the reliable region to obtain a reliable depth map and reliable region for target field detection.
[0006] In conjunction with the first aspect mentioned above, in one possible implementation, based on the depth information corresponding to each pixel in the depth image, the depth gradient continuity, instantaneous temporal stability, depth missing compensation degree, and historical temporal stability of each pixel are calculated, including: calculating the dispersion of depth values within a preset neighborhood based on the preset neighborhood depth values of each pixel, and determining the depth gradient continuity of the corresponding pixel based on the dispersion degree; calculating the depth change of the corresponding pixel based on the depth values of the corresponding pixels in the current frame and the previous frame, and determining the instantaneous temporal stability of the corresponding pixel based on the depth change amount; determining whether each pixel has depth missing; when a pixel does not have depth missing, the pixel is determined as a valid depth pixel; when a pixel has depth missing, depth compensation is performed on depth missing pixels that meet the compensation conditions, and the depth missing compensation degree of the corresponding pixel is determined based on the compensation result; if not, the depth missing compensation degree is determined to a preset minimum value; establishing a historical sliding window corresponding to each pixel, performing credibility filtering on the corresponding pixels of each frame within the historical sliding window, recording the binary state of whether the corresponding pixels of each frame pass the credibility filtering, and determining the historical temporal stability of the corresponding pixel based on the statistical results of the binary state.
[0007] In conjunction with the first aspect mentioned above, in one possible implementation, the effective contour information of the target field is extracted based on the reliable region, and the preset target detection region is adaptively corrected based on the effective contour information to obtain the target detection region corresponding to the target field. This includes: obtaining the initial target detection region corresponding to the pre-calibrated target field; extracting the effective contour of the target field based on the reliable region, and calculating the region center position, region width, region height, and region tilt angle of the target field based on the effective contour; calculating the position offset, width offset, height offset, and angle offset corresponding to the initial target detection region based on the region center position, region width, region height, and region tilt angle, respectively; synchronously correcting the center position, width, height, and rotation angle of the initial target detection region based on the position offset, width offset, height offset, and angle offset to obtain the target detection region; and restricting the position offset, width offset, height offset, and angle offset based on a preset step size constraint.
[0008] In conjunction with the first aspect mentioned above, in one possible implementation, spatial structure feature encoding is performed on the depth information within the target detection region to generate feature description information representing the three-dimensional spatial structure of the target field. This feature description information is then input into a lightweight neural network for occupancy state classification to obtain the classification result and classification confidence of the current frame. This includes: constructing a depth gradient field, a spatial normal vector field, and a three-dimensional contour edge map based on the depth information within the target detection region; using a preset neighborhood as a statistical window, calculating the local depth mean and local depth variance in the depth gradient field, the normal vector angle in the spatial normal vector field, and the contour maxima and minima in the three-dimensional contour edge map; fusing the local depth mean, local depth variance, normal vector angle, contour maxima, contour minima, and depth distance information within the target detection region to obtain feature description information representing the three-dimensional spatial structure of the target field; mapping the feature description information to feature dimensions to obtain structured features; and inputting the structured features into a lightweight neural network for occupancy state classification to obtain the classification result and classification confidence of the current frame.
[0009] In conjunction with the first aspect mentioned above, one possible implementation involves inputting structured features into a lightweight neural network for occupancy state classification to obtain the classification result and classification confidence of the current frame. This includes: performing multi-layer feature extraction on the structured features based on a feature extraction network to obtain deep features corresponding to the target field; performing global feature aggregation on the deep features to obtain global features corresponding to the target field; performing binary classification based on the global features to output the probability of an empty field and the probability of a field with a pig; and determining the classification result and classification confidence of the current frame based on the probability of an empty field and the probability of a field with a pig.
[0010] In conjunction with the first aspect mentioned above, in one possible implementation, temporal consistency analysis is performed based on the classification results and classification confidence of multiple consecutive frames to determine the current empty pen detection state of the target pen, and stability parameters used to characterize detection stability are calculated. This includes: obtaining the classification results and classification confidence of multiple consecutive frames; performing temporal smoothing based on the classification confidence of the current frame and the temporal classification confidence of the previous moment to obtain the temporal classification confidence of each corresponding frame; performing state transitions for the target pen between the pig-present state, the pending confirmation state, and the empty pen state based on the temporal classification confidence and preset state transition rules, and determining the current empty pen detection state of the target pen based on the number of consecutive frames that meet the state transition conditions; calculating the temporal stability parameters based on the temporal classification confidence of multiple consecutive frames, and obtaining the stability parameters by combining the proportion of the reliable region of the current target pen and the positional change of the target detection region in consecutive frames.
[0011] In conjunction with the first aspect mentioned above, in one possible implementation, a stability parameter is obtained by combining the proportion of the current target field's credible region and the positional change of the target detection region in consecutive frames. This includes: determining the temporal fluctuation value corresponding to the target field based on the temporal classification confidence of multiple consecutive frames; determining the proportion of the credible region based on the number of credible pixels within the credible region and the total number of pixels within the target detection region; determining the positional change of the target detection region based on the center position of the target detection region in two consecutive frames; and fusing the temporal fluctuation value, the proportion of the credible region, and the positional change of the target detection region to obtain the stability parameter.
[0012] In conjunction with the first aspect mentioned above, in one possible implementation, empty column detection is performed according to a preset rule based on the stability parameter to obtain the detection result. This includes: comparing the stability parameter with a preset stability threshold to determine whether the detection state corresponding to the current target column is stable; when the detection state is stable, outputting the empty column detection result corresponding to the current target column; when the detection state is unstable, determining whether the instability is caused by abnormal depth data quality based on the proportion of the reliable region; when the instability is caused by abnormal depth data quality, adjusting the reliability threshold used to generate pixel-level depth reliability and the adaptive correction parameter used to correct the target detection region, and re-establishing the detection result based on the adjusted reliability threshold. A reliable region is generated. Based on the regenerated reliable region, the effective contour information of the target field is extracted again, and the target detection region is corrected. Based on the corrected target detection region, spatial structure feature encoding, occupancy status classification, and time sequence consistency analysis are performed again to obtain the stability parameter. Based on the obtained stability parameter, the detection status corresponding to the current target field is determined again. When the obtained detection status is stable or the preset number of feedbacks is reached, the empty field detection result corresponding to the current target field is output. When the detection status is unstable and not caused by depth data quality anomalies, the current reliability threshold and adaptive correction parameter are kept unchanged, and the empty field detection result corresponding to the current target field is output.
[0013] Secondly, a confidence-driven vacant pen detection system for pigs is provided to implement any of the methods in the first aspect. The system includes: a depth image acquisition module for acquiring a depth image of the pig pen to be detected; a confidence region generation module for constructing pixel-level depth confidence based on the depth image, evaluating the confidence of each pixel in the depth image, and generating a confidence region based on the confidence of each pixel; a target detection region correction module for extracting effective contour information of the target pen based on the confidence region, and adaptively correcting the preset target detection region based on the effective contour information to obtain the target detection region corresponding to the target pen; an occupancy status classification module for encoding spatial structure features of the depth information within the target detection region, generating feature description information to characterize the three-dimensional spatial structure of the target pen, and inputting the feature description information into a lightweight neural network for occupancy status classification to obtain the classification result and classification confidence of the current frame; a temporal consistency analysis module for performing temporal consistency analysis based on the classification results and classification confidence of multiple consecutive frames to determine the vacant pen detection status of the current target pen, and calculating a stability parameter to characterize the detection stability; and a vacant pen detection module for detecting vacant pens according to preset rules based on the stability parameter to obtain the detection result.
[0014] This application provides a reliability-driven method and system for detecting empty pig pens. Using depth images as the sole data source, it acquires real-time distance information of the scene using active infrared imaging. Its acquisition quality is unaffected by changes in ambient light, enabling stable operation 24 / 7 in complex pig farm conditions including complete darkness, low light, strong light, and uneven lighting. This solves the problem that RGB image-based solutions cannot work at night or in complete darkness. By reconstructing the historical reliability calculation rules, it replaces the traditional historical floating-point score storage and backtracking recalculation mechanism with a statistical method based on pixel-level binary filtering pass rates within a fixed-length sliding window. This transforms the relationship between historical statistics and real-time thresholds from a recursive dependency to a weakly decoupled relationship, maintaining temporal noise reduction capabilities while eliminating statistical distortion and computational overload caused by bidirectional iterative derivation of thresholds and historical scores. A five-DOF adaptive ROI iterative correction mechanism is adopted, which simultaneously corrects the center position, width, height, and rotation angle of the detection area using the effective contour extracted by the reliable mask. This can compensate for multi-dimensional spatial deviations such as camera tilt, installation offset, and field deformation. Combined with the upper limit constraint of the correction step size in each dimension, the detection area can reliably lock the target field without divergence jitter during long-term operation. A unified spatial structure descriptor encoding scheme based on depth gradient field, spatial normal vector field, and 3D contour edge map is constructed. Local statistical features and original distance information are fused into feature vectors with 3D geometric representation capabilities, and a lightweight mapping layer is added to complete dimensional alignment. Compared with the traditional method of simply stitching the depth-derived map into the network, this method has a substantial improvement in both the breadth and depth of utilization of 3D correction spatial information. Furthermore, this application adds a credibility threshold upper limit truncation constraint, differentiated parameter reset rules for normal exit and forced termination exit scenarios, and a lifecycle alignment constraint for the ROI correction step size to the closed-loop feedback mechanism. This ensures that regardless of whether the feedback converges successfully, the system's pre-algorithm parameters can return to a reasonable range after each detection process, and the risks of parameter drift and rule mutual exclusion are controlled in long-term operation. Attached Figure Description
[0015] Figure 1 A system architecture diagram of a credibility-driven empty pig pen detection system provided for embodiments of this application; Figure 2 A flowchart illustrating a credibility-driven method for detecting empty pig pens, as provided in this application embodiment. Figure 1 ; Figure 3 A flowchart illustrating a credibility-driven method for detecting empty pig pens, as provided in this application embodiment. Figure 2 ; Figure 4 A flowchart illustrating a credibility-driven method for detecting empty pig pens, as provided in this application embodiment. Figure 3 ; Figure 5 A flowchart illustrating a credibility-driven method for detecting empty pig pens, as provided in this application embodiment. Figure 4 ; Figure 6 A flowchart illustrating a credibility-driven method for detecting empty pig pens, as provided in this application embodiment. Figure 5 ; Figure 7 A flowchart illustrating a credibility-driven method for detecting empty pig pens, as provided in this application embodiment. Figure 6 . Detailed Implementation
[0016] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.
[0017] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.
[0018] The credibility-driven method for detecting empty pig pens provided in this application can be applied to, for example... Figure 1 The credibility-driven empty pen detection system for pigs shown is as follows: Figure 1As shown, the system includes: a depth image acquisition module 101, used to acquire a depth image of the pig pen to be detected; a reliable region generation module 102, used to construct pixel-level depth reliability based on the depth image, evaluate the reliability of each pixel in the depth image, and generate a reliable region based on the reliability of each pixel; a target detection region correction module 103, used to extract the effective contour information of the target pen based on the reliable region, and adaptively correct the preset target detection region based on the effective contour information to obtain the target detection region corresponding to the target pen; an occupancy status classification module 104, used to encode the spatial structure features of the depth information in the target detection region, generate feature description information to characterize the three-dimensional spatial structure of the target pen, and input the feature description information into a lightweight neural network for occupancy status classification to obtain the classification result and classification confidence of the current frame; a temporal consistency analysis module 104, used to perform temporal consistency analysis based on the classification results and classification confidence of multiple consecutive frames, determine the empty pen detection state of the current target pen, and calculate the stability parameter used to characterize the detection stability; and an empty pen detection module 106, used to perform empty pen detection according to preset rules based on the stability parameter to obtain the detection result.
[0019] To address the technical problem that existing methods for detecting empty pig pens cannot operate stably under all-weather lighting conditions, this application provides a reliability-driven method for detecting empty pig pens. Figure 2 A flowchart illustrating the credibility-driven method for detecting empty pig pens provided in this application embodiment. Figure 1 ,like Figure 2 As shown, it includes: S201. Obtain the depth image of the pigpen to be detected.
[0020] Depth images refer to image data acquired by depth sensors, whose pixel values encode the physical distance from the sensor to the scene surface.
[0021] In one possible implementation, depth images can be acquired by continuously capturing data at a preset frame rate using a depth camera positioned above the pigpen. The type of depth camera can include, but is not limited to, sensors based on structured light, time-of-flight, or active stereo vision principles. Their common characteristic is that they calculate the three-dimensional distance information of the scene by actively emitting infrared light and receiving reflected signals.
[0022] It should be noted that the quality of depth image acquisition is not affected by changes in ambient lighting conditions. Even in complete darkness at night, active infrared emission can still ensure that the sensor can normally acquire the spatial structure and depth contour within the field.
[0023] S202. Construct pixel-level depth credibility based on depth image, evaluate the credibility of each pixel in the depth image, and generate a credibility region based on the credibility of each pixel.
[0024] Pixel-level depth confidence score refers to a quantitative score obtained by evaluating the quality of the depth information carried by each pixel in the depth image. It is used to measure the probability that the spatial distance reflected by that pixel is true and reliable. The reliable region is a set of connected regions in the depth image consisting of all pixels whose confidence scores meet preset conditions.
[0025] In one possible implementation, the system comprehensively utilizes multiple evaluation dimensions, including the depth consistency of a pixel within its spatial neighborhood, the depth stability between frames acquired at different times, the presence of depth loss due to occlusion or reflection, and the pixel's historical performance of consistently high reliability over a period of time. A weighted fusion method is used to calculate the overall reliability of each pixel. A dynamically adjustable baseline reliability threshold is set, and pixels with an overall reliability higher than the threshold are marked as reliable pixels. This generates a binary reliable mask, and the area covered by this mask is the reliable region.
[0026] S203. Extract the effective contour information of the target field based on the reliable region, and adaptively correct the preset target detection region based on the effective contour information to obtain the target detection region corresponding to the target field.
[0027] The preset target detection area refers to a reference space range that the system pre-sets for each pigpen during the initialization phase through a calibration process. It is typically a rectangular or polygonal frame with specific dimensions and positional attributes. Effective contour information refers to the outer shape information of the connected regions extracted from the reliable mask that describes the actual physical boundaries of the pen or the area occupied by the pigs.
[0028] In one possible implementation, the system takes a reliable mask as input and extracts the maximum connected contour corresponding to the target area using a contour detection algorithm. The geometric properties of this effective contour are further calculated, such as the centroid position, the width and height of the bounding rectangle, and the rotation angle of the principal axis. Based on the deviation between the calculated geometric properties and the corresponding parameters of the initially preset target detection region, the position, size, and even orientation of the target detection region are dynamically corrected using a preset adjustment strategy, ensuring that the corrected detection region fits the actual spatial range of the pen or pig in the current frame.
[0029] It should be noted that the adaptive correction mechanism in this step can compensate for the spatial misalignment between the detection area and the actual effective area caused by long-term slight changes in the camera installation posture, physical compression deformation of the pen, or pig standing position shift, ensuring that the objects of subsequent feature extraction always remain spatially aligned with the current state of the target pen.
[0030] S204. Encode the spatial structure features of the depth information within the target detection area to generate feature description information that characterizes the three-dimensional spatial structure of the target field. Input the feature description information into a lightweight neural network for occupancy state classification to obtain the classification result and classification confidence of the current frame.
[0031] Spatial structure feature encoding refers to the process of transforming a 3D point cloud or depth map within a detection area into a compact and information-rich feature vector by calculating local geometric attributes and performing statistical processing. Lightweight neural networks refer to deep neural networks that have been pruned and optimized to adapt their parameter and computational costs to the real-time inference conditions of edge devices.
[0032] In one possible implementation, the system can construct multiple spatial geometric mappings in parallel for the depth data within the target detection region. These include gradient fields reflecting the drastic local changes in depth values, normal vector fields reflecting local surface orientation, and contour maps reflecting abrupt depth changes. Based on this, multi-dimensional high-order statistical features such as mean, variance, and extreme values are extracted within each local statistical window. These statistical features are then fused with the original depth and distance features to form a structured descriptor that characterizes the three-dimensional morphology of the enclosure. This descriptor is fed into a lightweight backbone network for feature extraction and binary classification inference. The network outputs the probability that the current frame belongs to either "with pigs" or "empty enclosure," and outputs the higher-probability category and its corresponding confidence level as the classification result for the current frame.
[0033] It should be noted that the above encoding process transforms the original three-dimensional disordered point cloud or two-dimensional depth map into a standardized feature description with spatial geometric representation capabilities, enabling the lightweight network to effectively capture the three-dimensional structural differences between the pig's body size, posture, and the flat bottom surface of the empty pen.
[0034] S205. Perform temporal consistency analysis based on the classification results and classification confidence of multiple consecutive frames to determine the empty column detection status of the current target column, and calculate the stability parameter used to characterize the detection stability.
[0035] Temporal consistency analysis refers to a method that comprehensively utilizes the classification results and confidence fluctuations of multiple consecutive frames in the time dimension to filter out random and instantaneous single-frame false detections, so as to output a smoother and more reliable state determination.
[0036] In one possible implementation, the system incorporates a time-series-based state decision maker. This decision maker fuses single-frame classification confidence and historical smoothing confidence, and defines a series of transition conditions between states. The decision maker can set multiple states, including at least a state indicating the presence of pigs in the pen and an empty pen state indicating no pigs in the pen, and can set intermediate transition states to suppress instantaneous jumps. State transitions require a certain number of consecutive frames to be triggered. Preferably, the system extracts the time-series classification confidence sequence within the most recent time window, calculates the fluctuation value reflecting its dispersion, and simultaneously collects the area ratio of the current frame's reliable region within the detection range and the spatial offset of the detection region between adjacent frames. These multiple indicators are then fused to generate a comprehensive stability parameter.
[0037] Based on the above steps, the single-frame inference results are smoothed and verified in the time dimension, reducing detection jitter and false alarms caused by instantaneous noise from the sensor or short-term rapid movement of the target.
[0038] S206. Perform empty shoal detection according to preset rules based on stability parameters, and obtain the detection results.
[0039] Among them, the preset rule refers to a closed-loop adaptive adjustment logic. When the stability parameter indicates that there are abnormal fluctuations in the detection process, it does not directly output uncertain results, but attempts to repair the detection quality by adjusting the parameters of the preceding links and re-inferring.
[0040] In one possible implementation, the system stability parameter is compared with a preset stability threshold. If the stability parameter indicates that the current detection process is stable, the empty pen detection state determined in S205 is directly output as the final detection result. If the stability parameter indicates that the current detection process is unstable, the cause of the instability is further analyzed. When the instability is determined to be due to the degradation of deep data quality, the system triggers a closed-loop feedback mechanism, adjusts the confidence threshold parameter in S202 and the correction strength parameter of the target detection area in S203, and re-executes the re-inference process from S202 to S205 based on the adjusted parameters. The re-inference process can iterate several times until the detection state recovers to stability or the preset maximum number of feedbacks is reached. When the instability is determined not to be caused by data quality, but by a real change in the state of pigs within the pen, the system maintains the current parameters unchanged and directly outputs the current empty pen detection state as the final detection result.
[0041] It should be noted that this closed-loop feedback mechanism forms a full-link adaptive optimization closed loop from the sensing end to the decision-making end and back to the sensing end, which can dynamically suppress interference caused by environmental noise and equipment drift during operation.
[0042] This application embodiment solves the technical problem that traditional visible light image-based solutions cannot work properly in pig farms at night or in low light conditions by constructing an automated empty pen detection pipeline with full time domain, low latency, and high robustness from physical signal input to business status output.
[0043] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 3 As shown, the above S202 can be specifically implemented through the following S301 to S304, which are explained in detail below: S301. Based on the depth information corresponding to each pixel in the depth image, calculate the depth gradient continuity, instantaneous time stability, depth loss compensation degree, and historical time series stability of each pixel.
[0044] Depth gradient continuity refers to a metric characterizing the smoothness or continuity of depth values between the current pixel and its spatial neighborhood. A lower value indicates a higher probability that the pixel is at a depth abrupt change edge, reflective noise, or hole boundary. Depth loss compensation level refers to a graded quantization score assigned based on whether the pixel is a valid measurement value and whether it can be repaired using neighborhood information.
[0045] In one possible implementation, the system performs four-dimensional evaluation calculations for each pixel location in the depth image. For depth gradient continuity, the system acquires the depth values of all pixels within a preset spatial neighborhood centered on the current pixel, calculates a measure of the dispersion of depth values within that neighborhood, and uses the reciprocal or normalized result of this measure as the depth gradient continuity component of the corresponding pixel. Commonly used measures of dispersion include variance, standard deviation, or range. For instantaneous temporal stability, the system acquires the depth values at the same pixel coordinates in the current frame and the previous frame, calculates the absolute value of the difference between the two, and uses the normalized difference as its reciprocal or non-negative mapping as the instantaneous temporal stability component of the corresponding pixel. A larger difference indicates a more drastic change in the depth signal within a short time, resulting in poorer instantaneous stability. Regarding the depth loss compensation level, the system assesses each pixel in the depth image: if the pixel has a valid depth measurement value, it is directly assigned the highest score; if the pixel is missing data due to a depth hole, the system further determines whether the spatial location of the missing pixel meets the preset compensation conditions. For missing pixels that meet the compensation conditions, after interpolation compensation using the valid depth values of its neighborhood, an intermediate score is assigned based on the compensation quality; for missing pixels that do not meet the compensation conditions, a preset lowest score is directly assigned. Regarding historical temporal stability, the system maintains a historical sliding window of fixed frame length for each pixel location. It performs threshold filtering on the instantaneous depth confidence value of the corresponding pixel in each historical frame within the window, recording only binary states that pass or fail the filtering, and calculating the proportion of frames that pass the filtering to the total number of frames within the window as the historical temporal stability component of the corresponding pixel.
[0046] Preferably, the instantaneous time stability satisfies the following formula:
[0047] in, Indicates coordinates as The instantaneous time stability score of the pixel. Indicates the current frame's coordinates Depth value at that location, Indicates that the previous frame is at the same coordinates The depth value at that location, where k is a positive normalization coefficient used to map the depth difference to a reasonable score range. When the inter-frame depth difference is zero, Reaching the maximum value indicates that the pixel is completely stable in time; when the inter-frame depth difference approaches infinity, A value approaching zero indicates that the pixel has experienced a drastic, instantaneous change.
[0048] Preferably, the historical time series stability satisfies the following formula:
[0049] in, Indicates coordinates as The historical temporal stability score of the pixel at frame t, where N is the fixed frame length of the historical sliding window. It is a binary function. When the instantaneous depth confidence value of the pixel in the i-th frame is greater than the real-time confidence threshold used in the current frame, it takes the value of 1, otherwise it takes the value of 0. t is the time index.
[0050] Based on the above steps, the depth data quality of each pixel is quantitatively evaluated from four dimensions: spatial gradient continuity, inter-frame temporal consistency, data missing degree, and historical reliability pass rate. This lays a multi-dimensional information foundation for subsequent comprehensive reliability synthesis. The calculation of historical temporal stability avoids the problems of recursive dependency and computational power expansion while retaining the temporal noise reduction capability through a weak decoupling design.
[0051] S302. Construct the depth confidence level for each pixel based on depth gradient continuity, instantaneous time stability, depth loss compensation degree, and historical time series stability.
[0052] In one possible implementation, the system linearly weights and fuses the scores calculated from the four dimensions using a preset set of weighting coefficients to generate a comprehensive depth reliability score. All weighting coefficients are non-negative and sum to 1. The weighting of each dimension can be configured according to actual working conditions; for example, a higher weight can be assigned to the depth loss compensation level to penalize data loss areas, or a higher weight can be assigned to instantaneous temporal stability to suppress dynamic noise. The generated reliability score can be constrained within a preset numerical range, such as between 0 and 1; a higher score indicates more realistic and reliable depth data for that pixel.
[0053] S303. Compare the depth confidence level with the preset confidence level threshold to determine the target pixel that meets the confidence condition, and generate a confidence region based on the target pixel.
[0054] The preset confidence threshold is a benchmark threshold used to distinguish between trusted and untrusted pixels. This threshold can be set with a default initial value during the initialization of the detection process and dynamically adjusted according to the detection stability in the subsequent closed-loop feedback process.
[0055] In one possible implementation, the system iterates through all pixels in the depth image, comparing the depth confidence score of each pixel with the currently effective confidence threshold. For pixels with a confidence score greater than or equal to the confidence threshold, the system identifies them as target pixels and marks their corresponding positions as 1 in a binary mask image of the same size as the original depth image; for pixels with confidence scores lower than the confidence threshold, the corresponding positions in the mask are marked as 0. The set of connected regions formed by all pixels marked as 1 in this binary mask constitutes the confidence region.
[0056] Based on the above steps, the binary reliable mask generated by threshold filtering clearly defines the pixels that are reliable in spatial data and can participate in subsequent contour extraction and spatial positioning, as well as the pixels that should be removed due to low quality in depth data.
[0057] S304. Complete the depth holes in the trusted region to obtain a trusted depth map and trusted region for target field detection.
[0058] Among them, deep voids refer to isolated, small-area regions with missing depth values within or at the boundary of a reliable region due to local occlusion, infrared absorption, or other reasons.
[0059] In one possible implementation, the system analyzes the generated trustworthy mask and its corresponding depth map from the previous step, identifying hole pixels located within the trustworthy region with an area smaller than a preset hole area threshold. For these hole pixels, the system uses neighborhood mean interpolation or median interpolation to estimate and fill the depth value of the hole pixels using the effective depth values of the surrounding neighborhood that belong to the trustworthy region. After the completion process, the system outputs two sets of data: one is the completed trustworthy depth map, in which the pixel values of the original untrustworthy regions can be set to zero or retain their original values, while the small-area hole pixels have been repaired; the other is the updated binary trustworthy mask, in which the completed hole pixels can be included in the trustworthy pixel range within the updated mask.
[0060] It should be noted that the hole completion operation only processes discrete, small-area holes. For large, contiguous areas with missing depth, the system will not attempt to repair them, but will instead retain these areas as unreliable regions to avoid introducing false 3D geometric information through unreasonable interpolation repair.
[0061] This application's embodiments comprehensively examine the original depth signal from the perspectives of spatial smoothness, temporal stability, data integrity, and historical consistency. It accurately filters out typical pig farm noise such as water stain reflections, boundary flypoints, and random jumps. At the same time, through a binary statistical weak decoupling mechanism for historical temporal stability, it maintains temporal noise reduction capabilities while eliminating the hidden dangers of statistical distortion and computational overhead in traditional recursive schemes, providing a source guarantee of data quality for subsequent target detection area correction and spatial feature encoding.
[0062] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 4 As shown, the above S203 can be specifically implemented through the following S401 to S405, which are explained in detail below: S401. Obtain the initial target detection area corresponding to the pre-calibrated target field.
[0063] In one possible implementation, during the deployment phase, the system acquires visible light images containing the complete field area using an RGB-D camera mounted on a tracked robot. The operator draws a quadrilateral or polygonal frame covering the ground area of each field area on this image. The system parameterizes this frame into a set of five-dimensional reference vectors. These five-dimensional reference vectors include at least the frame's center x-coordinate, center y-coordinate, frame width, frame height, and the rotation angle of the frame's principal axis relative to the image's horizontal axis. These five-dimensional reference vectors are persistently stored in the edge computing device's configuration file. During each subsequent inference process, the system reads the initial target detection region parameters for the corresponding field area from the configuration file as the starting point for iterative correction.
[0064] S402. Extract the effective contour of the target field based on the reliable region, and calculate the center position, width, height and tilt angle of the target field based on the effective contour.
[0065] The effective contour refers to the set of boundary points of the connected region detected from the reliable mask generated by S202 that reflects the body contour of the pigs in the pen or the geometric boundary of the pen floor. The region tilt angle is the angle between the principal axis direction of the effective contour and the horizontal reference direction of the image.
[0066] In one possible implementation, the system uses a reliable mask as the input data source and runs a contour detection algorithm on this binary mask to extract the sequence of outer boundary points of all connected components. Based on selection criteria such as area, position, and overlap with the initial target detection region, the system selects the largest connected contour corresponding to the target field as the effective contour. On this basis, the system uses image moment calculation or minimum bounding rectangle fitting to solve for the geometric center coordinates of the effective contour as the region center position, the width and height of the equivalent minimum bounding rectangle as the region width and height, and the principal axis rotation angle of the minimum bounding rectangle as the region tilt angle. Unlike traditional schemes that only focus on the two-dimensional pixel range of the region, in this embodiment, the effective contour and all parameters are derived from the reliable mask after depth reliability filtering. Therefore, its ability to represent the true physical boundary of the field is significantly better than the contour extraction results directly on the original depth map or RGB image.
[0067] It should be noted that since the contour extraction operation in this step directly reuses the trusted mask generated by S202, instead of re-performing contour detection on the original depth image, the contour information and the confidence evaluation results are kept consistent throughout the detection pipeline, avoiding redundant calculations and potential data inconsistencies between upstream and downstream modules.
[0068] Based on the above steps, by directly extracting the effective contour from the reliable mask and calculating the physical parameters of the region center, width, height and tilt angle in one go, complete and accurate observation values are provided for subsequent multi-dimensional deviation calculations. This avoids the limitation of traditional schemes that rely only on the translation of the center point and cannot perceive the width deformation and posture tilt of the column.
[0069] S403. Calculate the position offset, width offset, height offset, and angle offset corresponding to the initial target detection area based on the area center position, area width, area height, and area tilt angle.
[0070] In one possible implementation, the system uses the five-dimensional reference parameters of the initial target detection area obtained in S401 as reference values, and the corresponding parameters calculated from the effective contour in S402 as observation values, calculating the deviation in each dimension one by one. The position offset is defined as the difference between the center coordinates of the observation area and the reference center coordinates, reflecting the translation requirement of the detection area in the two-dimensional plane; the width offset and height offset are defined as the differences between the width and height of the observation area and the reference width and height, respectively, reflecting the scaling requirement of the detection area; the angle offset is defined as the difference between the tilt angle of the observation area and the reference tilt angle, reflecting the rotation correction requirement of the detection area in orientation. These four offsets together constitute the multi-dimensional error signal driving subsequent ROI correction.
[0071] Based on the above steps, the spatial information of the effective contour is transformed into a multidimensional deviation relative to the initial datum, providing a clear input for the iterative solution of the subsequent correction equations.
[0072] S404. Based on the position offset, width offset, height offset, and angle offset, the center position, width, height, and rotation angle of the initial target detection area are synchronously corrected to obtain the target detection area.
[0073] In one possible implementation, the system adopts a frame-by-frame iterative update strategy, multiplying the offsets of each dimension calculated in the current frame by the corresponding correction step size coefficient, and then superimposing them on the target detection region parameters of the previous frame to achieve synchronous linkage correction of the five-dimensional parameters.
[0074] Preferably, the five-degree-of-freedom iterative correction satisfies the following set of equations:
[0075] in, , These are the x and y coordinates of the center of the target detection region after correction in the current frame. , These are the width and height of the corrected target detection region for the current frame, respectively. This is the rotation angle of the target detection region after correction in the current frame. These are the parameter values for the corresponding dimension of the previous frame; The five-dimensional baseline parameters for the initial target detection region; The x-coordinate of the region center, y-coordinate of the region center, region width, region height, and region tilt angle are calculated from the effective contour extracted from the current frame's reliable mask. These are the correction step size coefficients for each dimension, used to control the response speed of each frame to deviations in each dimension.
[0076] It should be noted that the above equations achieve joint synchronous correction with five degrees of freedom. Its core difference from traditional methods lies in the fact that the correction of the detection area is not merely a simple translation of the frame based on center position deviation, but simultaneously responds to changes in the effective contour in the dimensions of width, height, and rotation angle. When the effective area narrows due to physical deformation of the target area caused by long-term use, the width offset will drive the detection area to shrink synchronously; when the camera tilts slightly due to track movement, the angle offset will drive the detection area to rotate synchronously to maintain alignment with the actual orientation of the target area. This coupled correction mechanism ensures that the spatial consistency between the target detection area and the target area is not disrupted by independent adjustments in a single dimension.
[0077] Based on the above steps, by synchronously and iteratively superimposing the multidimensional deviation signal into the detection area parameters of the previous frame after step-size weighting, the linkage adaptive correction of the center position, size and orientation is realized, so that the target detection area can continuously and smoothly lock the current spatial range of the target field under dynamic working conditions.
[0078] S405. Limit the position offset, width offset, height offset and angle offset according to the preset step size constraint.
[0079] Among them, the preset step size constraint refers to the upper limit threshold set for the step size coefficient of each dimension, which is used to prevent the detection area from abruptly changing or diverging due to abnormal observations in a single frame.
[0080] In one possible implementation, the system sets independent upper limits for the correction step size in each of the five dimensions. When the product of the offset calculated in a frame and the corresponding step size coefficient exceeds the upper limit threshold, the system truncates the actual correction amount for that dimension to the upper limit value, ensuring that the variation of the detection area within a single frame remains controllable. The upper limit of the step size for each dimension can be calibrated based on the physical size range of the field and the typical installation deviation angle of the camera. For example, the upper limit of the step size for the width and height dimensions can be set to a preset ratio of the nominal size of the field, and the upper limit of the step size for the angle dimension can be set to a small angle value to prevent drastic rotational jumps in the detection box between consecutive frames.
[0081] Based on the above steps, by setting upper limit constraints on the step size on the correction path in each dimension, the ability of the correction mechanism to follow the deviation in real space is preserved, while effectively suppressing abnormal changes and divergences in the detection area caused by sensor noise or instantaneous occlusion, thus ensuring the spatial stability of the target detection area during long-term operation.
[0082] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 5 As shown, the above S204 can be specifically implemented through the following S501 to S505, which are explained in detail below: S501. Construct a depth gradient field, a spatial normal vector field, and a three-dimensional contour edge map based on the depth information within the target detection area.
[0083] The depth gradient field refers to the data field composed of the rate of change of the depth value of each pixel position within the target detection region in its spatial neighborhood, used to describe the degree of local undulation of the scene surface on the two-dimensional projection plane. The spatial normal vector field refers to the data field composed of the angle between the local surface normal vector corresponding to each pixel position and the camera optical axis, used to describe the degree of tilt of the local surface relative to the camera's viewing direction. The 3D contour edge map is a binary or grayscale image composed of the set of pixels at locations of abrupt depth changes in the depth map, used to annotate the geometric boundaries between different objects or surfaces in the scene.
[0084] In one possible implementation, after acquiring the corrected target detection region, the system uses the depth data within that region as a single input source and performs the construction of three basic geometric mappings in parallel. For the depth gradient field, the system uses the Sobel, Scharr, or Laplacian operators to perform convolution operations on the two-dimensional planar domain of the depth map, calculating the partial derivatives of the depth value at each pixel location in the horizontal and vertical directions. The magnitude or sum of squares of the partial derivatives is used as the gradient magnitude of that pixel, thus constructing a depth gradient field map of the same size as the target detection region. For the spatial normal vector field, the system takes a preset neighborhood window centered on each pixel location, fits a local plane to the coordinates of the 3D points within the neighborhood using the least squares method, calculates the unit normal vector of the local plane, and then calculates the angle between the normal vector and the camera optical axis direction vector, using this angle value as the value of the normal vector field. For the 3D contour edge map, the system generates a 3D contour edge map with labeled depth discontinuities by performing non-maximum suppression and double-threshold hysteresis thresholding on the depth gradient field.
[0085] Based on the above steps, by extracting three basic geometric mappings in parallel—depth gradient, spatial normal vector, and three-dimensional contour edge—the original one-dimensional distance information is upgraded to a three-dimensional spatial representation from local undulations and surface orientation to geometric boundaries. This enables subsequent statistical features to capture the essential differences in spatial morphology between the pig's body structure and the flat bottom surface of the empty pen from multiple geometric dimensions.
[0086] S502. Using a preset neighborhood as a statistical window, calculate the local depth mean and local depth variance in the depth gradient field, the angle between normal vectors in the spatial normal vector field, and the contour maxima and contour minima in the 3D contour edge map.
[0087] Among them, the local depth mean refers to the arithmetic mean of the depth values of all pixels within the statistical window, and the local depth variance refers to the dispersion of the depth values within the statistical window relative to the mean. The normal vector angle refers to the angle value directly read from the corresponding statistical window region in the spatial normal vector field. The contour maxima and contour minima refer to the local peak and valley values of the pixel values of the 3D contour edge map within the statistical window, respectively.
[0088] In one possible implementation, the system performs sliding window scanning on the depth gradient field, spatial normal vector field, and 3D contour edge map, respectively. The window size can be the same as or different from the neighborhood window used to construct the geometric mapping. For the depth gradient field, the system calculates the arithmetic mean and variance of the pixel depth within each window relative to the corresponding position in the original depth map, obtaining a local depth mean map and a local depth variance map. For the spatial normal vector field, the system extracts the angle between the normal vectors at the center of the window or after statistical aggregation within the window as the local normal vector feature. For the 3D contour edge map, the system finds the maximum and minimum pixel values within the window, which are then used as the contour maxima and minima, respectively. The extracted features are flattened or concatenated into a fixed-dimensional local feature vector.
[0089] It should be noted that by transforming the original geometric field data into statistical features through local statistical windows, we can achieve data compression and information compactness on the one hand, and enhance the robustness of the features to changes in the body shape and posture of pigs in the pen on the other hand.
[0090] Based on the above steps, by performing local window statistics on the three geometric fields, the geometric mapping information of the original grid format is transformed into a set of multidimensional statistical features with clear physical meaning and robustness to local deformation, providing refined and stable feature components for constructing a highly discriminative spatial structure descriptor.
[0091] S503. The local depth mean, local depth variance, normal vector angle, contour maxima, contour minima, and depth distance information within the target detection area are fused to obtain feature description information for characterizing the three-dimensional spatial structure of the target field.
[0092] Here, depth distance information refers to the original, untransformed physical distance values of each pixel within the target detection area. Feature description information refers to a fixed-dimensional one-dimensional feature vector composed of multiple heterogeneous feature components concatenated or combined as described above.
[0093] In one possible implementation, the system performs global pooling or pixel-by-pixel flattening on local statistical feature maps such as local depth mean, local depth variance, normal vector angle, contour maxima, and contour minima, transforming them into one-dimensional vectors. Simultaneously, the system downsamples or flattens the original depth values within the target detection region under the same size rules, generating a one-dimensional vector of depth distance information. Then, a vector concatenation operation is used to join all the one-dimensional vectors end-to-end, fusing them into a complete one-dimensional feature descriptor. The fusion order can follow a preset fixed arrangement, such as depth distance components, local depth mean components, local depth variance components, normal vector angle components, contour maxima components, and contour minima components. This feature descriptor comprehensively characterizes the spatial structure within the target area from multiple levels, including the original depth distance, local undulation, surface orientation, and geometric boundaries.
[0094] Based on the above steps, by unifying multidimensional geometric statistical features with the original depth distance information into a fixed-dimensional spatial structure descriptor, a complete and compact representation of the three-dimensional morphology of the target field from the bottom physical distance to the upper geometric attributes is achieved, providing a one-dimensional input feature containing rich spatial geometric clues for lightweight networks.
[0095] S504. Map the feature description information to the feature dimensions to obtain structured features.
[0096] In one possible implementation, the system adds a single-layer fully connected mapping layer at the front end of the lightweight neural network. This fully connected layer receives a fixed-dimensional one-dimensional feature descriptor as input and maps it to a new one-dimensional vector with dimensions that precisely match the requirements of subsequent network inputs through matrix multiplication. The system then reshapes this one-dimensional vector into a structured feature map with a predetermined height, width, and number of channels, for example, a single-channel square feature map. The weight parameters of this fully connected mapping layer are optimized through backpropagation synchronously with the weight parameters of the backbone network during model training, and are not initialized using random fixed values or offline irrelevant data.
[0097] As an example, if the fused one-dimensional feature descriptor has a dimension of 512, and the target lightweight backbone network requires a single-channel image of 224×224 as input, then the output dimension of the fully connected mapping layer is 50176.
[0098] Based on the above steps, a single fully connected mapping layer is added to complete the dimensional transformation and spatial reshaping from a one-dimensional spatial structure descriptor to a structured feature map that conforms to the network input specifications, ensuring smooth and lossless connection of the data flow throughout the entire classifier pipeline.
[0099] S505. Input the structured features into a lightweight neural network to classify the occupancy state, and obtain the classification result and classification confidence of the current frame.
[0100] In one possible implementation, the lightweight neural network employs a backbone architecture based on depthwise separable convolutions. The system first feeds the structured feature map into a feature extraction network for layer-by-layer abstraction. The feature extraction network consists of multiple cascaded inverse residual blocks. Each inverse residual block contains a pointwise convolution for expanding the number of channels, a depthwise convolution for spatial feature extraction, and a pointwise convolution for compressing the number of channels. A lightweight channel attention module is embedded within the bottleneck layer to dynamically adjust the importance weights of the feature channels. At the end of the feature extraction network, the system uses a global average pooling layer to globally aggregate the features of the final output feature map along the spatial dimension, compressing the two-dimensional spatial response of each feature channel into a scalar, forming a high-dimensional global feature vector independent of spatial location. This global feature vector is then fed into a classification head, which consists of two cascaded fully connected layers. The last fully connected layer outputs two values, corresponding to the original scores for the empty pen and pig-containing categories, respectively. A Softmax normalization function transforms these two scores into mutually exclusive probabilities of an empty pen state and a pig-containing state, the sum of which is 1. The system uses the category corresponding to the higher probability value as the classification result of the current frame, and uses the probability value itself as the classification confidence.
[0101] It should be noted that the global feature aggregation step eliminates the influence of the spatial location inside the target pen on the determination of occupancy status, so that the classification network focuses on the overall three-dimensional morphological characteristics of the entire pen area, rather than local textures or geometric details of specific locations. This helps to improve the adaptability to changes in the posture of pigs standing, lying down, turning around, etc. in different positions within the pen.
[0102] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 6 As shown, the above S205 can be specifically implemented through the following S601 to S603, which are explained in detail below: S601. Obtain the classification results and classification confidence of multiple consecutive frames, and perform temporal smoothing processing based on the classification confidence of the current frame and the temporal classification confidence of the previous time step to obtain the temporal classification confidence of each corresponding frame.
[0103] Among them, the temporal classification confidence score refers to a smoothed value that, after being smoothed in the temporal dimension, is used to replace the original classification confidence score of a single frame in subsequent state decisions. This value combines the observation information of the current frame and the smoothing results of historical frames.
[0104] In one possible implementation, the system employs an exponential moving average method to smooth the original classification confidence sequence. For frame t, the system reads the original classification confidence of the current frame and the temporal classification confidence of the previous time step stored in memory, and then performs a weighted sum of the two according to a preset historical retention weight to obtain the temporal classification confidence of the current frame. The smoothing weight can be fixed or dynamically adjusted according to the stability of the detection scene.
[0105] Preferably, the timing smoothing process satisfies the following formula:
[0106] in, This represents the temporal classification confidence score of frame t after temporal smoothing. This represents the temporal classification confidence score of the previous frame. This represents the original classification confidence score of the current frame output by the lightweight neural network in frame t. Weights are retained for historical states, with values ranging from (0,1). When the value is large, the smoothed confidence level depends more on the historical state and has stronger suppression of instantaneous single-frame noise; when When the value is small, the smoothed confidence score follows the inference results of the current frame more quickly.
[0107] Based on the above steps, by exponentially weighting and fusing the original classification confidence of the current frame with the historical time series confidence, the disturbance of single-frame inference noise on the state decision is effectively reduced without increasing the computational overhead, providing a smooth and reliable confidence input for the stable transition of the subsequent three-state machine.
[0108] S602. Based on the temporal classification confidence and preset state transition rules, the target pen is transitioned between the state of having pigs, the state of pending confirmation, and the state of empty pen. The empty pen detection state of the current target pen is determined based on the number of frames that continuously meet the state transition conditions.
[0109] The three-state finite state machine (FSM) models the occupancy status of a target pen as a set of three mutually exclusive discrete states: a pen with pigs, a pen awaiting confirmation, and an empty pen. The "with pigs" state indicates the presence of a pig in the pen, the "empty pen" state indicates no pigs are in the pen, and the "pending confirmation" state is an intermediate state between the two, used to absorb uncertainties caused by pig activity at pen boundaries or transient interference from probes or sensors. The state transition rules define the pre-defined logic under what conditions transitions between states are allowed.
[0110] In one possible implementation, the system maintains an independent continuous frame counter for each target field. This counter is managed independently of the state variables and is used only to record the cumulative number of frames that continuously meet the transition conditions in the current state. In each frame, the system compares the smoothed temporal classification confidence score with a preset empty field determination confidence threshold. The state transition rules are as follows: When the current state is "with pigs", if the temporal classification confidence of the current frame is greater than the empty pen determination confidence threshold, the counter increments by 1, and the state remains "with pigs". If the temporal classification confidence of the current frame is less than or equal to the empty pen determination confidence threshold, the counter is immediately cleared to zero, and the state remains "with pigs". Only when the counter continuously accumulates to the first preset frame number threshold, the state machine jumps from the "with pigs" state to the "awaiting confirmation" transition state in one go, and the counter is cleared to zero after the jump is completed.
[0111] When the current state is the pending confirmation transition state, if the temporal classification confidence of the current frame is greater than the empty pen determination confidence threshold, the counter increments by 1, and the state continues to remain in the pending confirmation transition state; if the temporal classification confidence of the current frame is less than or equal to the empty pen determination confidence threshold, the counter is immediately cleared to zero, and the state machine directly returns to the pig-in-state; only when the counter continuously accumulates to the second preset frame number threshold, the state machine jumps to the empty pen state, and the counter is cleared to zero.
[0112] When the current state is empty, if the temporal classification confidence of the current frame is less than the empty frame determination confidence threshold, the counter increments by 1 and the state remains empty; if the temporal classification confidence of the current frame is greater than or equal to the empty frame determination confidence threshold, the counter is immediately cleared to zero and the state remains empty; only when the counter continuously accumulates to the third preset frame number threshold, the state machine returns to the pending confirmation transition state.
[0113] It should be noted that in the above transition rules, there is no direct bidirectional jump between the pig-in-pen state and the empty-pen state. All state transitions must pass through a pending-confirmation transition state as an intermediate buffer, thereby preventing direct instantaneous jumps between the two extreme states of pig-in-pen and empty-pen caused by instantaneous interference in a single frame. Each counter reset operation is executed independently, and the reset condition is strictly decoupled from the state transition condition, with no ambiguous count overlap or condition intersection.
[0114] Based on the above steps, by constructing a three-state finite state machine and defining complete state transition conditions and independent counter clearing logic, while maintaining a sensitive response to real pig entry and exit events, decision jitter caused by instantaneous noise and local activities is avoided from the perspective of timing logic.
[0115] S603. Calculate the temporal stability parameters based on the temporal classification confidence of consecutive multiple frames, and obtain the stability parameters by combining the proportion of the current target field with the positional change of the target detection area in consecutive frames.
[0116] The temporal fluctuation value refers to the dispersion measure calculated from the temporal classification confidence sequence of the most recent consecutive frames, used to quantify the stability of the classification confidence over a current period. The confidence region proportion refers to the ratio of the number of pixels contained in the confidence region generated by S202 in the current frame to the total number of pixels in the current target detection region, used to measure the overall confidence quality level of the current depth data source. The positional change of the target detection region refers to the distance between the center positions of the target detection regions in two adjacent frames on the image plane, used to measure the spatial drift of the detection box between consecutive frames.
[0117] In one possible implementation, the system calculates the sub-indicators for the three dimensions as follows: For the temporal fluctuation value, the system extracts the temporal classification confidence sequence within the most recent preset frame window, calculates the variance or standard deviation of this sequence as the temporal fluctuation value, and the lower the fluctuation value, the more stable the recent classification confidence and the higher the reliability of the detection state. For the reliable region proportion, the system counts the number of pixels marked as reliable within the current target detection region in the binary reliable mask generated by S202, divides it by the total pixel area of the current target detection region, and obtains a ratio between 0 and 1. The higher the ratio, the more complete the effective coverage of the depth data in the current frame within the field area. For the positional change of the target detection region, the system takes the center coordinates of the target detection region in the current frame and the center coordinates of the target detection region in the previous frame, calculates the Euclidean distance between the two points as the positional change, and the larger this value, the worse the spatial position convergence of the detection region in the recent period. The system fuses the above three sub-indicators—temporal fluctuation value, reliable region proportion, and positional change—which have different dimensions and physical meanings, through weighted summation. After fusion, a stability parameter is obtained, which can collaboratively reflect the operational health of the entire detection process from three dimensions: the numerical stationarity of time-series confidence, the data quality of depth perception, and the geometric stability of spatial positioning.
[0118] Based on the above steps, by organically integrating the three independent metrics of time-series confidence fluctuation, data reliability coverage, and spatial location drift into a comprehensive stability parameter, a systematic quantitative evaluation of the operational quality of the entire detection chain is achieved, providing multi-dimensional comparable criteria for triggering decisions of closed-loop feedback and accurately tracing the causes of fluctuations.
[0119] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 7 As shown, the above S206 can be specifically implemented through the following S701 to S706, which are explained in detail below: S701. Compare the stability parameter with the preset stability threshold to determine whether the detection state corresponding to the current target field is stable.
[0120] The preset stability threshold is a quantitative threshold used to determine whether the output results of the entire detection process are reliable. This threshold can be set based on empirical data during the system initialization phase.
[0121] In one possible implementation, the system compares the stability parameter corresponding to the current frame with a preset stability threshold. When the comprehensive score, which is the result of fusing the stability parameter (indicating the fluctuation of the temporal classification confidence, the proportion of the reliable region, and the change in the target detection region position), falls within the stable interval defined by the stability threshold, the system determines that the detection state of the current target field is stable; otherwise, it is determined to be unstable.
[0122] It should be noted that the construction of this stability parameter covers three orthogonal dimensions: the smoothness of temporal inference, the quality level of deep data, and the convergence of spatial localization. Therefore, it can comprehensively reflect the current operational health of the detection pipeline and avoid misjudgments caused by fluctuations in a single dimension indicator.
[0123] S702. When the detection status is stable, output the empty column detection result corresponding to the current target column.
[0124] In one possible implementation, if the current detection state is determined to be stable, it indicates that the entire link from deep data acquisition, trusted region generation, ROI correction, feature encoding to time-series decision-making is in a normal and controlled state, and the current empty fence detection state output by the state machine in S205 is directly used as the final detection result.
[0125] S703. When the detection status is unstable, determine whether the instability is caused by abnormal depth data quality based on the proportion of the trusted region.
[0126] In one possible implementation, when S701 determines that the detection state is unstable, the system does not immediately output a risky result, but instead initiates fluctuation source analysis. The system reads the percentage of the reliable region in the current frame and compares it with a preset threshold for the percentage of the reliable region. This threshold is set based on the following: when depth data deteriorates due to sensor-level issues such as water stain reflection, infrared absorption, or depth holes, the percentage of the reliable region is usually significantly lower than the normal value; while when pigs actually enter or leave the pen or undergo significant posture changes, although the time-series classification confidence will fluctuate, the percentage of the reliable region of the depth data itself can still maintain a high level. Therefore, if the percentage of the reliable region is lower than this threshold, the system determines that the instability is caused by abnormal depth data quality, belonging to algorithm fluctuation anomalies; conversely, if the percentage of the reliable region is higher than or equal to this threshold, the system determines that the instability is more likely caused by changes in the actual state of the target within the pen, belonging to fluctuations in the actual state of the target.
[0127] Based on the above steps, by using the proportion of the credible region as a diagnostic signal for fluctuation tracing, accurate identification of the root causes of instability can be achieved.
[0128] S704. When the detection state is unstable due to abnormal depth data quality, adjust the confidence threshold used to generate pixel-level depth confidence and the adaptive correction parameters used to correct the target detection area, and execute the re-inference process based on the adjusted parameters.
[0129] In one possible implementation, when S703 determines that the current fluctuation is an abnormal fluctuation in the algorithm, the system checks whether the number of re-inferences executed in this round of detection has reached the preset maximum number of feedbacks. If the upper limit has not been reached, the system triggers a closed-loop feedback parameter tuning action. The parameter tuning action includes at least two aspects: First, increasing the confidence threshold, that is, appropriately raising the threshold requirement for judging whether a pixel belongs to a confident pixel, thereby filtering out more pixels on the edge of confidence that may be contaminated by reflection or noise; Second, increasing the ROI correction step size, that is, increasing the correction force when the target detection area responds to deviations in each dimension, so that the detection area can get rid of the influence of erroneous contour information caused by low-quality depth data more quickly. After adjusting the parameters, the system does not directly output the current result, but re-executes the complete forward inference and temporal decision process from S202 to S205 with the adjusted parameters, that is, triggering re-inference.
[0130] Based on the above steps, when the degradation of deep data quality causes detection instability, the system enhances its anti-interference capability in two dimensions: tightening data quality gating and strengthening spatial correction response. It also verifies the convergence effect after parameter tuning through a re-inference mechanism, achieving adaptive suppression of harsh working conditions without manual intervention.
[0131] S705. Based on the revised target detection area, re-encode the spatial structure features, classify the occupancy status, and perform time sequence consistency analysis to obtain the stability parameters again. Then, based on the newly obtained stability parameters, determine whether the detection status corresponding to the current target field is stable. Based on the re-determination result, decide whether to output the result or continue feedback.
[0132] In one possible implementation, the re-inference process triggered by the previous parameter tuning step will be re-executed sequentially: a new reliable region will be generated based on the new reliability threshold; valid contours will be re-extracted based on the new reliable region, and the target detection region will be corrected using an increased step size; spatial structure feature encoding and lightweight network occupancy state classification will be re-executed based on the new target detection region; and multi-frame temporal consistency analysis will be re-performed and new stability parameters will be calculated. The new stability parameters obtained from the re-inference will re-enter the decision logic of S701: if the new stability parameters have fallen back to the stable range of the preset stability threshold, the re-inference is considered to have converged, and the current empty column detection state is directly output as the final result; if the new stability parameters are still higher than the threshold and the cumulative number of re-inferences in this round has not yet reached the maximum feedback number, the parameter tuning and re-inference iterations of S703 and S704 will be executed again based on the adjusted parameters.
[0133] S706. When the detected state stabilizes again or the preset number of feedback attempts is reached, output the empty column detection result corresponding to the current target column, and reset the execution parameters according to the exit scenario differentiation. When the unstable detection state is not caused by abnormal depth data quality, keep the current confidence threshold and adaptive correction parameters unchanged, and output the empty column detection result corresponding to the current target column.
[0134] In one possible implementation, the closed-loop feedback process has two explicit exit paths and corresponding parameter reset rules. The first exit path corresponds to the normal feedback exit scenario: after re-inference, the stability parameter recovers to the stable range, and the cumulative number of feedback iterations in this round has not reached the maximum number of feedback iterations. In this scenario, the system outputs the detection result and restores the amplified ROI correction step size to the initial baseline step size value, but retains the increased confidence threshold. The second exit path corresponds to the forced termination exit scenario: the cumulative number of feedback iterations in this round has reached the preset maximum number of feedback iterations, but after re-inference, the stability parameter still has not fallen back to the stable range. In this scenario, the system determines that the current operating condition has exceeded the algorithm's adaptive compensation capability. To prevent the parameters from continuously deviating from a reasonable range and affecting the normal detection of subsequent frames, the system performs a comprehensive parameter reset, that is, restoring the confidence threshold to the system's initial default value, restoring the ROI correction step size to the initial baseline value, clearing the feedback count to zero, and outputting the empty-field detection state determined by the current state machine as the final result. For unstable scenarios in S703 where the target's true state fluctuates, the system's judgment parameters themselves do not need to be adjusted. All closed-loop parameter tuning, re-inference, and parameter reset operations are skipped. The current confidence threshold and adaptive correction parameters remain unchanged, and the empty fence detection state determined by the current state machine is directly output as the final result.
[0135] Based on the above steps, by setting differentiated parameter reset rules for the closed-loop feedback mechanism under two scenarios of normal exit and forced termination exit, and by clearly defining the temporal constraint that the ROI step size is amplified only within the re-inference lifecycle, defects such as closed-loop parameter divergence, threshold overflow failure and rule mutual exclusion are eliminated, ensuring the parameter convergence and output stability of the detection system under long-term continuous operation and extreme conditions.
Claims
1. A method for pig inventory detection based on credibility driving, characterized in that, The method includes: Obtain the depth image of the pigpen to be inspected; Pixel-level depth confidence is constructed based on the depth image, and the confidence of each pixel in the depth image is evaluated. A confidence region is generated based on the confidence of each pixel. The process of constructing pixel-level depth confidence based on the depth image, evaluating the confidence of each pixel in the depth image, and generating a confidence region based on the confidence of each pixel includes: Based on the depth information corresponding to each pixel in the depth image, the depth gradient continuity, instantaneous stability, depth missing compensation degree, and historical time series stability of each pixel are calculated respectively. The depth confidence level of each pixel is constructed based on the depth gradient continuity, the instantaneous temporal stability, the depth missing compensation degree, and the historical temporal stability. The depth confidence level is compared with a preset confidence threshold to determine the target pixel that meets the confidence condition, and a confidence region is generated based on the target pixel. The depth holes in the trusted region are filled to obtain a trusted depth map and trusted region for target field detection; The effective contour information of the target field is extracted based on the reliable region, and the preset target detection region is adaptively corrected based on the effective contour information to obtain the target detection region corresponding to the target field. The depth information within the target detection area is encoded with spatial structure features to generate feature description information that characterizes the three-dimensional spatial structure of the target field. The feature description information is then input into a lightweight neural network for occupancy state classification to obtain the classification result and classification confidence of the current frame. Based on the classification results and classification confidence scores of multiple consecutive frames, a temporal consistency analysis is performed to determine the empty column detection state of the current target column, and a stability parameter used to characterize the detection stability is calculated. The process of performing temporal consistency analysis based on the classification results and classification confidence scores of multiple consecutive frames to determine the empty column detection state of the current target column and calculating the stability parameter used to characterize the detection stability includes: Obtain the classification results and classification confidence scores for multiple consecutive frames; Temporal smoothing is performed based on the classification confidence of the current frame and the temporal classification confidence of the previous time step to obtain the temporal classification confidence of each frame. Based on the time-series classification confidence and the preset state transition rules, the target pen is transitioned between the state of having pigs, the state of pending confirmation, and the state of being empty. The empty pen detection state of the current target pen is determined based on the number of frames that continuously meet the state transition conditions. Temporal stability parameters are calculated based on the temporal classification confidence of multiple consecutive frames, and the stability parameters are obtained by combining the proportion of the current target field with the positional change of the target detection area in consecutive frames. Based on the stability parameters and preset rules, empty shoal detection is performed to obtain the detection results.
2. The method according to claim 1, characterized in that, The calculation of depth gradient continuity, instantaneous temporal stability, depth loss compensation degree, and historical temporal stability for each pixel based on the depth information corresponding to each pixel in the depth image includes: Based on the preset neighborhood depth value of each pixel, the degree of dispersion of the depth value within the preset neighborhood is calculated, and the depth gradient continuity of the corresponding pixel is determined according to the degree of dispersion. Based on the depth values of the corresponding pixels in the current frame and the previous frame, the depth change of the corresponding pixel is calculated, and the instantaneous stability of the corresponding pixel is determined according to the depth change. Determine whether each pixel has a depth defect; when the pixel does not have a depth defect, the pixel is determined as a valid depth pixel; when the pixel has a depth defect, perform depth compensation on the depth defect pixels that meet the compensation conditions, and determine the depth defect compensation degree of the corresponding pixel based on the compensation result; if it does not meet the conditions, the depth defect compensation degree is determined to a preset minimum value. Establish a historical sliding window corresponding to each pixel, perform confidence filtering on the corresponding pixels of each frame within the historical sliding window, record the binary state of whether the corresponding pixel of each frame passes the confidence filtering, and determine the historical temporal stability of the corresponding pixel based on the statistical results of the binary state.
3. The method according to claim 1, characterized in that, The step of extracting effective contour information of the target field based on the reliable region, and adaptively correcting the preset target detection region based on the effective contour information to obtain the target detection region corresponding to the target field includes: Obtain the initial target detection area corresponding to the pre-calibrated target field; The effective contour of the target field is extracted based on the reliable region, and the center position, width, height and tilt angle of the target field are calculated based on the effective contour. Calculate the position offset, width offset, height offset, and angle offset corresponding to the initial target detection area based on the center position of the area, the width of the area, the height of the area, and the tilt angle of the area, respectively. Based on the position offset, width offset, height offset, and angle offset, the center position, width, height, and tilt angle of the initial target detection area are synchronously corrected to obtain the target detection area. The position offset, width offset, height offset, and angle offset are limited according to the preset step size constraint.
4. The method according to claim 1, characterized in that, The process involves encoding spatial structure features of the depth information within the target detection area to generate feature description information characterizing the three-dimensional spatial structure of the target field. This feature description information is then input into a lightweight neural network for occupancy state classification to obtain the classification result and classification confidence score for the current frame. This includes: A depth gradient field, a spatial normal vector field, and a 3D contour edge map are constructed based on the depth information within the target detection area. Using a preset neighborhood as a statistical window, calculate the local depth mean and local depth variance in the depth gradient field, the angle between normal vectors in the spatial normal vector field, and the contour maxima and contour minima in the three-dimensional contour edge map. The local depth mean, local depth variance, normal vector angle, contour maxima, contour minima, and depth distance information within the target detection area are fused to obtain feature description information for characterizing the three-dimensional spatial structure of the target column. The feature description information is mapped to feature dimensions to obtain structured features; The structured features are input into a lightweight neural network for occupancy state classification to obtain the classification result and classification confidence of the current frame.
5. The method according to claim 4, characterized in that, The step of inputting the structured features into a lightweight neural network for occupancy state classification to obtain the classification result and classification confidence of the current frame includes: Multi-layer feature extraction is performed on the structured features based on the feature extraction network to obtain the deep features corresponding to the target field; Global feature aggregation is performed on the deep features to obtain the global features corresponding to the target field; Based on the global features, perform binary classification to output the probability of an empty pen and the probability of a pen containing pigs for the target pen. The classification result and classification confidence of the current frame are determined based on the probability of the empty pen state and the probability of the pig-in-pen state.
6. The method according to claim 1, characterized in that, The stability parameters are obtained by combining the proportion of the current target field's reliable region with the positional change of the target detection region in consecutive frames, including: The temporal fluctuation value corresponding to the target field is determined based on the temporal classification confidence of multiple consecutive frames; The proportion of the trusted region is determined based on the number of trusted pixels within the trusted region and the total number of pixels within the target detection region; The positional change of the target detection region is determined based on the center position of the target detection region in two consecutive frames. The stability parameter is obtained by fusing the temporal fluctuation value, the proportion of the reliable region, and the positional change of the target detection region.
7. The method according to claim 1, characterized in that, The step of performing empty berth detection according to the stability parameters and preset rules to obtain the detection result includes: The stability parameter is compared with a preset stability threshold to determine whether the detection state corresponding to the current target field is stable. When the detection state is stable, the empty column detection result corresponding to the current target column is output; When the detection state is unstable, determine whether the instability of the detection state is caused by abnormal depth data quality based on the proportion of the trusted region. When the unstable detection state is caused by abnormal depth data quality, the confidence threshold used to generate pixel-level depth confidence and the adaptive correction parameters used to correct the target detection region are adjusted, and the confidence region is regenerated based on the adjusted confidence threshold. The effective contour information of the target field is re-extracted based on the regenerated confidence region, and the target detection region is corrected again. Based on the revised target detection area, spatial structure feature encoding, occupancy status classification, and time sequence consistency analysis are performed again to obtain stability parameters. Based on the obtained stability parameters, it is determined whether the detection status corresponding to the current target field is stable. When the detection status is stable again or the preset number of feedbacks is reached, the empty column detection result corresponding to the current target column is output. When the unstable detection state is not caused by abnormal depth data quality, the current confidence threshold and adaptive correction parameters remain unchanged, and the empty column detection result corresponding to the current target column is output.
8. A credibility-driven system for detecting empty pig pens, used to execute the method described in any one of claims 1 to 7, characterized in that, The system includes: The depth image acquisition module is used to acquire the depth image of the pigpen to be inspected; A trusted region generation module is used to construct pixel-level depth confidence based on the depth image, evaluate the confidence of each pixel in the depth image, and generate trusted regions based on the confidence of each pixel. The steps of constructing pixel-level depth confidence based on the depth image, evaluating the confidence of each pixel in the depth image, and generating trusted regions based on the confidence of each pixel include: calculating the depth gradient continuity, instantaneous temporal stability, depth missing compensation degree, and historical temporal stability of each pixel based on the depth information corresponding to each pixel in the depth image; constructing the depth confidence corresponding to each pixel based on the depth gradient continuity, instantaneous temporal stability, depth missing compensation degree, and historical temporal stability; comparing the depth confidence with a preset confidence threshold to determine target pixels that meet the confidence conditions, and generating trusted regions based on the target pixels; and performing depth hole filling processing on the trusted regions to obtain a trusted depth map and trusted regions for target field detection. The target detection region correction module is used to extract the effective contour information of the target field based on the reliable region, and to adaptively correct the preset target detection region based on the effective contour information to obtain the target detection region corresponding to the target field. The occupancy status classification module is used to encode the spatial structure features of the depth information in the target detection area, generate feature description information to characterize the three-dimensional spatial structure of the target column, and input the feature description information into a lightweight neural network for occupancy status classification to obtain the classification result and classification confidence of the current frame. The temporal consistency analysis module is used to perform temporal consistency analysis based on the classification results and classification confidence of multiple consecutive frames, determine the current empty pen detection state of the target pen, and calculate stability parameters to characterize detection stability. The process of performing temporal consistency analysis based on the classification results and classification confidence of multiple consecutive frames to determine the current empty pen detection state of the target pen and calculate stability parameters to characterize detection stability includes: acquiring the classification results and classification confidence of multiple consecutive frames; performing temporal smoothing based on the classification confidence of the current frame and the temporal classification confidence of the previous time step to obtain the temporal classification confidence of each frame; based on the temporal classification confidence and preset state transition rules, performing state transitions for the target pen between the pig-present state, the pending confirmation state, and the empty pen state, and determining the current empty pen detection state of the target pen based on the number of consecutive frames that meet the state transition conditions; calculating the temporal stability parameters based on the temporal classification confidence of multiple consecutive frames, and combining the percentage of the current target pen's reliable region and the positional change of the target detection region in consecutive frames to obtain the stability parameters. The empty space detection module is used to perform empty space detection according to the stability parameters and preset rules to obtain the detection results.
Citation Information
Patent Citations
Pixel-level activity amount-based live pig health detection method and system
CN122091213A
Animal count identification method, device, medium, and electronic apparatus
WO2020030052A1
