A management platform and management system for monitoring pet health

Abstract

The present application relates to the technical field of pet health management, in particular to a management platform and system for monitoring pet health, which comprises: a data acquisition module: collecting multi-modal video stream data of pets in daily environment, separating the multi-modal video stream data to obtain color image sequences and depth image sequences; a semantic analysis module: performing semantic segmentation processing on the color image sequences, extracting semantic features, and obtaining terrain semantic feature data; a terrain mapping module: performing three-dimensional space mapping based on the terrain semantic feature data and the depth image sequences. In the present application, the obtained pure gait data can improve the accuracy and reliability of abnormality detection when compared with individual health baseline models, avoiding misjudgment of normal environmental adaptation behavior of pets as pathological symptoms, thereby laying a solid data foundation for generating highly targeted exercise rehabilitation recommendations and enhancing the scientific nature of health assessment.

Description

Technical Field

[0001] This invention relates to the field of pet health management technology, and in particular to a management platform and system for monitoring pet health. Background Technology

[0002] Pet health management technology refers to the continuous, non-invasive or minimally invasive data collection, analysis and evaluation of a pet's physiological state, behavioral patterns and living environment by using modern information technology, Internet of Things technology, artificial intelligence and biomedical sensing technology.

[0003] Existing pet health management technologies suffer from confusion between environmental and individual behavioral signals in practice. Their data analysis models are often built on a single data dimension lacking environmental context. For example, technologies that partially rely on accelerometers in wearable devices or simple cameras can monitor the intensity of a pet's activity or record its movement, but struggle to perceive the pet's physical environment. This inherent limitation prevents the system from distinguishing whether changes in a pet's gait stem from physiological pain or adaptive adjustments to the environment. For instance, when a pet walks on a freshly mopped, wet, slippery floor, its shortened stride and hesitant gait to prevent slipping can easily be misinterpreted as signs of joint pain or neurological abnormalities in the absence of environmental data, leading to false alarms. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a management platform and system for monitoring pet health.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a management system for monitoring pet health includes:

[0006] Data acquisition module: Acquires multimodal video stream data of pets in their daily environment, and separates the multimodal video stream data to obtain color image sequences and depth image sequences;

[0007] Semantic analysis module: Performs semantic segmentation on the color image sequence, extracts semantic features, and obtains terrain semantic feature data;

[0008] Terrain mapping module: performs three-dimensional spatial mapping based on the terrain semantic feature data and the depth image sequence, and calculates and generates a terrain compensation matrix with three-dimensional terrain compensation information;

[0009] Pose extraction module: Input the color image sequence into the label-free pose estimation network for feature extraction, and track and extract the two-dimensional skeletal key point coordinate sequence of the pet;

[0010] 3D Reconstruction Module: Merges the depth image sequence with the 2D skeletal key point coordinate sequence, performs coordinate transformation, and estimates the transformed 3D pose sequence data;

[0011] Gait analysis module: Combines the three-dimensional posture sequence data and the terrain compensation matrix to perform decoupling analysis and calculate the environmental decoupled gait feature set;

[0012] Health assessment module: Input the environmentally decoupled gait feature set into a pre-established individual health baseline model for anomaly detection and assessment to obtain gait anomaly assessment results;

[0013] Rehabilitation Recommendation Module: Based on the gait abnormality assessment results, a rehabilitation plan is matched, and corresponding exercise rehabilitation suggestions are generated.

[0014] Preferably, the step of separating the multimodal video stream data to obtain the color image sequence and the depth image sequence specifically includes:

[0015] The multimodal video stream data collected by the sensing device is acquired and cached in a local data buffer pool;

[0016] The multimodal video stream data in the local data buffer pool is parsed to extract the file header description information of the multimodal video stream data, and to read the video format identifier and hardware timestamp information.

[0017] The corresponding video demultiplexer is invoked according to the video format identifier, and the video demultiplexer is used to deinterleave and separate the multimodal video stream data to obtain the original color data stream and the original depth data stream.

[0018] The original color data stream is sampled using keyframes to generate the color image sequence;

[0019] The original depth data stream is time-synchronized and aligned according to the hardware timestamp information. Depth data frames with absolute time deviations exceeding a preset tolerance threshold are removed, and the retained time-matched depth image frames are stitched together to generate the depth image sequence.

[0020] Preferably, the step of performing semantic segmentation on the color image sequence to extract semantic features and obtain terrain semantic feature data specifically includes:

[0021] Each frame of the color image sequence is converted into feature tensor data by color space transformation.

[0022] The feature tensor data is input into a pre-trained mask segmentation network model to extract multi-scale spatial visual features from the feature tensor data.

[0023] The pixel-level classification prediction branch in the mask segmentation network model is used to perform pixel-by-pixel forward inference calculation on the multi-scale spatial visual features to identify the ground material category label and surface obstacle category label of each pixel.

[0024] According to the preset ground material attribute mapping table, the static friction coefficient associated with the ground material category label and the surface obstacle category label of each region in each frame image is found, and the found static friction coefficient is used as the surface friction reference coefficient of the corresponding pixel region.

[0025] Two-dimensional planar spatial coordinates of each pixel region in each frame of the image are extracted. The surface friction reference coefficient and the two-dimensional planar spatial coordinates are then concatenated using feature dimensions to generate terrain semantic feature data that characterizes the surface properties of the environment.

[0026] Preferably, the step of performing three-dimensional spatial mapping based on the terrain semantic feature data and the depth image sequence, and calculating and generating a terrain compensation matrix with three-dimensional terrain compensation information, specifically includes:

[0027] A perspective back-projection equation is constructed based on a pre-calibrated camera intrinsic parameter matrix. The perspective back-projection equation is then used to back-project the depth values ​​of pixels in the depth image sequence, converting them into a set of three-dimensional point cloud coordinates.

[0028] Extract the two-dimensional planar spatial coordinates and the surface friction reference coefficient contained in the terrain semantic feature data, and attach and map the surface friction reference coefficient to the corresponding three-dimensional point position in the three-dimensional point cloud coordinate set according to the correspondence to generate a semantic three-dimensional point cloud set.

[0029] The semantic 3D point cloud set is calculated using a local plane fitting reconstruction algorithm to obtain continuous terrain surface normal vectors and terrain slope data.

[0030] By combining the surface friction reference coefficient, the vertical component in the terrain surface normal vector, and the terrain slope data, a nonlinear environmental disturbance penalty function based on sliding critical risk is established. The compensation weight value of each spatial grid cell is calculated through the nonlinear environmental disturbance penalty function to construct the terrain compensation matrix.

[0031] Preferably, the step of inputting the color image sequence into a label-free pose estimation network for feature extraction, and tracing the extracted two-dimensional skeletal keypoint coordinate sequence of the pet, specifically includes:

[0032] The color image sequence is subjected to adaptive histogram equalization enhancement processing to improve edge contrast features, resulting in an enhanced color video sequence.

[0033] The enhanced color video sequence is input into the labelless pose estimation network. Global body features are extracted through downsampling convolutional layers, and spatial resolution restoration is performed on the global body features through upsampling deconvolutional layers to generate a set of joint probability heatmaps.

[0034] For each single-channel heatmap in the set of joint probability heatmaps, a non-maximum suppression algorithm is applied to find the local probability peak response location, and the local probability peak response location is used as the basic two-dimensional coordinate data;

[0035] Import the preset prior constraints on the connectivity of the quadruped skeleton, and use the prior constraints on the connectivity of the quadruped skeleton to perform topological relationship distance verification between adjacent joints on the basic two-dimensional coordinate data;

[0036] Remove outlier coordinates with abnormal distances from the basic two-dimensional coordinate data, and output the coordinate sequence of the two-dimensional skeletal key points that has been fully verified.

[0037] Preferably, the step of fusing the depth image sequence and the two-dimensional skeletal keypoint coordinate sequence for coordinate transformation, and estimating the transformed three-dimensional pose sequence data, specifically includes:

[0038] The time-synchronized depth image frame that strictly corresponds to the coordinate sequence of the two-dimensional skeleton key points is traversed in the depth image sequence.

[0039] The coordinates of key points in the two-dimensional skeleton key point coordinate sequence are analyzed, and the corresponding spatial depth observation values ​​are located in the found time-synchronized depth image frames.

[0040] Determine whether the spatial depth observation value is an invalid hole value; if it is an invalid hole value, extract the surrounding valid depth pixel values, and use the dual Gaussian weighted algorithm based on key point gravity to calculate the valid depth pixel values ​​to estimate the corrected depth value; if it is not an invalid hole value, use the spatial depth observation value directly as the corrected depth value.

[0041] By combining the key point coordinates and the corrected depth value, a coordinate system transformation is performed to calculate the three-dimensional physical coordinate data of each target joint.

[0042] The three-dimensional spatial physical coordinate data are spliced ​​and combined in chronological order to generate the three-dimensional attitude sequence data.

[0043] Preferably, the step of performing decoupling analysis by combining the three-dimensional pose sequence data and the terrain compensation matrix to calculate the environmental decoupled gait feature set specifically includes:

[0044] The three-dimensional posture sequence data is segmented using a dynamic time warping algorithm to identify ground contact events and ground lift events, and to divide the data into multiple continuous complete gait cycle sequences.

[0045] Calculate the original stride length, original leg alternation frequency, and original key joint flexion angle for each gait cycle time period in the multiple complete gait cycle sequences, and construct an initial gait characteristic state group from the calculation results.

[0046] Extract all spatial grid cells covered by the multiple complete gait cycle sequences, and read the terrain compensation weight value of the corresponding position from the terrain compensation matrix according to the coordinate index of all spatial grid cells;

[0047] The indicators in the initial gait feature state group are multiplied by the read terrain compensation weight value for attenuation calculation to generate the environment-decoupled gait feature set that is unaffected by environmental factors.

[0048] Preferably, the step of inputting the environmentally decoupled gait feature set into a pre-established individual health baseline model for anomaly detection and evaluating the gait anomaly assessment results specifically includes:

[0049] The pre-established individual health baseline model is invoked. The individual health baseline model is constructed by an isolated forest algorithm integrating multiple decision trees and is generated through unsupervised training based on long-term collected stable gait data.

[0050] The environmental decoupled gait feature set is mapped into the multidimensional feature vector space of the individual health baseline model. The number of segments required for the environmental decoupled gait feature set to be isolated in each decision tree is calculated. The average value of the number of segments is obtained to get the average path length of the environmental decoupled gait feature set in the multidimensional feature vector space.

[0051] The abnormal score value of the environmental decoupled gait feature set deviating from the healthy baseline distribution is calculated based on the obtained average path length.

[0052] The abnormal score is compared and analyzed with the system's preset dynamic alarm sensitivity threshold. If the abnormal score continuously exceeds the dynamic alarm sensitivity threshold to reach the preset alarm count limit, it is determined that the pet's gait force is asymmetrical, and the gait abnormality assessment result indicating the risk of early osteoarthritis is generated.

[0053] Preferably, the step of matching rehabilitation programs based on the gait abnormality assessment results and generating corresponding exercise rehabilitation recommendations specifically includes:

[0054] The generated gait abnormality assessment results are parsed, and the abnormality frequency index, the specific location identification code of the abnormal joint, and the degree of decline in motor function index contained in the gait abnormality assessment results are extracted.

[0055] Search criteria are constructed in a pre-set veterinary rehabilitation intervention expert knowledge database. Using the specific location identification code of the abnormal joint and the degree of decline in motor function as the matching primary key, intervention rule guidance entries that fully meet the criteria are retrieved.

[0056] Extract the intervention parameter templates defined in the intervention rule guidelines to generate a structured text of the initial rehabilitation plan, including the daily maximum exercise duration limit, the recommended suitable terrain type description, and the physiotherapy stretching exercise instructions designed for the home environment.

[0057] Based on the abnormality frequency index, the rehabilitation follow-up tracking cycle for the next comprehensive gait re-examination is set in the structured text of the initial rehabilitation plan.

[0058] The structured text of the initial rehabilitation plan and the rehabilitation follow-up tracking cycle are combined and formatted to generate the complete exercise rehabilitation recommendations.

[0059] The present invention also provides a management platform, including a server and at least one management system, wherein the management system is communicatively connected to the server.

[0060] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0061] In this invention, multimodal video stream data is collected to simultaneously acquire color image sequences and depth image sequences, providing a dual data foundation for subsequent environmental perception and subject posture analysis. Semantic segmentation of the color image sequences identifies external factors affecting gait, such as ground material and obstacles, and these are combined with the depth image sequences for three-dimensional spatial mapping. This allows for the calculation of a quantifiable terrain compensation matrix, which characterizes environmental interference factors such as slope and friction. Simultaneously, a label-free posture estimation network tracks and reconstructs a continuous three-dimensional posture sequence of the pet. Furthermore, decoupling analysis of the three-dimensional posture sequence using the aforementioned terrain compensation matrix removes gait artifacts induced by external environmental factors such as slippery or tilted surfaces, extracting an environmentally decoupled gait feature set that reflects only the pet's physiological condition. The resulting pure gait data, when compared with an individual health baseline model, improves the accuracy and reliability of anomaly detection, avoiding misinterpretation of normal environmental adaptation behaviors as pathological symptoms. This lays a solid data foundation for generating highly targeted exercise rehabilitation recommendations and enhances the scientific rigor of health assessments. Attached Figure Description

[0062] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0064] Please see Figure 1 The present invention provides a technical solution: a management system for monitoring pet health includes:

[0065] Data acquisition module: Collects multimodal video stream data of pets in their daily environment, and separates the multimodal video stream data to obtain color image sequences and depth image sequences;

[0066] Semantic analysis module: Performs semantic segmentation on color image sequences, extracts semantic features, and obtains terrain semantic feature data;

[0067] Terrain mapping module: Performs 3D spatial mapping based on terrain semantic feature data and depth image sequences, and calculates and generates a terrain compensation matrix with 3D terrain compensation information;

[0068] Pose extraction module: Input the color image sequence into the label-free pose estimation network for feature extraction, and track and extract the two-dimensional skeletal key point coordinate sequence of the pet;

[0069] 3D Reconstruction Module: Fuses depth image sequences and 2D skeleton keypoint coordinate sequences for coordinate transformation, and estimates the transformed 3D pose sequence data;

[0070] Gait analysis module: Combines 3D posture sequence data and terrain compensation matrix for decoupling analysis to calculate environmental decoupled gait feature set;

[0071] Health assessment module: Input the environmentally decoupled gait feature set into a pre-established individual health baseline model for anomaly detection and evaluation to obtain gait anomaly assessment results;

[0072] Rehabilitation Recommendation Module: Based on the gait abnormality assessment results, a rehabilitation plan is matched and corresponding exercise rehabilitation suggestions are generated.

[0073] In this embodiment, the steps of separating multimodal video stream data to obtain color image sequences and depth image sequences specifically include: acquiring multimodal video stream data collected by a sensing device and caching it in a local data buffer pool; parsing the multimodal video stream data in the local data buffer pool to extract the file header description information of the multimodal video stream data, and reading the video format identifier and hardware timestamp information; calling the corresponding video demultiplexer according to the video format identifier, and using the video demultiplexer to perform deinterleaving and separation processing on the multimodal video stream data to separate the original color data stream and the original depth data stream; performing keyframe sampling on the original color data stream to generate a color image sequence; performing time synchronization alignment verification on the original depth data stream according to the hardware timestamp information, removing depth data frames whose absolute time deviation exceeds a preset tolerance threshold, and stitching together the retained time-matched depth image frames to generate a depth image sequence.

[0074] Specifically, the process involves acquiring multimodal video stream data collected by sensing devices, caching the acquired multimodal video stream data in a local data buffer pool, performing packet splitting and parsing on the multimodal video stream data in the local data buffer pool, extracting the file header description information of the multimodal video stream data, reading the video format identifier and hardware timestamp information from the file header description information, calling the corresponding video demultiplexing program according to the video format identifier, and using the corresponding video demultiplexing program to perform deinterleaving and separation processing on the multimodal video stream data to separate the original color data stream and the original depth data stream. Keyframe extraction and sampling are performed on the original color data stream at fixed intervals to generate a color image sequence. Time synchronization alignment and verification are performed on the original depth data stream according to the hardware timestamp information, calculating the absolute value of the time deviation between the timestamp of the depth data frame and the timestamp of the color image frame, and comparing the absolute value of the time deviation with a preset tolerance threshold. A comparison is performed, and a preset tolerance threshold of 33 milliseconds is set. This value is calculated based on a video sampling rate of 30 frames per second. The specific calculation process is to divide 1000 milliseconds by 30 frames to calculate the interval time of a single frame. If the hardware timestamp information of a certain color image frame is 1000 milliseconds and the hardware timestamp information of an adjacent depth data frame is 1040 milliseconds, the absolute value of the time difference between the two is 40 milliseconds. The value of 40 milliseconds exceeds the preset tolerance threshold of 33 milliseconds, so the depth data frame with the deviation of 40 milliseconds is discarded. If the hardware timestamp information of another depth data frame is 1020 milliseconds, the corresponding deviation is 20 milliseconds. This value is lower than the preset tolerance threshold of 33 milliseconds, so the depth data frame with the deviation of 20 milliseconds is retained. Depth data frames with an absolute value of time deviation exceeding the preset tolerance threshold are discarded, and the retained time-matching depth image frames are sorted and stitched together in time sequence to generate a depth image sequence.

[0075] In this embodiment, the steps of performing semantic segmentation on a color image sequence to extract semantic features and obtain terrain semantic feature data specifically include: performing color space conversion on each frame of the color image sequence to convert it into feature tensor data; inputting the feature tensor data into a pre-trained mask segmentation network model to extract multi-scale spatial visual features of the feature tensor data; using the pixel-level classification prediction branch in the mask segmentation network model to perform pixel-by-pixel forward inference calculation on the multi-scale spatial visual features to identify the ground material category label and surface obstacle category label of each pixel; according to a preset ground material attribute mapping table, finding the static friction coefficient associated with the ground material category label and surface obstacle category label of each region in each frame of the image, and using the found static friction coefficient as the surface friction reference coefficient of the corresponding pixel region; extracting the two-dimensional planar spatial position coordinates of each pixel region in each frame of the image, and performing feature dimension concatenation processing on the surface friction reference coefficient and the two-dimensional planar spatial position coordinates to fuse and generate terrain semantic feature data representing the surface attributes of the environment.

[0076] Specifically, each frame in the color image sequence undergoes color space transformation, converting it into feature tensor data. A pre-trained mask segmentation network model is then constructed. This model employs an encoder-decoder architecture. The encoder contains five cascaded residual convolutional blocks, used for layer-by-layer downsampling and extraction of low-level texture information to high-level semantic information from the input feature tensor data. The decoder contains five feature pyramid upsampling layers symmetrical to the encoder, fusing the feature maps of the corresponding encoder layers through lateral feature connections to restore spatial resolution. The model's input data is color image feature tensor data with lateral resolution multiplied by vertical resolution multiplied by three-channel size. The output data is a dual-channel prediction result containing ground material category probability maps and surface obstacle category probability maps. During training, the cross-entropy loss function is used to calculate the error between the prediction results and manually labeled classification data. The backpropagation algorithm is used to update the convolutional kernel weights in the encoder and decoder, and training is continuously iterated for 20 iterations. After zero cycles until the loss function curve converges on the validation set, the feature tensor data is input into a pre-trained mask segmentation network model. Multi-scale spatial visual features of the feature tensor data are extracted. The pixel-level classification prediction branch in the mask segmentation network model is used to perform pixel-by-pixel forward inference calculation on the multi-scale spatial visual features, outputting the category with the highest probability. The ground material category label and surface obstacle category label of each pixel are identified. According to the preset ground material attribute mapping table, the static friction coefficient associated with the ground material category label and surface obstacle category label of each region in each frame image is found. For example, the static friction coefficient value corresponding to the wood floor label is 0.3. The two-dimensional planar spatial position coordinates of each pixel region in each frame image are extracted. The surface friction reference coefficient and the two-dimensional planar spatial position coordinates are concatenated along the feature dimension according to the channel direction to generate terrain semantic feature data representing the surface attributes of the environment.

[0077] In this embodiment, the step of performing three-dimensional spatial mapping based on terrain semantic feature data and depth image sequences to calculate and generate a terrain compensation matrix with three-dimensional terrain compensation information specifically includes: constructing a perspective back-projection equation based on a pre-calibrated camera intrinsic parameter matrix; using the perspective back-projection equation to back-project the depth values ​​of pixels in the depth image sequence, converting them into a three-dimensional point cloud coordinate set; extracting the two-dimensional planar spatial position coordinates and surface friction reference coefficients contained in the terrain semantic feature data; attaching and mapping the surface friction reference coefficients to the corresponding three-dimensional points in the three-dimensional point cloud coordinate set according to the correspondence, generating a semantic three-dimensional point cloud set; using a local plane fitting reconstruction algorithm to calculate the semantic three-dimensional point cloud set, solving for continuous terrain surface normal vectors and terrain slope data; combining the surface friction reference coefficients, the vertical component in the terrain surface normal vectors, and the terrain slope data, establishing a nonlinear environmental disturbance penalty function based on sliding critical risk; calculating the compensation weight value of each spatial grid cell through the nonlinear environmental disturbance penalty function, constructing a terrain compensation matrix; the formula for calculating the compensation weight value is: ;in, This represents the compensation weight value. This represents an exponential function with the natural constant as its base. This represents the preset penalty amplification factor. Represents terrain slope data. This represents the reference coefficient for surface friction. This represents a small constant pre-defined to prevent the denominator from being zero. This represents the vertical component in the terrain surface normal vector.

[0078] Specifically, a perspective back-projection equation is constructed based on a pre-calibrated camera intrinsic parameter matrix. This equation is then used to back-project the depth values ​​of pixels in the depth image sequence, converting them into a 3D point cloud coordinate set containing 3D coordinate data. Two-dimensional planar spatial coordinates and surface friction reference coefficients from the terrain semantic feature data are extracted. Following the projection correspondence between 2D and 3D space, the surface friction reference coefficients are mapped to the corresponding 3D points in the 3D point cloud coordinate set, generating a semantic 3D point cloud set carrying physical friction information. A local plane fitting reconstruction algorithm is used to perform point cloud plane fitting calculations on the semantic 3D point cloud set, obtaining continuous terrain surface normal vectors and terrain slope data reflecting the degree of inclination. Combining the surface friction reference coefficients, the vertical component of the terrain surface normal vectors, and the terrain slope data, a nonlinear environmental disturbance penalty function based on sliding critical risk is established. The compensation weight value for each spatial grid cell is calculated using this nonlinear environmental disturbance penalty function, constructing a terrain compensation matrix. The formula for calculating the compensation weight value is as follows: ,in, This represents the compensation weight value calculated according to the formula. This represents an exponential function with the natural constant as its base. This represents the preset penalty amplification factor. This represents terrain slope data obtained by fitting point clouds. This represents the surface friction reference coefficient found by semantic mapping. This represents a small constant pre-defined to prevent the denominator from being zero. This represents the vertical component of the terrain surface normal vector, with a penalty amplification factor set. The value is 2.0, setting a small constant. The value is 0.01. If the terrain slope data of a certain grid is at this time... The value is 0, regardless of the values ​​of other parameters. The overall calculation of the exponent term is 0, and the calculated compensation weight value is 0. If the terrain slope data of a certain grid is equal to 1, then... The tangent value is 0.2, and the surface friction reference coefficient is... The vertical component is 0.4. The value is 0.9. Substituting it into the formula, the compensation weight value corresponding to this grid is approximately 0.906, generating a terrain compensation matrix with three-dimensional terrain compensation information.

[0079] In this embodiment, the steps of inputting a color image sequence into a label-free pose estimation network for feature extraction and tracing the extraction of the pet's two-dimensional skeletal keypoint coordinate sequence specifically include: performing adaptive histogram equalization enhancement processing on the color image sequence to improve edge contrast features and obtain an enhanced color video sequence; inputting the enhanced color video sequence into the label-free pose estimation network, extracting global body features through downsampling convolutional layers, and performing spatial resolution restoration processing on the global body features through upsampling deconvolutional layers to generate a set of joint probability heatmaps; applying a non-maximum suppression algorithm to each single-channel heatmap in the set of joint probability heatmaps to find local probability peak response positions, and using the local probability peak response positions as basic two-dimensional coordinate data; importing preset quadrupedal skeleton connectivity prior constraints, and using the quadrupedal skeleton connectivity prior constraints to perform topological relationship distance verification between adjacent joints on the basic two-dimensional coordinate data; removing outlier coordinate points with abnormal distances in the basic two-dimensional coordinate data, and outputting a fully verified two-dimensional skeletal keypoint coordinate sequence.

[0080] Specifically, adaptive histogram equalization enhancement is performed on the color image sequence. This is achieved by calculating the cumulative distribution function of pixel grayscale values ​​in local image regions and remapping the grayscale values ​​to a uniform distribution, resulting in an enhanced color video sequence. A label-free pose estimation network is then constructed, employing a high-resolution network topology. This network includes a backbone feature extraction branch and three parallel multi-resolution feature fusion branches. The backbone feature extraction branch consists of multiple residual modules that extract the basic pixel features of the full-size image. The parallel branches maintain high-resolution feature maps during processing and periodically receive strong semantic information upsampled from the low-resolution branches. The network input data is set as enhanced color video frames with a size of 256 x 256 x 3 channels, and the output data is set as a set of joint probability heatmaps with a size of 64 x 64 x 17 channels. During training, the mean squared error loss function is used to compare the difference between the predicted joint probability heatmap set and the actual Gaussian distribution heatmap. An adaptive moment estimation algorithm is then used to perform gradient descent and update the results. All network layer convolutional weights are fixed after 150 rounds of forward and backward iterations and are considered trained. The enhanced color video sequence is input into the label-free pose estimation network. Global body features are extracted through downsampling convolutional layers, and spatial resolution restoration is performed on the global body features through upsampling deconvolutional layers to generate a set of joint probability heatmaps. Non-maximum suppression algorithm is applied to each single-channel heatmap in the set of joint probability heatmaps to find the local probability peak response position within a 3x3 sliding window and use it as the basic two-dimensional coordinate data. Preset quadrupedal skeleton connectivity prior constraints are imported, and the maximum reasonable topological distance between adjacent joints is set to 1.5 times the distance between the skeleton nodes. The topological relationship distance between adjacent joints is checked using the quadrupedal skeleton connectivity prior constraints. If the distance between the forelimb keypoint and the scapular keypoint exceeds the set constraint length, it is judged as an outlier and the outlier coordinates in the basic two-dimensional coordinate data are removed. The fully validated two-dimensional skeleton keypoint coordinate sequence is output.

[0081] In this embodiment, the steps of fusing depth image sequences and 2D skeleton keypoint coordinate sequences for coordinate transformation and estimating the transformed 3D pose sequence data specifically include: traversing the depth image sequence to find time-synchronized depth image frames that strictly correspond to the 2D skeleton keypoint coordinate sequence; parsing the keypoint coordinates in the 2D skeleton keypoint coordinate sequence and locating the corresponding spatial depth observation value in the found time-synchronized depth image frames; determining whether the located spatial depth observation value is an invalid hole value; if it is an invalid hole value, extracting the surrounding valid depth pixel values ​​and calculating the valid depth pixel values ​​using a dual Gaussian weighted algorithm based on keypoint gravity to estimate the corrected depth value; if it is not an invalid hole value, directly using the spatial depth observation value as the corrected depth value; combining the keypoint coordinates and the corrected depth value, performing coordinate system transformation to calculate the 3D spatial physical coordinate data of each target joint; splicing and combining the continuous 3D spatial physical coordinate data according to the time evolution order to generate 3D pose sequence data; when the spatial depth observation value is an invalid hole value, the formula for calculating the corrected depth value is: ;in, This indicates the correction depth value. This represents the total number of valid depth pixel values ​​in the surrounding area. Indicates the first One effective depth pixel value, Indicates the first The spatial pixel distance between an effective depth pixel and an invalid hole value on the image plane Indicates the first The topological gravitational distance between each effective depth pixel and its corresponding keypoint coordinates on the image plane. This represents the preset spatial Gaussian kernel standard deviation. This represents the preset standard deviation of the gravitational Gaussian kernel.

[0082] Specifically, the process involves traversing the depth image sequence by timestamp comparison to find time-synchronized depth image frames that strictly correspond to the 2D skeleton keypoint coordinate sequence. The 2D keypoint coordinates in the 2D skeleton keypoint coordinate sequence are analyzed. Within the found time-synchronized depth image frames, the corresponding spatial depth observation value is located based on the coordinate correspondence. It is then determined whether the located spatial depth observation value is an invalid hole value (displayed as 0). If it is an invalid hole value, valid depth pixel values ​​within an 8x8 neighborhood of that coordinate are extracted. A dual Gaussian weighted algorithm based on keypoint gravity is used to perform weighted summation and normalization calculations on the valid depth pixel values. When the spatial depth observation value is an invalid hole value, the formula for calculating the corrected depth value is as follows: ,in, This represents the corrected depth value obtained through double Gaussian weighting. This represents the total number of extracted surrounding valid depth pixel values. Incrementing index representing the effective depth pixels, Indicates the first The specific depth value of each effective depth pixel. Indicates the first The spatial physical pixel distance between an effective depth pixel and an invalid hole value on the image plane Indicates the first The geometric topological gravitational distance between each effective depth pixel and its corresponding keypoint coordinates on the image plane. This represents the preset spatial Gaussian kernel standard deviation. This represents the preset gravitational Gaussian kernel standard deviation, and sets the spatial Gaussian kernel standard deviation. The value is 5 pixels, and the standard deviation of the gravitational Gaussian kernel is set. The value is 10 pixels. These two sets of values ​​are obtained based on the infrared speckle hole diffusion distribution characteristics of the camera. If the spatial depth observation value is not an invalid hole value, the spatial depth observation value is used as the corrected depth value. Combined with the key point coordinates and the corrected depth value, coordinate system transformation multiplication is performed according to the camera's internal parameter matrix to calculate the three-dimensional spatial physical coordinate data of each target joint. The continuous three-dimensional spatial physical coordinate data are spliced ​​and combined according to the time evolution order to generate three-dimensional attitude sequence data.

[0083] In this embodiment, the steps for decoupling analysis by combining three-dimensional posture sequence data and terrain compensation matrix to calculate the environmentally decoupled gait feature set specifically include: segmenting the three-dimensional posture sequence data using a dynamic time warping algorithm to identify ground contact events and ground lift events, and dividing it into multiple continuous complete gait cycle sequences; calculating the original stride length, original leg alternation frequency, and original key joint bending angle within each gait cycle time period in the multiple complete gait cycle sequences, and constructing an initial gait feature state group from the calculation results; extracting all spatial grid cells covered by the multiple complete gait cycle sequences, and reading the terrain compensation weight value at the corresponding position from the terrain compensation matrix based on the coordinate index of all spatial grid cells; multiplying each index in the initial gait feature state group by the read terrain compensation weight value for attenuation calculation, and generating an environmentally decoupled gait feature set unaffected by environmental factors.

[0084] Specifically, the dynamic time warping algorithm is used to segment the 3D posture sequence data by time series distance. Based on the abrupt change of the minimum value of the lowest point of the limb end in the longitudinal depth axis, the timestamp segments of the limbs at the ground contact event and the ground lift event are identified. According to the regular alternation period of the timestamp segments, multiple continuous complete gait cycle sequences are divided. The original stride length, original leg alternation frequency, and original key joint flexion angle are measured in each gait cycle time period of the multiple complete gait cycle sequences. The original stride length, original leg alternation frequency, and original key joint flexion angle are then combined in array form. This process constructs an initial gait feature state group. It extracts all spatial grid cells covered by the three-dimensional physical location of multiple complete gait cycle sequences. Based on the two-dimensional horizontal and vertical coordinate index positions of all spatial grid cells, it reads the terrain compensation weight value of the corresponding grid position from the terrain compensation matrix. It performs attenuation operation by multiplying each index in the initial gait feature state group by the read terrain compensation weight value. It then multiplies the original stride length value with the extracted terrain compensation weight value less than 1 to obtain the true stride distance. The falsely magnified stride data is restored during the calculation process. The calculation process does not introduce polynomial parameter transformation calculations, generating an environmentally decoupled gait feature set that is not affected by environmental factors.

[0085] In this embodiment, the steps of inputting the environmentally decoupled gait feature set into a pre-established individual health baseline model for anomaly detection and evaluation to obtain gait anomaly assessment results specifically include: calling the pre-established individual health baseline model, which is constructed by an isolated forest algorithm integrating multiple decision trees and generated through unsupervised training based on long-term collected stable gait data; mapping the environmentally decoupled gait feature set into the multidimensional feature vector space of the individual health baseline model, calculating the number of segments required for the environmentally decoupled gait feature set to be isolated in each decision tree, calculating the average number of segments, and obtaining the average path length of the environmentally decoupled gait feature set in the multidimensional feature vector space; calculating the anomaly score value of the environmentally decoupled gait feature set deviating from the health baseline distribution based on the obtained average path length; comparing and analyzing the anomaly score value with the system's preset dynamic alarm sensitivity threshold, and determining that the anomaly score value continuously exceeds the dynamic alarm sensitivity threshold to reach the preset alarm count limit, then determining that the pet's gait force is asymmetrical, and generating a gait anomaly assessment result indicating the risk of early osteoarthritis.

[0086] Specifically, a pre-established individual health baseline model is invoked. This model is constructed using an isolated forest algorithm that integrates multiple decision trees. The individual health baseline model contains 100 completely randomly generated binary trees. Each binary tree consists of a root node, internal splitting nodes based on decision conditions, and leaf nodes storing the final results, all connected by directed edges. The input data is set to multidimensional unlabeled feature vectors containing multiple stride lengths and multiple joint flexion angles. The output data is set to an anomaly score that measures the degree of deviation between the current input features and the overall training data distribution. The individual health baseline is then trained. The model randomly samples 256 unlabeled samples from long-term collected gait data and assigns them to the root node of each binary tree. A feature vector dimension is randomly selected in the feature space, and a dividing point is randomly generated between the maximum and minimum values ​​of that dimension. Sample values ​​are then assigned to the left and right subtree nodes according to their magnitude. This spatial division process is recursively executed downwards until a node contains only one sample or a pre-set maximum tree depth limit is reached. This generates an unsupervised anomaly detection training model. The environment-decoupled gait feature set is mapped and input into the multidimensional feature vector of the individual health baseline model. In the spatial calculation, the number of splits required to traverse from the root node down to the isolated leaf node in each decision tree for the environmental decoupled gait feature set is calculated. The number of isolation splits recorded in 100 binary trees is accumulated and divided by 100 to obtain the average number of splits. This average number of splits yields the average path length of the environmental decoupled gait feature set in the multidimensional feature vector space. Based on this average path length, the abnormal score value of the environmental decoupled gait feature set deviating from the healthy baseline distribution is calculated using the isolated forest benchmark scoring formula. The dynamic alarm sensitivity threshold is set to 0.65, based on 30 days of pet history records. The average abnormal score is calculated by adding 1.5 times the variance distribution value. The preset alarm frequency threshold is set at 5 consecutive gait cycles. The purpose of using 5 cycles is to eliminate the influence of data misjudgment caused by the pet's single occasional jumping action by using the continuous occurrence frequency. The abnormal score value is compared with the preset dynamic alarm sensitivity threshold of 0.65. If the abnormal score value continuously exceeds the dynamic alarm sensitivity threshold of 0.65 and the occurrence frequency reaches the preset 5 alarm frequency threshold, it is determined that the pet's gait force is asymmetrical, and a gait abnormality assessment result indicating the risk of early osteoarthritis is generated.

[0087] In this embodiment, the steps of matching rehabilitation plans based on gait abnormality assessment results and generating corresponding exercise rehabilitation recommendations specifically include: parsing the generated gait abnormality assessment results and extracting the abnormality frequency index, abnormal joint specific location identification code, and motor function decline level index contained in the gait abnormality assessment results; constructing search query conditions in a preset veterinary rehabilitation intervention expert knowledge database, using the abnormal joint specific location identification code and motor function decline level index as matching primary keys to retrieve intervention rule guidance entries that fully meet the conditions; extracting the intervention parameter templates defined in the intervention rule guidance entries to generate an initial rehabilitation plan structured text that includes a daily maximum exercise duration limit, a description of recommended suitable activity terrain types, and instructions for physiotherapy stretching movements designed for the home environment; setting the rehabilitation follow-up tracking cycle for the next comprehensive gait re-examination in the initial rehabilitation plan structured text based on the abnormality frequency index; and combining the initial rehabilitation plan structured text and the rehabilitation follow-up tracking cycle to generate complete exercise rehabilitation recommendations.

[0088] Specifically, the generated gait abnormality assessment results are analyzed, and the abnormality frequency index, the specific joint identification code, and the motor function decline level index are extracted. A retrieval query command for condition matching is constructed in a pre-set veterinary rehabilitation intervention expert knowledge database containing a large amount of veterinary text data. The specific joint identification code and the motor function decline level index are combined as the matching primary key of the database query statement. Based on the extracted specific joint identification code, the query is set to the left hind limb knee joint of a dog, and the motor function decline level index corresponds to the second-level moderate impairment standard. The primary key command is input into the database to traverse and match internal forms, retrieving the data row containing the intervention rule guidance item that is completely equal to the left hind limb knee joint and the second-level condition. The intervention parameter template paragraph text defined in the intervention rule guidance item data row is extracted, and the paragraph text is combined to generate a query including the daily maximum exercise rate. The initial rehabilitation plan, consisting of a time limit, recommended suitable terrain types, and instructions for therapeutic stretching exercises designed for home environments, is structured. Based on the retrieved data, the maximum daily exercise time limit is set to 15 minutes. The recommended suitable terrain types are assigned to flat indoor areas with non-slip mats. Based on the abnormal occurrence frequency index, a rehabilitation follow-up tracking period date parameter is added to the end of the initial rehabilitation plan structured text to set the next comprehensive gait re-examination. If the extracted abnormal occurrence frequency index is recorded as more than 3 times per day, the rehabilitation follow-up tracking period is set to 14 days after the re-examination. If the abnormal occurrence frequency index is recorded as once per day, the rehabilitation follow-up tracking period is relaxed to 30 days after the re-examination. All paragraphs and rehabilitation follow-up tracking period parameters in the initial rehabilitation plan structured text are vertically spliced ​​and formatted to generate a complete exercise rehabilitation recommendation.

Claims

1. A management system for monitoring pet health, characterized in that, The system includes: Data acquisition module: Acquires multimodal video stream data of pets in their daily environment, and separates the multimodal video stream data to obtain color image sequences and depth image sequences; Semantic analysis module: Performs semantic segmentation on the color image sequence, extracts semantic features, and obtains terrain semantic feature data; Terrain mapping module: performs three-dimensional spatial mapping based on the terrain semantic feature data and the depth image sequence, and calculates and generates a terrain compensation matrix with three-dimensional terrain compensation information; Pose extraction module: Input the color image sequence into the label-free pose estimation network for feature extraction, and track and extract the two-dimensional skeletal key point coordinate sequence of the pet; 3D Reconstruction Module: Merges the depth image sequence with the 2D skeletal key point coordinate sequence, performs coordinate transformation, and estimates the transformed 3D pose sequence data; Gait analysis module: Combines the three-dimensional posture sequence data and the terrain compensation matrix to perform decoupling analysis and calculate the environmental decoupled gait feature set; Health assessment module: Input the environmentally decoupled gait feature set into a pre-established individual health baseline model for anomaly detection and assessment to obtain gait anomaly assessment results; Rehabilitation Recommendation Module: Based on the gait abnormality assessment results, a rehabilitation plan is matched, and corresponding exercise rehabilitation suggestions are generated.

2. The pet health monitoring management system according to claim 1, characterized in that, The step of separating the multimodal video stream data to obtain a color image sequence and a depth image sequence specifically includes: The multimodal video stream data collected by the sensing device is acquired and cached in a local data buffer pool; The multimodal video stream data in the local data buffer pool is parsed to extract the file header description information of the multimodal video stream data, and to read the video format identifier and hardware timestamp information. The corresponding video demultiplexer is invoked according to the video format identifier, and the video demultiplexer is used to deinterleave and separate the multimodal video stream data to obtain the original color data stream and the original depth data stream. The original color data stream is sampled using keyframes to generate the color image sequence; The original depth data stream is time-synchronized and aligned according to the hardware timestamp information. Depth data frames with absolute time deviations exceeding a preset tolerance threshold are removed, and the retained time-matched depth image frames are stitched together to generate the depth image sequence.

3. The pet health monitoring management system according to claim 1, characterized in that, The steps of performing semantic segmentation on the color image sequence, extracting semantic features, and obtaining terrain semantic feature data specifically include: Each frame of the color image sequence is converted into feature tensor data by color space transformation. The feature tensor data is input into a pre-trained mask segmentation network model to extract multi-scale spatial visual features from the feature tensor data. The pixel-level classification prediction branch in the mask segmentation network model is used to perform pixel-by-pixel forward inference calculation on the multi-scale spatial visual features to identify the ground material category label and surface obstacle category label of each pixel. According to the preset ground material attribute mapping table, the static friction coefficient associated with the ground material category label and the surface obstacle category label of each region in each frame image is found, and the found static friction coefficient is used as the surface friction reference coefficient of the corresponding pixel region. Two-dimensional planar spatial coordinates of each pixel region in each frame of the image are extracted. The surface friction reference coefficient and the two-dimensional planar spatial coordinates are then concatenated using feature dimensions to generate terrain semantic feature data that characterizes the surface properties of the environment.

4. The pet health monitoring management system according to claim 3, characterized in that, The step of performing three-dimensional spatial mapping based on the terrain semantic feature data and the depth image sequence, and calculating and generating a terrain compensation matrix with three-dimensional terrain compensation information, specifically includes: A perspective back-projection equation is constructed based on a pre-calibrated camera intrinsic parameter matrix. The perspective back-projection equation is then used to back-project the depth values ​​of pixels in the depth image sequence, converting them into a set of three-dimensional point cloud coordinates. Extract the two-dimensional planar spatial coordinates and the surface friction reference coefficient contained in the terrain semantic feature data, and attach and map the surface friction reference coefficient to the corresponding three-dimensional point position in the three-dimensional point cloud coordinate set according to the correspondence to generate a semantic three-dimensional point cloud set. The semantic 3D point cloud set is calculated using a local plane fitting reconstruction algorithm to obtain continuous terrain surface normal vectors and terrain slope data. By combining the surface friction reference coefficient, the vertical component in the terrain surface normal vector, and the terrain slope data, a nonlinear environmental disturbance penalty function based on sliding critical risk is established. The compensation weight value of each spatial grid cell is calculated through the nonlinear environmental disturbance penalty function to construct the terrain compensation matrix.

5. The pet health monitoring management system according to claim 1, characterized in that, The steps of inputting the color image sequence into a label-free pose estimation network for feature extraction, and tracing the extraction of the pet's two-dimensional skeletal keypoint coordinate sequence, specifically include: The color image sequence is subjected to adaptive histogram equalization enhancement processing to improve edge contrast features, resulting in an enhanced color video sequence. The enhanced color video sequence is input into the labelless pose estimation network. Global body features are extracted through downsampling convolutional layers, and spatial resolution restoration is performed on the global body features through upsampling deconvolutional layers to generate a set of joint probability heatmaps. For each single-channel heatmap in the set of joint probability heatmaps, a non-maximum suppression algorithm is applied to find the local probability peak response location, and the local probability peak response location is used as the basic two-dimensional coordinate data; Import the preset prior constraints on the connectivity of the quadruped skeleton, and use the prior constraints on the connectivity of the quadruped skeleton to perform topological relationship distance verification between adjacent joints on the basic two-dimensional coordinate data; Remove outlier coordinates with abnormal distances from the basic two-dimensional coordinate data, and output the coordinate sequence of the two-dimensional skeletal key points that has been fully verified.

6. The pet health monitoring management system according to claim 1, characterized in that, The step of fusing the depth image sequence and the two-dimensional skeleton keypoint coordinate sequence, performing coordinate transformation, and estimating the transformed three-dimensional pose sequence data specifically includes: The time-synchronized depth image frame that strictly corresponds to the coordinate sequence of the two-dimensional skeleton key points is traversed in the depth image sequence. The coordinates of key points in the two-dimensional skeleton key point coordinate sequence are analyzed, and the corresponding spatial depth observation values ​​are located in the found time-synchronized depth image frames. Determine whether the spatial depth observation value is an invalid hole value; if it is an invalid hole value, extract the surrounding valid depth pixel values, and use the dual Gaussian weighted algorithm based on key point gravity to calculate the valid depth pixel values ​​to estimate the corrected depth value; if it is not an invalid hole value, use the spatial depth observation value directly as the corrected depth value. By combining the key point coordinates and the corrected depth value, a coordinate system transformation is performed to calculate the three-dimensional physical coordinate data of each target joint. The three-dimensional spatial physical coordinate data are spliced ​​and combined in chronological order to generate the three-dimensional attitude sequence data.

7. The pet health monitoring management system according to claim 1, characterized in that, The steps for decoupling analysis by combining the three-dimensional pose sequence data and the terrain compensation matrix to calculate the environmental decoupled gait feature set specifically include: The three-dimensional posture sequence data is segmented using a dynamic time warping algorithm to identify ground contact events and ground lift events, and to divide the data into multiple continuous complete gait cycle sequences. Calculate the original stride length, original leg alternation frequency, and original key joint flexion angle for each gait cycle time period in the multiple complete gait cycle sequences, and construct an initial gait characteristic state group from the calculation results. Extract all spatial grid cells covered by the multiple complete gait cycle sequences, and read the terrain compensation weight value of the corresponding position from the terrain compensation matrix according to the coordinate index of all spatial grid cells; The indicators in the initial gait feature state group are multiplied by the read terrain compensation weight value for attenuation calculation to generate the environment-decoupled gait feature set that is unaffected by environmental factors.

8. The pet health monitoring management system according to claim 1, characterized in that, The steps of inputting the environmentally decoupled gait feature set into a pre-established individual health baseline model for anomaly detection and evaluation to obtain gait anomaly assessment results specifically include: The pre-established individual health baseline model is invoked. The individual health baseline model is constructed by an isolated forest algorithm integrating multiple decision trees and is generated through unsupervised training based on long-term collected stable gait data. The environmental decoupled gait feature set is mapped into the multidimensional feature vector space of the individual health baseline model. The number of segments required for the environmental decoupled gait feature set to be isolated in each decision tree is calculated. The average value of the number of segments is obtained to get the average path length of the environmental decoupled gait feature set in the multidimensional feature vector space. The abnormal score value of the environmental decoupled gait feature set deviating from the healthy baseline distribution is calculated based on the obtained average path length. The abnormal score is compared and analyzed with the system's preset dynamic alarm sensitivity threshold. If the abnormal score continuously exceeds the dynamic alarm sensitivity threshold to reach the preset alarm count limit, it is determined that the pet's gait force is asymmetrical, and the gait abnormality assessment result indicating the risk of early osteoarthritis is generated.

9. The pet health monitoring management system according to claim 1, characterized in that, The steps for matching rehabilitation plans and generating corresponding exercise rehabilitation recommendations based on the gait abnormality assessment results specifically include: The generated gait abnormality assessment results are parsed, and the abnormality frequency index, the specific location identification code of the abnormal joint, and the degree of decline in motor function index contained in the gait abnormality assessment results are extracted. Search criteria are constructed in a pre-set veterinary rehabilitation intervention expert knowledge database. Using the specific location identification code of the abnormal joint and the degree of decline in motor function as the matching primary key, intervention rule guidance entries that fully meet the criteria are retrieved. Extract the intervention parameter templates defined in the intervention rule guidelines to generate a structured text of the initial rehabilitation plan, including the daily maximum exercise duration limit, the recommended suitable terrain type description, and the physiotherapy stretching exercise instructions designed for the home environment. Based on the abnormality frequency index, the rehabilitation follow-up tracking cycle for the next comprehensive gait re-examination is set in the structured text of the initial rehabilitation plan. The structured text of the initial rehabilitation plan and the rehabilitation follow-up tracking cycle are combined and formatted to generate the complete exercise rehabilitation recommendations.

10. The management platform of the pet health monitoring management system according to any one of claims 1-9, characterized in that, It includes a server and at least one management system as described in any one of claims 1-8, wherein the management system is communicatively connected to the server.

Images