Pet full-life-cycle universal data set framework system

Abstract

The invention belongs to the technical field of pet data acquisition and analysis, and particularly relates to a pet full-life-cycle universal data set framework system which comprises an intelligent necklace, a camera, a microphone array, an intelligent system terminal and an acquisition control system integrated in the intelligent system terminal. Signal output ends of the intelligent necklace, the camera and the microphone array are connected with the intelligent system terminal; the acquisition control system comprises a data annotation system and a data analysis and processing system, and the output end of the data annotation system is connected with the data analysis and processing system; and the intelligent system terminal is used as core equipment for data acquisition and management. According to the scheme, the whole life cycle is fully covered, and comprehensive data support is provided for studying health management, behavior development and the like of the whole life cycle of the pet. Multi-modal data fusion meets the training requirements of the AI model in multi-task scenes such as behavior recognition, health monitoring and emotion recognition, and the accuracy and generalization ability of the model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of pet data acquisition and analysis, and in particular to a universal data set framework system for pets throughout their entire life cycle. Background Art

[0002] As artificial intelligence becomes increasingly used in areas such as pet health management and behavioral research, high-quality datasets have become a key foundation for AI model training. Currently, existing pet-related datasets have many limitations:

[0003] Narrow data coverage: Most data sets only focus on a specific stage of pets (such as infancy or adulthood), lacking systematic collection of data on the entire life cycle of pets from birth to aging, and cannot fully reflect the physiological and behavioral changes of pets at different growth stages.

[0004] Single modality: Existing data sets are mostly based on images or text data, and the collection of multimodal data such as sound and physiological indicators is insufficient, making it difficult to meet the training needs of AI models in multi-task scenarios such as behavior analysis and health monitoring.

[0005] Weak generalization ability: The pet samples in the dataset are mostly concentrated in specific breeds, ages or regions, and cannot effectively adapt to the diverse characteristics of pets of different breeds, ages and genders. As a result, the AI ​​models trained based on such datasets have poor accuracy and generalization ability in practical applications.

[0006] Imperfect labeling system: The labeling process of some data sets lacks standardized procedures, the accuracy and consistency of the labeling results are difficult to ensure, and there is a lack of expert verification, which affects data quality and model training effects. Summary of the Invention

[0007] The purpose of the present invention is to provide a universal data set framework system for the entire life cycle of pets to solve the problems raised in the above background technology.

[0008] To achieve the above objectives, the present invention provides the following technical solutions: a universal dataset framework system for the entire life cycle of pets, comprising:

[0009] Smart collar, camera, microphone array, smart system terminal and acquisition control system integrated in the smart system terminal;

[0010] The signal output ends of the smart collar, camera, and microphone array are connected to the smart system terminal;

[0011] The acquisition and control system includes a data marking system and a data analysis and processing system, and the output end of the data marking system is connected to the data analysis and processing system;

[0012] The intelligent system terminal serves as the core device for data collection and management, receiving data from the smart collar, camera, and microphone array. It is also used to record the daily routines of pets and process the received data.

[0013] Preferably, the smart collar includes a collar body and a heart rate sensor, an accelerometer, and a wireless transmission module integrated on the collar body. The signal output ends of the heart rate sensor and accelerometer are connected to the wireless transmission module. The wireless transmission module is externally connected to the intelligent system terminal. The heart rate sensor and accelerometer collect the pet's heart rate and activity level physiological data in real time, and transmit the data to the intelligent system terminal through the wireless transmission module.

[0014] Preferably, the wireless transmission module includes a Bluetooth or Wi-Fi module.

[0015] Preferably, the camera includes a depth camera and an ordinary surveillance camera, wherein the depth camera is deployed to utilize depth perception capabilities to capture key points of pet postures and provide data support for posture estimation tasks; the ordinary surveillance camera is installed in the pet's daily activity area to record pet behavior videos, covering daily behaviors and abnormal behavior scenarios.

[0016] Preferably, the key points of the pet's posture include the position of the head and the positions of the limbs.

[0017] Preferably, the microphone array is installed in the pet's activity space to record the sound data of the pet's calls and wheezing in different situations, including calls welcoming the owner home, calls when meeting strangers, and pathological sound samples of coughing and wheezing.

[0018] Preferably, the data annotation system includes an automatic annotation layer, a semi-automatic annotation layer, and an expert verification layer;

[0019] Automatic annotation layer: The collected data is automatically annotated using preset rules and algorithms;

[0020] Semi-automatic annotation layer: CVAT tools are used to perform behavioral annotation on the key frames of the video data in the data. Computer vision algorithms are used to perform preliminary analysis on the content of the key frames and extract possible behavioral features. The annotators then make corrections and improvements based on this to improve annotation efficiency.

[0021] Expert verification layer: Veterinarians and dog trainers are invited to review and verify the labeled data. For the highly professional health monitoring data and pathological sound samples in the data, experts will annotate and correct them based on their own professional knowledge to ensure the accuracy and reliability of the data annotation.

[0022] Preferably, the data analysis and processing system includes a data cleaning module, a data preprocessing module, and a data integration and storage system;

[0023] The data cleaning module uses the Python library Pandas to clean the data and process missing values ​​and outliers. For missing physiological data, linear interpolation or machine learning-based prediction models are used to fill in the missing data.

[0024] Data preprocessing module: Use NumPy to standardize the data. For image data, the normalization method is used to scale the pixel values ​​to the [0, 1] range. For sound data, the Librosa library is used to extract Mel-frequency cepstral coefficient features. OpenCV is used to crop, scale, and enhance image and video data to improve data quality.

[0025] Data integration and storage system: Integrate the labeled, cleaned and pre-processed data and store it in the database according to the structure of the data set framework. Use relational databases to store structured data and non-relational databases to store unstructured data such as images, videos and audio, to facilitate data query, management and call.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] Comprehensive coverage of the entire life cycle: By systematically collecting multimodal data on pets from birth to aging, we fully record the physiological and behavioral changes of pets at different growth stages, providing comprehensive data support for research on health management, behavioral development, etc. throughout the life cycle of pets.

[0028] Multimodal data fusion: Integrating multimodal information such as time series data, image / video data, and audio data can more realistically and comprehensively reflect the pet's status, meet the training needs of AI models in multi-task scenarios such as behavior recognition, health monitoring, and emotion recognition, and improve the model's accuracy and generalization ability.

[0029] Strong generalization: The dataset covers pet samples of different breeds, ages, genders, and origins, effectively adapting to the diverse characteristics of pets. AI models trained on this dataset can operate accurately in a wider range of practical application scenarios.

[0030] High-quality labeling system: A three-level labeling system is used, combining automatic labeling, semi-automatic labeling, and expert verification to ensure the accuracy and consistency of data labeling, improve the quality of the data set, and provide a reliable data foundation for AI model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a system logic block diagram of the present invention;

[0032] Figure 2 This is a system block diagram of the camera of the present invention;

[0033] Figure 3 This is a system block diagram of the smart collar of the present invention. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0035] In the description of the present invention, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside", etc., indicating directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0036] Example 1:

[0037] See also Figure 1-3 , the present invention provides a technical solution: a universal data set framework system for the entire life cycle of a pet, comprising: a smart collar, a camera, a microphone array, an intelligent system terminal, and an acquisition and control system integrated in the intelligent system terminal;

[0038] Among them, the signal output ends of the smart collar, camera, and microphone array are connected to the intelligent system terminal; the acquisition and control system includes a data labeling system and a data analysis and processing system, and the output end of the data labeling system is connected to the data analysis and processing system; the intelligent system terminal serves as the core device for data acquisition and management, receiving data from the smart collar, camera, and microphone array, and is also used to record the daily life patterns of pets and process the received data.

[0039] Example 2:

[0040] See also Figure 1-3The present invention provides a technical solution: the smart collar includes a collar body and a heart rate sensor, accelerometer, and wireless transmission module integrated into the collar body. The signal output terminals of the heart rate sensor and accelerometer are connected to the wireless transmission module, which is externally connected to an intelligent system terminal. The heart rate sensor and accelerometer collect real-time physiological data on the pet's heart rate and activity level, and transmit the data to the intelligent system terminal via the wireless transmission module. The wireless transmission module includes a Bluetooth or Wi-Fi module.

[0041] The collar itself uses smart collar devices such as FitBark and Whistle, with built-in heart rate sensors, accelerometers, etc., which can collect physiological data such as pet heart rate, breathing rate, activity level, etc. in real time, and transmit the data to the smart system terminal via Bluetooth or Wi-Fi.

[0042] Depth cameras: Deploy depth cameras such as Kinect and Intel RealSense;

[0043] Intelligent system terminal: As the core device for data collection and management, it receives data from smart collars, cameras, microphone arrays and other devices, and is also used to record the daily routines of pets, such as eating habits (feeding time, food amount), excretion management and other information.

[0044] The cameras include depth cameras and standard surveillance cameras. The depth cameras utilize depth perception to capture key pet posture points, providing data support for posture estimation. Standard surveillance cameras are installed in the pet's daily activity areas to record pet behavior videos, covering both daily and unusual behavior scenarios. Key pet posture points include head and limb positions.

[0045] The microphone array is installed in the pet's activity space to record the pet's barking and wheezing sound data in different situations, including welcoming the owner home, barking when meeting strangers, and pathological sound samples of coughing and wheezing.

[0046] Example 3:

[0047] See also Figure 1-3 ,The present invention provides a technical solution: the data annotation system includes an automatic annotation layer, a semi-automatic annotation layer, and an expert verification layer;

[0048] Automatic annotation layer: Automatically annotate time series data collected by sensors (such as heart rate, activity level, etc.) using preset rules and algorithms. For example, based on the normal physiological indicator range corresponding to the pet's age and breed, determine whether the heart rate and respiratory rate are normal and annotate them;

[0049] Semi-automatic annotation layer: Use tools such as CVAT to annotate video keyframes with behaviors. Computer vision algorithms perform preliminary analysis of video content to extract possible behavioral features, which annotators then make corrections and improvements to improve annotation efficiency.

[0050] Expert Verification: Veterinarians, dog trainers, and other professionals are invited to review and verify the labeled data. For highly specialized data, such as health monitoring data and pathological sound samples, experts label and correct them based on their expertise to ensure the accuracy and reliability of the data annotations.

[0051] The data analysis and processing system includes a data cleaning module, a data preprocessing module, and a data integration and storage system;

[0052] Data analysis and processing system

[0053] Data cleaning module: Use the Python library Pandas to clean the data and handle missing values ​​and outliers in the data. For missing physiological data, linear interpolation or machine learning-based prediction models are used to fill in the missing data. For example, for missing heart rate data for a certain period of time, the linear interpolation formula is used based on the heart rate data of the previous and next time points:

[0054]

[0055] Where (x1, y1) and (x2, y2) are known data points, x is the horizontal coordinate of the data point to be filled, and the filling value is calculated.

[0056] Data preprocessing module: Use NumPy to standardize the data. For image data, the normalization method is used to scale the pixel values ​​to the [0, 1] range. For sound data, the Librosa library is used to extract features such as Mel-Frequency Cepstral Coefficients (MFCC). The formula is:

[0057] MFCC = dct(log(S))

[0058] Where S is the Mel spectrum and DCT is the discrete cosine transform. Use OpenCV to crop, scale, and enhance image and video data to improve data quality.

[0059] Data integration and storage system: This system integrates labeled, cleaned, and preprocessed data and stores it in a database according to the dataset framework. Relational databases (such as MySQL) are used to store structured data (such as basic information and behavioral records), while non-relational databases (such as MongoDB) are used to store unstructured data such as images, video, and audio, facilitating data query, management, and access.

[0060] The hardware devices (smart collars, cameras, microphone arrays) are responsible for collecting multimodal data from the pet's entire life cycle in real time and transmitting this data to the intelligent system terminal. The intelligent system terminal uploads the received data to the software's data annotation system for annotation. The annotated data then enters the data analysis and processing system for cleaning, preprocessing, and other operations. Finally, the data integration and storage system uniformly stores and manages the processed data. Throughout this process, the hardware provides the software with the raw data source, and the software processes the data collected by the hardware. The two work together to build a complete universal dataset for the entire pet life cycle.

[0061] 1. Dataset Overview

[0062] Goal: To establish a multimodal dataset covering the entire life cycle of dogs, from birth to aging, to support the development of AI models for health monitoring, behavior analysis, emotion recognition, and more.

[0063] Applications: behavior recognition, health monitoring, pathology detection, posture estimation, sound analysis, etc.

[0064] Features:

[0065] It has strong generalizability and is suitable for different breeds, ages and genders.

[0066] Contains multimodal information such as time series data, image / video data, and audio data.

[0067] 2. Dataset framework:

[0068] A: Basic Information

[0069] Individual ID: unique identifier.

[0070] Breed: The name or category of the dog breed (divided into small, medium, and large according to body size).

[0071] Gender: Male / Female.

[0072] Age groups: Juvenile (0–1 years), Adult (1–8 years), Elderly (8 years and above).

[0073] Weight range: light (<10kg), medium (10–25kg), heavy (>25kg).

[0074] Sources: Pet stores, breeders, shelters.

[0075] Example: Using regression and classification models to generate (growth curves) of age and height change curves B: Behavior & Activity

[0076] Daily behavior sequence:

[0077] Timestamped behavioral records (such as playing, eating, resting, etc.).

[0078] Activity intensity (low, medium, high).

[0079] Environmental adaptability:

[0080] Type of residence (indoor, outdoor).

[0081] The effects of environmental changes on behavior (e.g., changes in exploratory behavior in a novel environment).

[0082] Social Interaction:

[0083] Ratings for the duration and quality of interactions with other pets or humans.

[0084] C: Posture Data

[0085] Posture Tags:

[0086] Standard postures (standing, sitting, lying down, etc.).

[0087] Special postures (such as alert state, comfortable and relaxed state, etc.).

[0088] Time series posture data:

[0089] Use the depth camera to capture pose key points (such as head and limb positions).

[0090] Supports pose estimation tasks.

[0091] D: Sound Data

[0092] Call sample:

[0093] Recordings of calls in different situations (such as welcoming the owner home, meeting strangers, etc.).

[0094] Sound feature extraction (such as frequency range, duration, loudness, etc.).

[0095] Sound event annotation:

[0096] Each audio segment is annotated with events (such as barking, whining, coughing, etc.) to facilitate the training of sound classification models.

[0097] E: Health Monitoring

[0098] General physiological indicators:

[0099] Heart rate range (normal, high, low).

[0100] Respiratory rate range (normal, slightly fast, slightly slow).

[0101] Body temperature range (normal, fever, hypothermia).

[0102] Abnormal behavior patterns:

[0103] Behaviors that deviate from normal patterns (such as excessive licking, decreased appetite, etc.) are identified by LSTM or other algorithms.

[0104] Pathological sound samples:

[0105] Contains samples of sounds such as coughing and wheezing that may indicate health problems, along with their corresponding diagnostic results (if available).

[0106] F: Excretion Habits

[0107] Excretion place: inside the cage, outdoors, etc.

[0108] Characteristics of excrement:

[0109] Color, texture (hard, soft, liquid).

[0110] Frequency of defecation (number of times per day).

[0111] G: Diet & Nutrition

[0112] Dietary preferences: food type (dry food, wet food, homemade food), brand.

[0113] Diet plan:

[0114] Feeding schedule and amount of each feeding.

[0115] Water drinking situation (daily water intake and frequency).

[0116] H: Image & Video Data

[0117] Appearance image:

[0118] Full-body photo of your pet (front, side, back).

[0119] Close-ups of characteristic parts (such as ears, eyes, nose, and tail).

[0120] Dynamic video:

[0121] Behavior recording video (such as walking, playing, eating, etc.).

[0122] Videos of abnormal behavior (such as convulsions, falls, etc.).

[0123] I: Labels and Annotations Behavior Labels: Each behavior record must be labeled (such as "normal", "abnormal").

[0124] Health label: Each health record must be labeled (such as "healthy", "potential disease").

[0125] Sound labeling: Each audio clip must be labeled (e.g., "normal barking," "coughing").

[0126] 3. Data Collection Resources The following are specific tools and resources that can be used for data collection:

[0127] A: Hardware equipment

[0128] Smart collar:

[0129] FitBark, Whistle, etc. can collect physiological data such as heart rate, respiratory rate, activity level, etc. Cameras and sensors:

[0130] Depth cameras (such as Kinect and Intel RealSense) are used to capture pose data.

[0131] Ordinary surveillance cameras are used to record behavioral videos.

[0132] Microphone Array:

[0133] Used to record sound data such as pet calls and panting.

[0134] Intelligent system terminal:

[0135] Used to record daily routines, such as eating habits (such as feeding time and food amount).

[0136] B: Labeling system

[0137] Three-level annotation system:

[0138] Automatic annotation layer: time series annotation of sensor data

[0139] Semi-automatic annotation layer: behavior annotation of video keyframes (Tool: CVAT)

[0140] Expert Verification Layer: Medical Behavior Annotations by Veterinarians / Dog Trainers

[0141] C: Software (data annotation) tools:

[0142] Labelbox, Supervisely: used to label behavior, posture, sound and other data.

[0143] Audacity: For processing and annotating audio data.

[0144] D: Data analysis tools:

[0145] Python libraries (such as Pandas and NumPy): used for data cleaning and preprocessing.

[0146] OpenCV: used to process image and video data.

[0147] Librosa: used to process audio data.

[0148] E: Public Datasets

[0149] In addition to directly collecting data, you can also combine the following public datasets:

[0150] Stanford Dogs Dataset: Provides a large amount of image data of different dog breeds.

[0151] Pet-related datasets on Kaggle: such as the "PetFinder.my Adoption Prediction" dataset, which contains basic information and behavior descriptions of pets.

[0152] YouTube video data: Behavior and sound data can be extracted from pet-related videos.

[0153] F: Partner institution

[0154] Veterinary Hospital:

[0155] Collaborate to obtain health monitoring data (such as physical examination reports and medical records).

[0156] Pet boarding center:

[0157] Obtain behavioral data of various pets in different environments.

[0158] Pet community platform:

[0159] Enrich the dataset with user-submitted pet photos, videos, and descriptions.

[0160] G: Data Collection Collaborative Resources Medical institution: Banfield Pet Hospital Network (300+ branches medical records) Research institution: AKC Canine Health Foundation Genetic Database Hardware supplier: Garmin pet tracking device SDK access community platform: Rover.com millions of pet users data authorization Four: Sample data table structure

[0161] The following is an example of a simplified data table structure for storing multimodal data:

[0162]

[0163]

[0164] By using the above framework and classification, we can construct a universal dataset covering the entire life cycle of pet dogs. This dataset not only supports tasks such as behavior recognition, health monitoring, and posture estimation, but also has the ability to generalize and be applied to pets of different breeds, ages, and genders.

[0165] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention; therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is limited by the appended claims rather than the above description. Therefore, it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any figure signs in the claims should not be regarded as limiting the claims involved.

[0166] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A universal dataset framework system for the entire life cycle of pets, characterized by: include: Smart collar, camera, microphone array, smart system terminal and acquisition control system integrated in the smart system terminal; The signal output ends of the smart collar, camera, and microphone array are connected to the smart system terminal; The acquisition and control system includes a data marking system and a data analysis and processing system, and the output end of the data marking system is connected to the data analysis and processing system; The intelligent system terminal serves as the core device for data collection and management, receiving data from the smart collar, camera, and microphone array. It is also used to record the daily routines of pets and process the received data.

2. A universal data set framework system for the entire life cycle of pets according to claim 1, characterized in that: The smart collar includes a collar body and a heart rate sensor, accelerometer, and wireless transmission module integrated on the collar body. The signal output ends of the heart rate sensor and accelerometer are connected to the wireless transmission module. The wireless transmission module is externally connected to the intelligent system terminal. The heart rate sensor and accelerometer collect the pet's heart rate and activity level physiological data in real time, and transmit the data to the intelligent system terminal through the wireless transmission module.

3. The pet life cycle universal data set framework system according to claim 2, characterized in that: The wireless transmission module includes a Bluetooth or Wi-Fi module.

4. The pet life cycle universal data set framework system according to claim 1, characterized in that: The cameras include depth cameras and ordinary surveillance cameras. The depth cameras are deployed to utilize depth perception capabilities to capture key points of pet postures and provide data support for posture estimation tasks. Ordinary surveillance cameras are installed in the pet's daily activity area to record pet behavior videos, covering daily behavior and abnormal behavior scenarios.

5. The pet life cycle universal data set framework system according to claim 4, characterized in that: The pet posture key points include the head position and the limb positions.

6. The pet life cycle universal data set framework system according to claim 1, characterized in that: The microphone array is installed in the pet's activity space to record the pet's barking and wheezing sound data in different situations, including welcoming the owner home, barking when meeting strangers, and pathological sound samples of coughing and wheezing.

7. The pet life cycle universal data set framework system according to claim 1, characterized in that: The data annotation system includes an automatic annotation layer, a semi-automatic annotation layer, and an expert verification layer; Automatic annotation layer: The collected data is automatically annotated using preset rules and algorithms; Semi-automatic annotation layer: CVAT tools are used to perform behavioral annotation on the key frames of the video data in the data. Computer vision algorithms are used to perform preliminary analysis on the content of the key frames and extract possible behavioral features. The annotators then make corrections and improvements based on this to improve annotation efficiency. Expert verification layer: Veterinarians and dog trainers are invited to review and verify the labeled data. For the highly professional health monitoring data and pathological sound samples in the data, experts will annotate and correct them based on their own professional knowledge to ensure the accuracy and reliability of the data annotation.

8. The pet life cycle universal data set framework system according to claim 1, characterized in that: The data analysis and processing system includes a data cleaning module, a data preprocessing module, and a data integration and storage system; The data cleaning module uses the Python library Pandas to clean the data and process missing values ​​and outliers. For missing physiological data, linear interpolation or machine learning-based prediction models are used to fill in the missing data. Data preprocessing module: Use NumPy to standardize the data. For image data, the normalization method is used to scale the pixel values ​​to the [0, 1] range. For sound data, the Librosa library is used to extract Mel-frequency cepstral coefficient features. OpenCV is used to crop, scale, and enhance image and video data to improve data quality. Data integration and storage system: Integrate the labeled, cleaned and pre-processed data and store it in the database according to the structure of the data set framework. Use relational databases to store structured data and non-relational databases to store unstructured data such as images, videos and audio, to facilitate data query, management and call.

Images