Animal behavior identification method and device based on millimeter wave perception

Abstract

The invention provides an animal behavior identification method and device based on millimeter wave perception. A specific embodiment of the method comprises the following steps: acquiring a millimeter wave point cloud sequence of a target animal; and inputting the millimeter wave point cloud sequence into an animal behavior recognition model matched with a target category corresponding to the target animal to obtain an animal behavior of the target animal, the animal behavior recognition model is trained in the following mode: determining initial model parameters of the animal behavior recognition model to be trained based on model parameters of a pre-training model obtained in a pre-training stage, wherein a first sample millimeter wave point cloud sequence used in the pre-training stage comprises millimeter wave point cloud sequences of various types of animals; obtaining a second sample millimeter wave point cloud sequence and a corresponding animal behavior tag; identifying the second sample millimeter wave point cloud sequence by using a to-be-trained animal behavior identification model to obtain a predicted animal behavior; model parameters are adjusted based on differences in predicted animal behavior and animal behavior tags.

Description

Technical Field

[0001] This invention relates to the field of animal behavior research technology, and in particular to an animal behavior recognition method and device based on millimeter-wave sensing. Background Technology

[0002] Animal behavior can reflect the animal's state. Research on animal behavior helps to understand the overall state of animals and meet animal welfare requirements. Therefore, methods for identifying animal behavior have received increasing attention in recent years.

[0003] Taking domestic pets as an example, pet behavior monitoring refers to the real-time capture, analysis, and quantification of behavioral patterns in common domestic pets such as dogs, cats, and rabbits, providing objective evidence for pet health management, welfare assessment, and early warning of abnormalities. In recent years, the number of companion animals has surged, and cage / semi-cage keeping scenarios (such as airline crates, rabbit cages, and cat villas) have become the mainstream pet ownership method in urban households. At the same time, pets cannot express pain or emotions independently, and their sub-health or pathological states are often only indirectly reflected through behavioral changes. For example, dogs may repeatedly circle or excessively lick their paws when anxious; cats may exhibit prolonged sitting and abnormal gait before the onset of urinary tract diseases; and rabbits may show subtle movements such as lying on their abdomen and a sudden decrease in activity when experiencing abdominal pain. Timely detection of these abnormal behaviors is crucial for reducing sudden death rates in pets and improving animal welfare. Therefore, accurately identifying animal behavior is an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method and apparatus for animal behavior recognition based on millimeter wave sensing, in order to eliminate or improve one or more defects existing in the prior art.

[0005] According to a first aspect, an animal behavior recognition method based on millimeter-wave sensing is provided. The method includes: acquiring a millimeter-wave point cloud sequence of a target animal obtained from a millimeter-wave radar device; inputting the millimeter-wave point cloud sequence into a pre-trained animal behavior recognition model that matches the target animal's corresponding target category to obtain the animal behavior of the target animal; wherein the animal behavior recognition model is trained by: acquiring model parameters of a pre-trained model obtained in a pre-training phase, and determining initial model parameters of the animal behavior recognition model to be trained based on the model parameters of the pre-trained model; wherein the first sample millimeter-wave point cloud sequence used in the pre-training phase includes millimeter-wave point cloud sequences of multiple animal categories; acquiring a second sample millimeter-wave point cloud sequence and its corresponding animal behavior label; wherein the second sample millimeter-wave point cloud sequence includes millimeter-wave point cloud sequences of the target category animal; using the animal behavior recognition model to be trained to recognize the second sample millimeter-wave point cloud sequence to obtain predicted animal behavior; and adjusting the model parameters of the animal behavior recognition model to be trained based on the difference between the predicted animal behavior and the animal behavior label.

[0006] According to a second aspect, an animal behavior recognition device based on millimeter-wave sensing is provided, comprising: an acquisition unit configured to acquire a millimeter-wave point cloud sequence of a target animal obtained from a millimeter-wave radar device; and a recognition unit configured to input the millimeter-wave point cloud sequence into a pre-trained animal behavior recognition model that matches the target category corresponding to the target animal, thereby obtaining the animal behavior of the target animal. The animal behavior recognition model is trained by: acquiring model parameters of a pre-trained model obtained in a pre-training phase, and determining initial model parameters of the animal behavior recognition model to be trained based on the model parameters of the pre-trained model. The first sample millimeter-wave point cloud sequence used in the pre-training phase includes millimeter-wave point cloud sequences of multiple animal categories. A second sample millimeter-wave point cloud sequence and its corresponding animal behavior label are acquired, wherein the second sample millimeter-wave point cloud sequence includes millimeter-wave point cloud sequences of the target category animal. The animal behavior recognition model to be trained is used to recognize the second sample millimeter-wave point cloud sequence to obtain predicted animal behavior. The model parameters of the animal behavior recognition model to be trained are adjusted based on the difference between the predicted animal behavior and the animal behavior label.

[0007] According to a third aspect, a computer-readable storage medium is provided that stores a computer program / instructions thereon, wherein the computer program / instructions, when executed by a processor, implement the steps of the method as described in any of the first aspects.

[0008] According to a fourth aspect, a computing device is provided, including a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the computing device performs the steps of the method as described in any one of the first aspects. The animal behavior recognition method and apparatus based on millimeter-wave sensing provided in this specification can acquire millimeter-wave point cloud sequences of target animals obtained from millimeter-wave radar equipment, and input these sequences into a pre-trained animal behavior recognition model that matches the target animal's corresponding target category to obtain the target animal's behavior. The animal behavior recognition model is trained as follows: First, the model parameters of the pre-trained model obtained in the pre-training stage are acquired, and the initial model parameters of the animal behavior recognition model to be trained are determined based on these parameters. The first sample millimeter-wave point cloud sequence used in this pre-training stage includes millimeter-wave point cloud sequences of various animal categories. Next, a second sample millimeter-wave point cloud sequence and its corresponding animal behavior label are acquired. This second sample millimeter-wave point cloud sequence includes millimeter-wave point cloud sequences of the target category animals. Then, the animal behavior recognition model to be trained is used to recognize the second sample millimeter-wave point cloud sequence to obtain predicted animal behavior. Finally, based on the difference between the predicted animal behavior and the animal behavior label, the model parameters of the animal behavior recognition model to be trained are adjusted, thereby achieving the training of the animal behavior recognition model to be trained. In this embodiment, millimeter-wave radar equipment can continuously acquire millimeter-wave point cloud sequences of target animals without their awareness. These sequence can contain subtle animal behaviors, providing continuous and reliable data support for subsequent behavior recognition. Furthermore, a pre-training phase allows for the creation of cross-animal category pre-trained models that learn general behavioral dynamics representations across multiple animal categories. Based on these pre-trained models, a small number of labeled samples can be used to specifically train behavior recognition models for various animal categories, thereby making the output of these models more targeted and accurate.

[0009] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0010] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0011] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0012] Figure 1 A flowchart of an animal behavior recognition method based on millimeter-wave sensing according to one embodiment is shown; Figure 2 A schematic diagram illustrating an example of establishing the association between millimeter-wave point cloud sequences and the behavior of animals in cages; Figure 3 This diagram illustrates an example of model training that includes a pre-training phase and a fine-tuning phase. Figure 4 A schematic diagram is shown illustrating one application scenario in which the embodiments of this specification can be applied; Figure 5 A schematic block diagram of an animal behavior recognition device based on millimeter-wave sensing according to one embodiment is shown. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0014] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0015] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0016] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0017] It is understood that the ordinal numbers such as "first" and "second" mentioned in this specification are only used to distinguish multiple objects of the same or different categories (such as components, steps, parameters, etc.), and do not indicate the priority, importance or order relationship between objects, nor do they constitute a limitation on the technical features.

[0018] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0019] As mentioned earlier, timely detection of abnormal animal behavior is crucial for reducing sudden animal mortality and improving animal welfare. Therefore, accurately identifying animal behavior is an urgent problem to be solved.

[0020] Currently, mainstream animal behavior recognition methods mainly include wearable device-based solutions and optical device-based solutions. Wearable device-based solutions require animals to wear collars, harnesses, or implanted tags. Built-in inertial measurement units (IMUs) acquire information such as acceleration and angular velocity to infer behavior. However, this approach suffers from problems such as discomfort, slippage, hair entanglement, and skin abrasion. It also requires frequent charging or battery replacements, increasing the burden on owners and easily causing stress to animals, hindering its widespread use. Optical device-based solutions typically rely on visible light or infrared cameras to continuously photograph animals (e.g., caged pets), analyzing behavior using images or video streams. However, optical sensors inevitably capture images of owners and the home environment, posing a privacy risk. Furthermore, the accuracy of recognition fluctuates greatly due to factors such as changes in lighting, cage obstruction, and reflections from pet fur, making stable operation under all-weather conditions difficult.

[0021] Because animal activity can disrupt wireless signals, emerging wireless sensing technologies such as millimeter waves have been considered ideal for "zero-wearing" monitoring in recent years due to their non-contact, low-power, and non-metallic material-penetrating characteristics. Therefore, researchers have begun exploring animal behavior recognition solutions based on wireless sensing technology. However, most existing wireless sensing methods are designed for large-scale human activities indoors. Some animals are small-scale, and environments are complex, making behavior recognition in everyday scenarios highly susceptible to confusion. For example, there is a lack of specific design for the small-scale behaviors of caged pets (such as whisker twitching, abdominal movements, and ear turning), and the complex multipath interference from cage mesh, food bowls, and water bottles is not adequately considered. This makes it difficult to accurately map signal characteristics to behavioral patterns, failing to truly meet the needs of pet behavior recognition in caged settings.

[0022] Therefore, embodiments of this specification provide a method for animal behavior recognition based on millimeter-wave sensing. For example... Figure 1 As shown, Figure 1 A flowchart of an animal behavior recognition method based on millimeter-wave sensing according to one embodiment is shown. It will be understood that this method can be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities. Figure 1 As shown, this animal behavior recognition method based on millimeter-wave sensing may include the following steps 101-102, specifically: Step 101: Obtain the millimeter-wave point cloud sequence of the target animal based on the millimeter-wave radar equipment.

[0023] In this embodiment, a millimeter-wave radar device can be used to acquire a millimeter-wave point cloud sequence containing a preset space of the target animal. For example, the millimeter-wave radar device may include an FMCW (Frequency Modulated Continuous Wave) radar array. Taking a caged animal (e.g., a small pet) as an example, the FMCW radar device can be activated to continuously emit a frequency-modulated continuous wave. This beam needs to achieve complete coverage of the entire cage space to ensure there are no blind spots for observation of the animal. The radar receiver captures in real time the changes in reflected signals caused by the animal's limb movements, trunk posture changes, and subtle movements of heartbeat and respiration. This reflected signal is then processed in various ways, such as signal denoising, coordinate calibration, and micro-velocity calculation, ultimately generating a millimeter-wave point cloud sequence of the caged animal. This millimeter-wave point cloud sequence can describe the three-dimensional coordinates (x, y, z) and velocity (v) information of the most active parts of the animal, reducing data redundancy and focusing on key behavioral characteristics.

[0024] In some examples, the target animal can be an animal kept in a cage. Based on this, step 101 above can specifically include the following steps (1) and (2): Step (1): Obtain the raw signal sequence containing information about the target animal using a millimeter-wave radar device.

[0025] In this example, a raw signal sequence containing information about the target animal can be acquired using a millimeter-wave radar device. This raw signal sequence can also contain information about the cage, and if there are other items inside the cage (e.g., food bowls, water bottles, etc.), information about those items can also be included.

[0026] Step (2): Based on the pre-stored prior information of the cage structure, the millimeter-wave point cloud sequence of the target animal is identified from the original signal sequence.

[0027] In this example, prior information about the cage structure can be acquired and stored in advance. This prior information may include the location information of the cage and its internal facilities (e.g., food bowls, water bottles, etc.). Based on the pre-stored prior information, the millimeter-wave point cloud sequence of the target animal can be identified from the original signal sequence. For example, based on the location information of the cage and its internal facilities, the information of the cage and its internal facilities can be removed from the original signal sequence to obtain the millimeter-wave point cloud sequence of the target animal. The point cloud in this sequence may include three-dimensional coordinates and micro-velocity information.

[0028] For example, preset signal processing can be performed on the echo from a millimeter-wave radar device to generate a point cloud at the target animal scale. Specifically, the preset signal processing may include the following steps 1)-5). Step 1) Perform range-FFT (Range Fast Fourier Transform) on the original chirp signal (linear frequency modulated signal) sequence acquired by the millimeter-wave radar equipment to obtain the range spectrum.

[0029] Step 2) employs an improved static clutter suppression algorithm to eliminate strong static reflections from the cage (e.g., metal mesh) and items within the cage (e.g., food bowls, water bottles, etc.) based on pre-stored prior information about the cage structure. Here, the improved static clutter suppression algorithm can refer to information masking based on prior information about the cage geometry. Since the cage geometry is known (e.g., length × width × height, fixed position of the cage mesh), the spatial coordinate range of static items such as cage walls, metal rods, food bowls, and water bottles can be pre-calibrated. Then, based on the calibration, an "unusable area mask" is established in the range-azimuth-elevation coordinate system, and the coordinates of the reflection point are automatically identified for each frame of point cloud signal. If the point is located in the static area marked by the geometric template (e.g., cage wall or food bowl), it is directly attenuated or set to zero.

[0030] Step 3) Apply the Capon beamforming algorithm to form the range-azimuth matrix. Here, Capon beamforming is also called Minimum Variance Distortionless Response (MVDR) Beamforming.

[0031] Step 4) Noise points are filtered out using a dual threshold of CFAR (Constant False Alarm Rate) and voxel density.

[0032] Step 5) Introduces a super-resolution MUSIC (Multiple Signal Classification) algorithm to estimate the pitch angle on each range-azimuth cell, and simultaneously performs a Doppler-FFT (Doppler Fast Fourier Transform) to extract millimeter-level radial velocity. The result is a point cloud carrying both three-dimensional coordinates and micro-velocity information, with a density and accuracy sufficient to describe sub-centimeter-level movements such as ear tip twitching and beard swaying.

[0033] In this example, taking an animal in a cage as an example, the subtle movements of the animal's limbs, torso, and even breathing and heartbeat disturb the millimeter-wave signal. After being collected by the front-end FMCW radar array, millimeter-wave point clouds and micro-Doppler spectra are generated through multipath suppression filtering and spatiotemporal joint beamforming. Addressing the issues of strong static reflections caused by the cage's metal mesh, food bowl, water bottle, etc., and the sparse point cloud resulting from the animal's small size and low range of motion, this invention first incorporates the cage's geometry as a priori into channel modeling. At the signal level, specular reflection components unrelated to the animal are eliminated, thereby obtaining an accurate point cloud at the target animal scale.

[0034] Step 102: Input the millimeter-wave point cloud sequence into a pre-trained animal behavior recognition model that matches the target category corresponding to the target animal to obtain the animal behavior of the target animal.

[0035] In this embodiment, for different categories of animals, such as large dogs, small dogs, cats, rabbits, cows, sheep, etc., corresponding animal behavior recognition models can be pre-trained. Based on this, the millimeter-wave point cloud sequence of the target animal can be input into the pre-trained animal behavior recognition model that matches the target category of the target animal, and the animal behavior recognition model will output the animal behavior of the target animal.

[0036] In this embodiment, the animal behavior recognition model for the target category can be trained through the following steps 1021-1024, specifically: Step 1021: Obtain the model parameters of the pre-trained model obtained in the pre-training stage, and determine the initial model parameters of the animal behavior recognition model to be trained based on the model parameters of the pre-trained model.

[0037] In this embodiment, a self-supervised learning approach can be used for pre-training to obtain a pre-trained model. Pre-training involves training a model on a large, general dataset (in this example, a large number of millimeter-wave point cloud sequences of various animal categories) using a pre-defined, self-supervised learning task (e.g., predicting the next frame of millimeter-wave point cloud, reconstructing occluded millimeter-wave point cloud sequences, etc.). This process allows the model to learn "general knowledge" or "basic representations" about the dataset. In this example, the first sample millimeter-wave point cloud sequence used in the pre-training stage can include millimeter-wave point cloud sequences of various animal categories. Here, the model parameters of the pre-trained model obtained in the pre-training stage can be obtained, and the initial model parameters of the animal behavior recognition model to be trained can be determined based on the model parameters of the pre-trained model. For example, some network layers of the animal behavior recognition model to be trained can be the same as some network layers of the pre-trained model. Based on this, some model parameters of the pre-trained model can be used as some initial model parameters of the animal behavior recognition model to be trained. It is understandable that pre-trained models can be various machine learning models, such as PointNet / PointNet++, PointTransformer, etc.

[0038] During the pre-training phase, unlabeled animal point cloud sequences can be fed into a self-supervised neural network to establish the association between "animal behavior in cages and millimeter-wave features". Practical research has found that animal behavior exhibits both "intra-activity continuity" and "inter-activity continuity". The core of pre-training lies in analyzing the "intra-activity continuity" and "inter-activity continuity" of animal behavior, enabling the neural network to implicitly learn animal kinematics.

[0039] Please continue reading Figure 2 , Figure 2 A schematic diagram illustrating an example of establishing a correlation between millimeter-wave point cloud sequences and the behavior of animals in cages is shown. Figure 2 The example shown uses a pet in a cage to illustrate the concepts of "continuity within activities" and "continuity between activities" in animal behavior.

[0040] Intra-activity continuity can refer to the fact that the same action is composed of consecutive point cloud frames, with smooth and coherent spatial and temporal dimensions. For example... Figure 1 As shown in the upper part, taking the animal's licking behavior as an example, it can be naturally inferred from the licking action of the paw in the i-th frame that the tongue will continue to extend forward in the i+1-th frame. Even if we jump to the i+9-th frame, we can still predict the trajectory of the tongue and the relative position of the paw, simply because the licking behavior evolves periodically under the constraints of local joints.

[0041] Activity continuity can refer to the fact that when a pet switches from one behavior to another, it still adheres to skeletal-muscle dynamics and the physical boundaries of its cage. For example... Figure 1As shown in the lower part, the squatting urination posture in frame j does not instantly jump to the jumping escape in frame j+2. Instead, a transition occurs first in frame j+1, with the hind legs extended and the center of gravity shifting forward. Subsequently, from frame j+1 to frame j+31, the pet experiences a complete cycle of standing, looking around, and squatting again. This process conforms to the alarm-excretion-burying habitual chain of felines.

[0042] By simultaneously constraining the two types of continuity mentioned above, the self-supervised neural network in the pre-training stage is forced to learn the spatiotemporal coherence shaped by the pet's joint angles, fur occlusion, and cage wall reflections when reconstructing randomly masked frame sequences. This allows for the establishment of a stable mapping between caged pet behavior and millimeter-wave point clouds under label-free conditions. It can be understood that, based on cross-species self-supervised neural networks, pre-trained models can encode general behavioral representations of animals of different breeds and fur densities under label-free conditions. Ultimately, this enables the continuous recognition of various typical behaviors such as eating, drinking, grooming, excretion, anxious circling, dozing, jumping, and abdominal pain with the ground in a fine-tuning task with a very small number of labeled samples.

[0043] Step 1022: Obtain the second sample millimeter-wave point cloud sequence and its corresponding animal behavior tags.

[0044] In this embodiment, during the fine-tuning stage, a second sample millimeter-wave point cloud sequence and its corresponding animal behavior label can be obtained. This second sample millimeter-wave point cloud sequence may include millimeter-wave point cloud sequences of animals of the target category, for use in training animal behavior recognition for animals of the target category. It is understood that the sample set used to train the animal behavior recognition model may include multiple second sample millimeter-wave point cloud sequences; here, only one second sample millimeter-wave point cloud sequence is used for illustrative description.

[0045] Step 1023: Use the animal behavior recognition model to be trained to identify the second sample millimeter-wave point cloud sequence to obtain the predicted animal behavior.

[0046] In this embodiment, the second sample millimeter-wave point cloud sequence can be input into the animal behavior recognition model to be trained, and the animal behavior recognition model to be trained can output the predicted animal behavior.

[0047] Step 1024: Adjust the model parameters of the animal behavior recognition model to be trained based on the difference between the predicted animal behavior and the animal behavior label.

[0048] In this embodiment, a preset loss function can be used to calculate the difference loss between the predicted animal behavior output by the animal behavior recognition model to be trained for the second sample millimeter-wave point cloud sequence and the animal behavior label corresponding to the second sample millimeter-wave point cloud sequence. The model parameters of the animal behavior recognition model to be trained are adjusted with the goal of minimizing the difference loss, thereby achieving the training of the animal behavior recognition model. As an example, the preset loss function can be various loss functions, such as the cross-entropy loss function, the Kullback-Leibler Divergence loss function, etc.

[0049] In some examples, the pre-training phase may include the following steps 1)-3), specifically: Step 1) Extract the features of each millimeter-wave point cloud in the first sample millimeter-wave point cloud sequence to obtain the sample feature sequence.

[0050] In this example, for each first sample millimeter-wave point cloud in the first sample millimeter-wave point cloud sequence, the sample features corresponding to that sample millimeter-wave point cloud can be extracted, thereby obtaining the sample feature sequence corresponding to the first sample millimeter-wave point cloud sequence.

[0051] Step 2), a masked feature sequence is obtained from some sample features in the masked sample feature sequence, and a reconstructed millimeter-wave point cloud sequence is generated based on the masked feature sequence.

[0052] In this example, a subset of sample features in the sample feature sequence can be randomly masked. For instance, based on the statistical period of an animal's heartbeat-breathing rhythm, a preset proportion (e.g., 70%, 75%, 60%, etc.) of sample features can be randomly masked within each statistical period to obtain the masked feature sequence. Then, a reconstructed millimeter-wave point cloud sequence is generated based on the masked feature sequence. For example, feature encoding, feature reconstruction, and point cloud sequence reconstruction can be performed on the masked feature sequence.

[0053] Step 3) Determine the difference between the first sample millimeter-wave point cloud sequence and the reconstructed millimeter-wave point cloud sequence, and adjust the model parameters of the pre-trained model based on the determined difference.

[0054] In this example, the difference loss between the first sample millimeter-wave point cloud sequence and the reconstructed millimeter-wave point cloud sequence can be calculated according to a preset loss function. The model parameters of the pre-trained model are then adjusted with the goal of minimizing this difference loss. Here, the preset loss function can be one or more of the following: point-based loss function, distribution-based loss function, and feature-based loss function, etc.

[0055] As an example, a pre-defined joint loss function can be used to calculate the difference between the first sample millimeter-wave point cloud sequence and the reconstructed millimeter-wave point cloud sequence, wherein the joint loss function... It may include distance loss terms. and the difference loss term in the high-dimensional feature space The specific formula can be: , in, It can represent hyperparameters, which can be set according to actual needs. For example, the value can be set to a fixed value of 0.1. This value setting can simultaneously achieve the dual goals of suppressing the clustering of point cloud feature centers and accelerating the convergence speed of network training.

[0056] here, It can be determined by distance constraints. With distribution constraint terms Together they form the distance constraint term. Using optimal transportation cost as a metric, the geometric displacement difference between the first sample millimeter-wave point cloud (also known as the original point cloud) and the reconstructed millimeter-wave point cloud is calculated to ensure the geometric accuracy of the reconstructed millimeter-wave point cloud. This can include various distances, such as Wasserstein distance, Euclidean distance, Manhattan distance, etc. Distribution constraint term. This introduces an entropy regularization term, which penalizes the unreasonable distribution of sparse animal limb point clouds that excessively pulls them towards the center. This corrects the red clusters that appear during the reconstruction process, replacing them with a green distribution that more closely resembles the animal's true body outline. For example, , Among them, distance constraint term It can be used to measure the geometric positional difference between the first sample millimeter-wave point cloud and the reconstructed millimeter-wave point cloud, and is defined as: , in, ∈R3 can represent the three-dimensional coordinates of the i-th point in the first sample millimeter-wave point cloud; , can represent the three-dimensional coordinates of the j-th point in the reconstructed millimeter-wave point cloud; N can represent the number of points in the first sample millimeter-wave point cloud; R3 can represent the Euclidean distance operator, used to measure geometric distance in space; R3 can represent three-dimensional real space.

[0057] Among them, the distribution constraint term It can be used to constrain the spatial distribution characteristics of reconstructed point clouds, preventing center collapse during the reconstruction process. It is defined as: , in, It can represent the probability density of the original point cloud in the kth voxel (or spatial region); V can represent the probability density of the reconstructed point cloud in the kth voxel; V can represent the total number of spatial voxels; α can represent the weighting coefficient of the distribution regularization term (generally taken as 0.01 to 0.2).

[0058] High-dimensional feature space difference loss term It can be defined as the mean square error of the global features: , in, d can represent the global high-dimensional feature vectors of the original point cloud and the reconstructed point cloud, respectively; d can represent the feature dimension.

[0059] High-dimensional feature space difference loss term By minimizing the feature differences between the original point cloud and the reconstructed point cloud in the high-dimensional feature space, the density, shape, and curvature features of the reconstructed point cloud are further constrained. This enables the point cloud generation network and the point cloud feature extraction network to form a mutually coupled feedback mechanism during training, thereby effectively preventing the network from getting stuck in local optima and significantly improving the point cloud reconstruction accuracy of pet micro-movement behavior, providing a high-quality feature foundation for subsequent behavior recognition.

[0060] Please continue reading Figure 3 , Figure 3 This diagram illustrates an example of model training that includes a pre-training phase and a fine-tuning phase. Figure 3 In the example shown, the solid arrow "→" represents the pre-training stage, and the dashed arrow "-->" represents the fine-tuning stage. In this example, the pre-trained model may include, in sequence, a point cloud feature extraction network, a feature compression network, a masking module, an animal behavior understanding network, an activity feature reconstruction network, and a point cloud generation network. Point cloud feature extraction network can be used to extract point cloud features of each millimeter-wave point cloud in a millimeter-wave point cloud sequence; Feature compression networks can be used to process point cloud feature sequences output by point cloud feature extraction networks based on self-attention mechanisms. The masking module can be used to mask the feature sequences output by the feature compression network; Animal behavior understanding networks can be used to process masked feature sequences; Activity feature reconstruction networks can be used to reconstruct features from feature sequences processed by animal behavior understanding networks. Point cloud generation networks can be used to generate point cloud sequences based on features reconstructed from features.

[0061] In practical use, the millimeter-wave point cloud sequence corresponding to the animal can be input into the point cloud feature extraction network. In this example, the structure of the point cloud feature extraction network can include multiple layers (e.g., two or three layers) of one-dimensional convolutional layers, ReLU activation function, and batch normalization connected in sequence to achieve effective extraction of local geometric features of the animal point cloud. After the point cloud feature extraction network, a feature compression network based on a self-attention mechanism can be connected. By calculating the semantic association weights between point cloud features, the sparse animal point cloud features are globally aggregated and redundancy removed. Finally, the features are mapped to a high-dimensional embedding space through a fully connected layer, thereby transforming the sparse animal point cloud into a feature vector rich in behavioral semantics.

[0062] Subsequently, based on the statistical period of the animal's heartbeat-respiration rhythm, a masking module can be used to randomly mask the feature sequences in the high-dimensional embedding space at a preset ratio (e.g., 75%), and Gaussian noise can be used to fill the mask positions to simulate a scenario with missing feature sequences and force the pre-trained model to learn temporal continuity. Afterward, the masked feature sequences are fed into the animal behavior understanding network, enabling the network to fully learn the behavioral dynamics of the caged animal during the pre-training phase through the continuity constraints of "intra-activity" and "inter-activity." In this example, the animal behavior understanding network can employ a Transformer Encoder. For the masked feature frames, the active feature reconstruction network can recover them based on the optimal transport loss. This active feature reconstruction network can employ a lightweight Transformer Decoder structure to reduce computational complexity while ensuring reconstruction accuracy. After the activity feature reconstruction network is set up, a point cloud generation network can be set up. This point cloud generation network can include a single fully connected layer. The point cloud generation network restores the reconstructed high-dimensional features to the original point cloud coordinates and micro-velocity information, thus completing the point cloud sequence reconstruction process and obtaining the reconstructed point cloud sequence.

[0063] In practice, the behavior of caged animals (e.g., pets) is mainly divided into two categories: "micromotor behaviors," such as licking paws and breathing rhythms, and "manifest motor behaviors," such as jumping, circling, and digging in bedding. The former only causes localized millimeter-level displacement of limbs, while the latter leads to centimeter-level movement of the torso. In daily monitoring, pets spend more than 80% of their time in "micromotor behaviors," while millimeter-wave point clouds in this state are only sparsely distributed in truly active local areas, making them easily misjudged as "stillness" or "feigned sleep." Worse still, different "micromotor behaviors" often produce point cloud sequences with similar trends, making them almost indistinguishable at the point cloud level. To address this problem, embodiments of this invention propose a self-supervised strategy tailored to pet size. The self-supervised strategy involves providing a temporal point cloud and randomly masking several frames with a step size consistent with the pet's heartbeat and breathing rhythm. A self-attention mechanism allows the network to simultaneously learn the behavioral evolution and cage geometric constraints between unmasked frames within a global perspective. Subsequently, the network is required to reconstruct the complete sequence, forcing the model to not only reproduce point positions when completing the masked frames but also the spatiotemporal coherence shaped by joint angles, fur occlusion, and cage wall reflections. Through this process, the network learns the coupled dynamics of "cage-pet-behavior" without instruction, enabling it to accurately distinguish easily confused behaviors such as licking paws and abdominal pain with minimal subsequent annotation.

[0064] Meanwhile, preventing point cloud collapse during the self-supervised reconstruction phase becomes a key challenge. Traditional Chamfer Distance only considers the average point-to-point error, often pulling sparse point clouds towards the geometric center, causing the reconstructed animal to lose its original head-to-body-tail topology. This example introduces the concept of optimal transport, treating the point cloud as a quality distribution and transforming the reconstruction task into minimizing the Wasserstein Distance between two distributions. Simultaneously, an implicit topological feature space is jointly learned, allowing the network to consider both local geometric details and the overall skeletal structure during optimization, thus preventing training from getting trapped in local optima and ensuring that the final learned behavioral representation is both refined and robust.

[0065] Please continue reading Figure 3 ,based on Figure 3 The structure of the pre-trained model shown, in some examples, may include a point cloud feature extraction network, a feature compression network, an animal behavior understanding network, and a behavior classification network connected in sequence. Furthermore, the model parameters of the animal behavior recognition model to be trained may include the network parameters of the point cloud feature extraction network, feature compression network, and animal behavior understanding network of the pre-trained model.

[0066] In this example, the training of the animal behavior recognition model can be achieved during the fine-tuning stage. Here, the animal behavior recognition model to be trained can refer to an untrained or incompletely trained animal behavior recognition model. In this example, the animal behavior recognition model to be trained may include a point cloud feature extraction network, a feature compression network, an animal behavior understanding network, and a behavior classification network connected in sequence. Furthermore, the model parameters of the animal behavior recognition model to be trained may include the network parameters of the point cloud feature extraction network, feature compression network, and animal behavior understanding network of the pre-trained model. That is, the animal behavior recognition model retains the point cloud feature extraction network, feature compression network, and animal behavior understanding network from the pre-trained model, as well as the network parameters of these networks.

[0067] Specifically, the fine-tuning stage retains the point cloud feature extraction network, feature compression network, and animal behavior understanding network that have been trained in the pre-training stage, and the weight parameters of these networks can be frozen to reuse the learned general behavioral feature extraction capabilities.

[0068] During the fine-tuning phase, a small number of second-sample millimeter-wave point cloud sequences labeled with animal behavior can be input into the animal behavior model, and the feature sequences are no longer masked at this stage. The high-dimensional features output by the animal behavior understanding network are directly fed into a newly added behavior classification network. This behavior classification network can include multiple (e.g., two, three, etc.) residual blocks, fully connected layers, and activation function layers connected in sequence. Specifically, multiple (e.g., two, three, etc.) residual blocks are connected in series, followed by fully connected layers and a Softmax activation function. The fully connected layers map the high-dimensional features to the behavior category dimension, and the Softmax activation function outputs the probability distribution of each behavior, ultimately completing the recognition of animal behavior.

[0069] In some examples, the animal behavior models can identify animal behaviors that may include, but are not limited to, one or more of the following: eating, drinking, grooming, excretion, anxious circling, sleeping, jumping, abdominal pain on the ground, etc.

[0070] In some examples, adjusting the model parameters of the animal behavior recognition model to be trained in step 1024 may specifically include the following: fixing the network parameters of the point cloud feature extraction network, feature compression network, and animal behavior understanding network in the animal behavior recognition model to be trained, and adjusting the network parameters of the behavior classification network.

[0071] In this example, when adjusting the model parameters of the animal behavior recognition model to be trained, the network parameters of the point cloud feature extraction network, the feature compression network, and the animal behavior understanding network can be fixed. That is, during the fine-tuning stage, the network parameters obtained from the pre-trained model are not adjusted; only the network parameters of the behavior classification network are adjusted.

[0072] Please continue reading Figure 4 , Figure 4 This diagram illustrates an application scenario in which embodiments of this specification can be applied. For example... Figure 4 As shown, Figure 4 Taking a caged cat as an example, FMCW (Frequency Modulated Continuous Wave) radar can collect signals from the caged cat around the clock without any contact or wearing, and generate a point cloud sequence of the cat using the radar signals. Then, the point cloud sequence is input into an animal behavior recognition model that matches the cat's category. This model classifies the caged cat's behavior to obtain its animal behavior profile. In this application scenario, because the millimeter-wave radar only outputs coarse-grained point cloud coordinates describing the most active parts of the pet, rather than any visual image, the privacy intrusion during the application of this invention is far less than that of optical solutions. Furthermore, millimeter-wave signals are not sensitive to light conditions and can work stably even at night or in environments with severe cage obstruction. Therefore, this invention can capture the pet's subtle behaviors around the clock without the pet's awareness, providing continuous and reliable data support for pet health management, early disease warning, and intelligent pet product linkage control.

[0073] According to another embodiment, an animal behavior recognition device based on millimeter-wave sensing is provided. This millimeter-wave sensing-based animal behavior recognition device can be deployed in any device, platform, or cluster of devices with computing and processing capabilities.

[0074] Figure 5 A schematic block diagram of an animal behavior recognition device based on millimeter-wave sensing according to one embodiment is shown. Figure 5As shown, the millimeter-wave sensing-based animal behavior recognition device 500 may include: an acquisition unit 501 configured to acquire a millimeter-wave point cloud sequence of a target animal obtained from a millimeter-wave radar device; and a recognition unit 502 configured to input the millimeter-wave point cloud sequence into a pre-trained animal behavior recognition model that matches the target category of the target animal, to obtain the animal behavior of the target animal. The animal behavior recognition model is trained in the following manner: acquiring the model parameters of the pre-trained model obtained in the pre-training stage, and determining the initial model parameters of the animal behavior recognition model to be trained based on the model parameters of the pre-trained model. The first sample millimeter-wave point cloud sequence used in the pre-training stage includes millimeter-wave point cloud sequences of multiple animal categories; acquiring a second sample millimeter-wave point cloud sequence and its corresponding animal behavior label, wherein the second sample millimeter-wave point cloud sequence includes millimeter-wave point cloud sequences of the target category of animals; using the animal behavior recognition model to be trained to recognize the second sample millimeter-wave point cloud sequence to obtain predicted animal behavior; and adjusting the model parameters of the animal behavior recognition model to be trained based on the difference between the predicted animal behavior and the animal behavior label.

[0075] According to another embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed in a computer, causes the computer to perform... Figure 1 The method described herein. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0076] According to another embodiment, a computing device is also provided, including a processor, a memory, and a computer program / instructions stored in the memory, characterized in that the processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the computing device performs the following functions: Figure 1 The steps of the described method.

[0077] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0078] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0079] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for animal behavior recognition based on millimeter-wave sensing, characterized in that, The method includes: Obtain millimeter-wave point cloud sequences of target animals based on millimeter-wave radar equipment; The millimeter-wave point cloud sequence is input into a pre-trained animal behavior recognition model that matches the target category corresponding to the target animal to obtain the animal behavior of the target animal. The animal behavior recognition model is trained in the following manner: The model parameters of the pre-trained model obtained in the pre-training stage are obtained, and the initial model parameters of the animal behavior recognition model to be trained are determined based on the model parameters of the pre-trained model. The first sample millimeter-wave point cloud sequence used in the pre-training stage includes millimeter-wave point cloud sequences of various types of animals. Obtain a second sample millimeter-wave point cloud sequence and its corresponding animal behavior label, wherein the second sample millimeter-wave point cloud sequence includes millimeter-wave point cloud sequences of animals of the target category; The animal behavior is predicted by using the animal behavior recognition model to be trained to identify the second sample millimeter-wave point cloud sequence. Based on the difference between the predicted animal behavior and the animal behavior label, the model parameters of the animal behavior recognition model to be trained are adjusted.

2. The method according to claim 1, characterized in that, The pre-training phase includes: Extract the features of each first sample millimeter-wave point cloud from the first sample millimeter-wave point cloud sequence to obtain the sample feature sequence; A partial sample feature sequence is masked to obtain a masked feature sequence, and a reconstructed millimeter-wave point cloud sequence is generated based on the masked feature sequence. The differences between the first sample millimeter-wave point cloud sequence and the reconstructed millimeter-wave point cloud sequence are determined, and the model parameters of the pre-trained model are adjusted based on the determined differences.

3. The method according to claim 2, characterized in that, Determining the difference between the first sample millimeter-wave point cloud sequence and the reconstructed millimeter-wave point cloud sequence includes: Using a pre-defined joint loss function, the difference between the first sample millimeter-wave point cloud sequence and the reconstructed millimeter-wave point cloud sequence is calculated, wherein the joint loss function includes a distance loss term and a high-dimensional feature space difference loss term.

4. The method according to claim 1, characterized in that, The pre-trained model comprises, in sequence, a point cloud feature extraction network, a feature compression network, a masking module, an animal behavior understanding network, an activity feature reconstruction network, and a point cloud generation network; wherein... The point cloud feature extraction network is used to extract the point cloud features of each millimeter-wave point cloud in the millimeter-wave point cloud sequence; The feature compression network is used to process the point cloud feature sequence output by the point cloud feature extraction network based on a self-attention mechanism; The masking module is used to mask the feature sequence output by the feature compression network. The animal behavior understanding network is used to process the masked feature sequences; The activity feature reconstruction network is used to reconstruct features from the feature sequences processed by the animal behavior understanding network. The point cloud generation network is used to generate point cloud sequences based on the features after feature reconstruction.

5. The method according to claim 4, characterized in that, The animal behavior recognition model to be trained includes the point cloud feature extraction network, the feature compression network, the animal behavior understanding network, and the behavior classification network connected in sequence; and the model parameters of the animal behavior recognition model to be trained include the network parameters of the point cloud feature extraction network, the feature compression network, and the animal behavior understanding network of the pre-trained model.

6. The method according to claim 5, characterized in that, The adjustment of model parameters for the animal behavior recognition model to be trained includes: The network parameters of the point cloud feature extraction network, the feature compression network, and the animal behavior understanding network in the animal behavior recognition model to be trained are fixed, and the network parameters of the behavior classification network are adjusted.

7. The method according to claim 1, characterized in that, The target animal is an animal kept in a cage; and the acquisition of the millimeter-wave point cloud sequence of the target animal based on millimeter-wave radar equipment includes: The original signal sequence containing information about the target animal is acquired using a millimeter-wave radar device, wherein the original signal sequence also contains information about the cage. Based on pre-stored prior information about the cage structure, the millimeter-wave point cloud sequence of the target animal is identified from the original signal sequence, wherein the prior information about the cage structure includes the location information of the cage and the fixing facilities inside the cage.

8. The method according to claim 1, characterized in that, The target animal's animal behavior includes at least one of the following: eating, drinking, grooming, excretion, anxious circling, sleeping, jumping, and abdominal pain.

9. A computing device, comprising a processor, a memory, and computer programs / instructions stored in the memory, characterized in that, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the computing device implements the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 8.

Images