A multi-modal based intelligent pet behavior correction dog training method and system

Abstract

A kind of based on multimodal intelligent correction pet behavior training dog method, obtain the multimodal perception data of pet, the multimodal perception data include acoustic signal, motion signal and trajectory data, construct acoustic feature tensor, motion feature tensor and position feature tensor, form unified coupling tensor;The coupling tensor is input to pre-trained behavior recognition model, and behavior label is output;Emotion recognition module outputs emotion label, and adopts sliding window mode to update personality label;Behavior label, emotion label and personality label are input to intervention strategy generation model and the intervention strategy of output adaptation is included in pacification type strategy and stimulation type strategy;According to historical feedback effect record, the feedback intensity parameter of intervention strategy is adjusted, feedback convergence index and emotion recovery index are calculated, and feedback level and delay strategy of intervention strategy are dynamically revised.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pet products, in particular to a method and system for training a pet based on multi-modal intelligent correction of pet behavior. BACKGROUND

[0002] With the increasing proportion of pet-keeping in cities, pet behavior management and training have become an important issue for many pet-keeping families. Traditional dog training methods rely on manual observation and experience-based judgment, which lack scientificity and systematicness, and are prone to cause pet negative emotions or behavior deterioration due to response lag and improper training methods.

[0003] In recent years, the development of artificial intelligence and Internet of Things technology has provided new ideas for pet behavior recognition and correction. Some studies have attempted to use single modal data such as image recognition and voice analysis to identify pet emotions and analyze behavior. For example, existing technology 1 (CN201910350198.9) discloses a dog emotion recognition model training method, which mainly extracts and identifies image data and sound data independently to determine the emotional state of dogs. However, this method fails to effectively integrate the dynamic correlation between different modalities and lacks the ability to continuously model the evolution process of behavior and emotions.

[0004] In addition, existing technology 2 (CN110175526B) provides a knowledge graph-based intelligent recognition method for monitoring video dog posture and behavior, which relies on a pre-constructed dog behavior rule graph to infer behavior categories. Although it has certain scalability, this method heavily relies on artificial knowledge definition and is difficult to cover complex or unexpected behavior scenarios, and lacks end-to-end adaptability.

[0005] In summary, existing technologies have the problem of weak modal information fusion capability in pet behavior and emotion recognition, making it difficult to extract unified behavior features from multi-source data; at the same time, behavior classification relies on artificial rules or static models, lacking generalization ability and real-time performance; and lacking a personalized response mechanism, unable to dynamically optimize training strategies according to pet personality differences.

[0006] Therefore, the present application proposes a method and system for training a pet based on multi-modal intelligent correction of pet behavior, which constructs a unified multi-modal tensor structure, fuses signals such as sound, motion and trajectory for modeling, and cooperates with a pre-trained end-to-end behavior recognition and emotion perception model to improve the recognition accuracy of complex behaviors and potential emotional states. At the same time, combined with the individualized label and training feedback mechanism, a flexible and continuously optimized behavior intervention strategy is realized, effectively overcoming the limitations of existing technologies. SUMMARY

[0007] The present application aims to provide a multi-modal based intelligent pet behavior correction dog training method and system to solve the problem of not automatically detecting pet behavior correction when not needing to manually train pet dogs in the prior art.

[0008] To achieve the above object, the present application provides the following technical solutions:

[0009] A multi-modal based intelligent pet behavior correction dog training method, comprising the following steps:

[0010] Step S1, acquiring multi-modal perception data of the pet, the multi-modal perception data comprising acoustic signals, motion signals and trajectory data;

[0011] Step S2, time aligning, feature extracting and tensor modeling the multi-modal perception data, constructing acoustic feature tensors, motion feature tensors and position feature tensors, and splicing the three to form a unified coupled tensor;

[0012] Step S3, inputting the coupled tensor into a pre-trained behavior recognition model to identify the behavior type and output a behavior label;

[0013] Step S4, acquiring physiological features of the pet and extracting emotion-related parameters from the acoustic signals and motion signals and inputting them into an emotion recognition module to make the emotion recognition module output an emotion label;

[0014] Step S5, statistically modeling based on historical behavior data, training response data and emotional response data of the pet to form a pet personality label, and updating the personality label in a sliding window manner, the personality label comprising behavior stability, emotional susceptibility, stimulus sensitivity, stimulus response delay and reward acceptance preference indicators;

[0015] Step S6, inputting the behavior label, emotion label and personality label into an intervention strategy generation model and outputting an adapted intervention strategy, the intervention strategy comprising a pacification strategy and a stimulation strategy;

[0016] Step S7, adjusting feedback intensity parameters of the intervention strategy according to historical feedback records, calculating feedback convergence index and emotional recovery index, and dynamically correcting feedback levels and delay strategies of the intervention strategy;

[0017] Step S8, monitoring the response state of the pet's emotion and behavior after the intervention strategy is executed, calculating a feedback response score, and automatically switching the feedback type or enhancing the intervention intensity if the score does not meet the expectation.

[0018] Preferably, in step S2, the step of constructing the acoustic feature tensor comprises: frame division and frequency domain analysis on the original sound, a spectrum analysis unit is used for frequency domain transformation processing on each frame, a plurality of acoustic feature vectors are extracted based on mel frequency cepstral coefficients, short-time energy and spectral centroid, and the evolution of the plurality of acoustic features with time frames is combined to form an acoustic feature tensor;

[0019] The step of constructing the motion feature tensor comprises: obtaining linear acceleration information of the pet, an angular velocity measurement unit is used to obtain angular velocity information of the pet in each direction, the acceleration information and the angular velocity information are aligned and fused according to a unified time window, and the fused motion feature sequence is constructed as a motion feature tensor.

[0020] The step of constructing the position feature tensor comprises: real-time acquisition of the current coordinates of the pet, analysis of the moving track, deviation mode and space boundary state of the pet combined with a time stamp, and construction of the track sequence as a position feature tensor.

[0021] Preferably, in step S3, the training method of the behavior recognition model is: using a model with a kernel nesting mechanism, training a group of perturbation response kernels, inputting a tensor field, mapping to a high-dimensional embedding space, using a soft geodesic kernel classifier to construct a behavior probability, using cross-entropy loss combined with a tensor regularization term for training, and using the trained model for online inference.

[0022] Preferably, in step S4, the step of obtaining the emotion label by the emotion recognition module comprises: time series difference and frequency domain jitter rate analysis on the acoustic signal, extraction of frequency fluctuation rate, peak shift amplitude and high frequency burst rate; identifying continuous barking segments, calculating the average duration and maximum duration of each bark; extracting the linear acceleration direction change and rhythm fluctuation of the pet from the motion signal, calculating the pace rhythm consistency index; analyzing the high frequency vibration component of the angular velocity in a small range to determine whether there is a fear or stress reaction; combining the emotion parameters extracted from the acoustic signal and the motion signal into a group of emotion feature vectors, and outputting the corresponding emotion label according to the emotion feature vector.

[0023] Preferably, in step S5, the step of extracting feedback response data based on the long-term behavior history of the pet and extracting its personality characteristics to generate a personality label comprises: collecting the performance records of each pet in different behavior situations and marking the behavior characteristics exhibited by the pet; constructing a personality feature vector according to the indexes of barking intensity, attack tendency, feedback response delay and reward acceptance frequency in the long-term behavior record; mapping the personality feature vector and the current emotion feature vector to generate a personality label.

[0024] Preferably, in step S6, the intervention strategies include at least: sound cues, voice praise, vibration stimulation, electrical stimulation, feeding rewards, playing the owner's voice, and soothing light; each intervention strategy is equipped with parameters including: intensity level, duration, delay time window, and feedback modality; for each combination of behavior label and emotion label, an initial candidate combination of intervention strategies is generated based on preset rules; the intervention strategy generation model is a behavior-emotion joint scoring matrix, used to jointly judge the current behavior recognition result and emotion recognition result.

[0025] Preferably, in step S7, the behavioral convergence index and emotional recovery index of the intervention strategy are dynamically calculated based on the behavioral convergence speed and emotional fluctuation trend, so as to correct the feedback level and delay strategy of the intervention strategy.

[0026] Preferably, in step S8, feedback data is recorded after the intervention strategy is implemented. The feedback data includes strategy type, response duration, emotion change rate, and behavioral improvement magnitude. A feedback response scoring model is constructed based on the feedback data to evaluate the effectiveness of the intervention strategy. If the score is lower than a preset threshold, the intervention strategy is automatically adjusted.

[0027] Preferably, in step S8, when executing the intervention strategy after behavior recognition, the current behavior label, emotion label, selected intervention strategy, and short-term behavior and emotion changes after intervention are recorded; the recorded information is used to form a training sample quadruple and stored in the database; the performance dimensions of the pet under different types of behavior and emotion states are extracted from the database, and the performance dimensions include performance stability, soothing acceptance, and stimulus tolerance; each performance dimension is normalized through statistical methods to form a local personality vector; the local personality vector is used as an auxiliary input and embedded into the behavior recognition model, emotion recognition model, and intervention strategy generation model.

[0028] A multimodal intelligent dog training system for correcting pet behavior, based on the aforementioned multimodal intelligent dog training method for correcting pet behavior, includes the following modules:

[0029] The sensing and acquisition module is used to collect the pet's acoustic signals, motion signals, and location signals;

[0030] Edge recognition module, including behavior recognition model and emotion recognition model;

[0031] The intervention decision-making module includes a behavior-emotion-feedback mapping unit and an intensity regulation unit;

[0032] The execution control module is used to implement intervention strategies for pets;

[0033] The data feedback module is used to record data and continuously optimize behavior recognition, emotion recognition, and intervention strategies based on the recorded data.

[0034] Compared with existing technologies, the beneficial effects of this invention are as follows: By using multimodal perception (including acoustic signals, motion signals, and trajectory data) for tensor modeling and combining it with a deep learning behavior recognition model, specific behaviors of pets, such as excessive barking, running around, and crossing boundaries, can be identified in real time, significantly improving the accuracy and timeliness of behavior detection. It comprehensively assesses the pet's emotional state, such as anxiety, fear, and excitement, thereby avoiding blindly applying stimulation under negative emotions and preventing stress reactions. By statistically modeling the pet's historical behavior data and feedback response records, it dynamically generates "personality tags," including multiple dimensions such as emotional susceptibility, feedback delay, and reward preference, enabling each pet to be matched with the most suitable intervention plan based on its personalized profile. Unlike traditional single-stimulus training methods, this invention constructs a "stimulus-soothing" dual-channel feedback system, automatically selecting methods such as sound soothing, feeding rewards, vibration, and electrical stimulation based on the behavioral-emotional joint tags and feedback strategy matrix, achieving a non-invasive priority training principle and improving training effectiveness and pet compliance. By recording behavioral improvement and emotional recovery trends after each feedback, calculating feedback response scores, and dynamically adjusting intervention strategies, a closed-loop control logic of "behavior-feedback-response-readjustment" is formed to ensure the effectiveness and safety of long-term intervention. Attached Figure Description

[0035] Figure 1 This is a flowchart of a dog training method according to an embodiment of the present invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Please see Figure 1 This invention provides a multimodal intelligent method for correcting pet behavior, comprising the following steps:

[0038] Step S1: Acquire multimodal perception data of the pet, including acoustic signals, motion signals and trajectory data;

[0039] Step S2: Perform time alignment, feature extraction, and tensor modeling on the multimodal sensing data to construct acoustic feature tensors, motion feature tensors, and position feature tensors, and then concatenate the three to form a unified coupled tensor.

[0040] Step S3: Input the coupling tensor into the pre-trained behavior recognition model, determine the behavior type, and output the behavior label;

[0041] Step S4: Obtain the pet's physiological characteristics and extract emotion-related parameters from acoustic and motion signals, input them into the emotion recognition module, and make the emotion recognition module output emotion labels.

[0042] Step S5: Statistical modeling is performed based on the pet's historical behavior data, training response data, and emotional reaction data to form pet personality tags. The personality tags are updated using a sliding window method. The personality tags include indicators such as behavioral stability, emotional susceptibility, stimulus sensitivity, stimulus response delay, and reward acceptance preference.

[0043] Step S6: Input the behavioral tags, emotion tags, and personality tags into the intervention strategy generation model and output the appropriate intervention strategy, which includes a soothing strategy and a stimulus strategy.

[0044] Step S7: Based on historical feedback effectiveness records, adjust the feedback intensity parameters of the intervention strategy, calculate the feedback convergence index and emotional recovery index, and dynamically correct the feedback level and delay strategy of the intervention strategy.

[0045] Step S8: After the intervention strategy is implemented, monitor the pet's emotional and behavioral response status, calculate the feedback response score, and automatically switch the feedback type or increase the intervention intensity if the score does not meet expectations.

[0046] Preferably, in step S2, the step of constructing the acoustic feature tensor includes: performing frame division and frequency domain analysis on the original sound; using a spectrum analysis unit to perform frequency domain transformation processing on each frame; extracting multiple acoustic feature vectors based on Mel frequency cepstral coefficients, short-time energy, and spectral centroid; and combining the evolution of multiple acoustic features over time frames to form an acoustic feature tensor; expressed as: ,in For acoustic feature dimensions, For time frames;

[0047] The steps for constructing the motion feature tensor include: acquiring the pet's linear acceleration information; using an angular velocity measurement unit to acquire the pet's angular velocity information in various directions; aligning and fusing the acceleration and angular velocity information according to a unified time window; and constructing the fused motion feature sequence as a motion feature tensor; represented as: ,in For motion feature dimensions, This represents the time step length.

[0048] The steps for constructing the location feature tensor include: real-time acquisition of the pet's current coordinates, analysis of the pet's movement trajectory, deviation patterns, and spatial boundary states using timestamps, and constructing the trajectory sequence as a location feature tensor. Combining timestamp analysis of the pet's movement trajectory, deviation patterns, and spatial boundary states, the location tensor generation unit constructs the trajectory sequence as a location feature tensor, represented as: ,in For location feature dimension, This represents the length of the sampling time period.

[0049] The three tensors are concatenated to construct a coupled tensor field:

[0050]

[0051] It is used to comprehensively reflect the sound behavior characteristics, motion behavior characteristics and location behavior characteristics within the current time window, and serves as the input to the behavior recognition model.

[0052] After the three feature tensors are concatenated to construct a coupled tensor field, the second processor ensures that each modal feature has an effective temporal correspondence and behavioral correlation in the recognition model by maintaining the feature channel dimension and processing in time synchronization, thereby avoiding recognition failure caused by information misalignment between modalities. After the coupled tensor field is input into the pre-trained behavior recognition model, a global spatiotemporal feature representation is first extracted by an encoding network, followed by behavior state discrimination by a classification network. The recognition model employs a multi-layer nonlinear mapping structure to learn the implicit coupling relationship between cross-modal features and has the ability to quantitatively evaluate the intensity of behavioral perturbations. The recognition model output includes two result branches: one is the behavior type prediction result, indicating whether a predefined undesirable behavior type exists; the other is the perturbation amplitude score result, reflecting the degree of deviation of the current behavior from the normal state. Based on the relationship between the recognition output result and the set multi-level response thresholds, the second processor selects and activates the corresponding feedback unit in the warning module, forming a hierarchical corrective feedback mechanism. The multi-level feedback mechanism includes at least three output states: when the disturbance amplitude score is below the first threshold, only an audio prompt is triggered; when the score is between the first and second thresholds, a vibration prompt is triggered; when the score is above the second threshold, an electric shock stimulation module is triggered, realizing an upgraded behavioral intervention strategy from gentle to forceful.

[0053] In this embodiment, a family of tensor perturbation response functions is used. The responses of the trimodal feature tensor to external stimuli under different behavioral patterns are represented as follows:

[0054]

[0055] in, It is a perturbation kernel function, representing a typical behavior pattern. The response characteristics can be obtained through offline training based on empirical behavioral data, and their structure can be a family of Gaussian functions, a Laplace kernel, a radial basis function (RBF), or a tensor attention kernel; For spatial support of the tensor field, it represents a globally weighted model of information from all spatiotemporal locations within the current observation window; convolution symbol This indicates that kernel convolution or feature correlation operations are performed between the kernel function and specific modal channels or combined channels of the coupled tensor field, which can realize the extraction of perturbation-sensitive features in local regions; This indicates a predicted response to a behavior.

[0056] The family of tensor perturbation response functions Based on multimodal coupled tensor fields Based on the perturbation distribution characteristics, it is used to simulate typical behavioral patterns. The study examines the combined auditory, motor, and positional responses of pets to external stimuli within a specific time window. This family of functions is constructed using a kernel response mapping approach, essentially a kernel convolutional functional mapping mechanism capable of capturing the nonlinear interference characteristics and spatiotemporal evolution relationships between different modes in a tensor field.

[0057] In this embodiment, the second processor operates on the coupled tensor field in a non-Euclidean space structure. To perform behavior classification, the domain is a Riemannian manifold. Establish a classification function, expressed as:

[0058]

[0059] Instead of using traditional linear distance metrics for discrimination, this classification function achieves higher fitting ability and discrimination accuracy for tensor distribution features of complex behaviors by evaluating geodesic distances within the manifold structure.

[0060] Represented as:

[0061]

[0062] in, Indicates the first The tensor kernel center of a typical behavior sample can be obtained through prior sample clustering or learning mechanisms to represent the typical characteristic state of a specific behavior category (such as "quiet", "running wildly", "crossing the boundary"). For tensor With behavioral center sample In manifold The geodesic distance can be calculated using the shortest path length under the Riemannian metric, thus avoiding the projection distortion problem caused by Euclidean distance in high-dimensional curved space; The weight coefficient corresponding to each sample center represents its representativeness and confidence in the identification and judgment, and can be determined by the frequency of training data samples or supervised learning strategies. This is the Sigmoid function, used to map the weighted exponential distance to the probability of an action occurring between 0 and 1.

[0063] Preferably, in step S3, the training method of the behavior recognition model is as follows: using a model with a kernel nesting mechanism, a set of perturbation response kernels are trained, the input tensor field is mapped to a high-dimensional embedding space, a soft geodesic kernel classifier is used to construct behavior probabilities, and cross-entropy loss combined with tensor regularization term is used for training. After training, it is used for online inference.

[0064] In this embodiment, a model with a kernel nesting mechanism is used. Train a set of perturbation response kernels { }, represented as:

[0065]

[0066] Different behavioral patterns are assigned to different perturbation kernel response models.

[0067] In this embodiment, the input tensor field Mapped to a high-dimensional embedding space, it is represented as:

[0068]

[0069] The behavior probabilities are constructed using a soft geodesic kernel classifier and are represented as follows:

[0070]

[0071] In this embodiment, cross-entropy loss combined with tensor regularization is used for training, as follows:

[0072]

[0073] Where: the first term is the behavior classification loss; the second term is the perturbation energy regularization term, which is used to penalize non-stationary tensor perturbations; It is the Frobenius norm.

[0074] In step S4, the steps for the emotion recognition module to obtain emotion tags include: performing time-series differential and frequency-domain jitter analysis on the acoustic signal to extract parameters including frequency fluctuation rate, peak offset amplitude, and high-frequency burst ratio; identifying continuous barking segments and calculating the average duration and maximum duration of each bark; extracting the linear acceleration direction change and rhythm fluctuation of the pet from the motion signal and calculating the gait rhythm consistency index; analyzing the high-frequency vibration components of angular velocity within a small range to determine whether there is fear or stress response; combining the emotion parameters extracted from the acoustic signal and motion signal into a set of emotion feature vectors, and outputting the corresponding emotion tag based on the emotion feature vector.

[0075] Features such as dominant frequency, frequency fluctuation amplitude, barking duration, interval density, and high-frequency energy ratio of sound are extracted from acoustic signals; acceleration fluctuation frequency, gait balance (based on left-right acceleration symmetry), tail wagging amplitude, and tremor rhythm are extracted from motion signals; and wandering paths, movement acceleration rate, and duration of dwell time in a region are extracted from position signals. All perceptual features are uniformly mapped to a standard emotion feature space and combined to form a temporal emotion feature sequence. A standard emotion classification system is established, including at least five categories: "excitement," "anxiety," "fear," "calm," and "aggression." A labeled training sample library is constructed using supervised learning; different emotion response mapping templates are defined for different dog breeds and personalities to enhance cross-individual transferability. A Transformer temporal modeling neural network structure is used to model the temporal evolution pattern of features; the input is a multimodal emotion feature sequence, and the output is the emotion probability distribution at the current moment; a soft-labeled cross-entropy loss function and a temporal smoothing regularization term are used for joint training; if the current emotion recognition output is a negative emotion (such as anxiety / fear), electrical stimulation intervention signals are automatically suppressed and replaced with soothing speech or damping feedback.

[0076] In step S5, the steps of extracting feedback response data based on the pet's long-term behavioral history and generating personality tags include: collecting performance records for each pet in different behavioral situations and marking the behavioral characteristics exhibited; constructing a personality feature vector based on indicators such as barking intensity, aggression, feedback response delay, and reward acceptance frequency from the long-term behavioral records; and mapping the personality feature vector with the current emotional feature vector to generate personality tags (including: preferring vocal soothing over vibrational feedback, or being particularly sensitive to electrical stimulation, etc.).

[0077] In step S6, the intervention strategies include at least: sound cues, verbal praise, vibration stimulation, electrical stimulation, feeding rewards, playing the owner's voice, and soothing light. Each intervention strategy is accompanied by parameters including: intensity level, duration, delay window, and feedback modality. For each combination of behavioral and emotional labels, an initial candidate combination of intervention strategies is generated based on preset rules. The intervention strategy generation model is a behavior-emotion joint scoring matrix, used to jointly judge the current behavior recognition result and emotion recognition result. A behavior-emotion-feedback comparison matrix is ​​established based on the behavior classification results (excessive barking, aimless running, chasing behavior, etc.) and emotion labels (anxiety, excitement, fear, indifference, etc.). Different combinations are mapped to corresponding feedback types in the comparison matrix: ("excitement + boundary crossing" is mapped to low-intensity vibration; "anxiety + non-aggressive behavior" is mapped to sound soothing or delayed processing). The processing principle is a non-invasive priority principle, that is, sound cues and vibration cues are given priority under the same intervention effectiveness, while electrical stimulation has a lower priority than sound cues and vibration cues.

[0078] In step S7, the behavioral convergence index and emotional recovery index of the intervention strategy are dynamically calculated based on the behavioral convergence speed and emotional fluctuation trend, thereby correcting the feedback level and delay strategy of the intervention strategy. The stimulus intensity is dynamically adjusted based on the behavioral convergence speed and emotional fluctuation trend; the behavioral convergence index and emotional disturbance recovery index for each feedback type are recorded to set a feedback intensity correction factor; when the pet shows rapid adaptation to a certain low-intensity stimulus, the stimulus level is automatically reduced; conversely, the intensity is moderately increased until effective. If the emotional state is identified as "excessive fright" or "persistent suppression," the system selects a sound soothing mode and plays a voice clip of the owner or a soothing background sound from a preset audio database; it connects to other smart devices in the home (automatic feeders, smart toys) to trigger physical soothing feedback such as gentle stroking and releasing treats.

[0079] In step S8, feedback data is recorded after the intervention strategy is implemented. The feedback data includes strategy type, response duration, emotion change rate, and behavioral improvement magnitude. A feedback response scoring model is constructed based on the feedback data to evaluate the effectiveness of the intervention strategy. If the score is lower than a preset threshold, the intervention strategy is automatically adjusted.

[0080] In step S8, when implementing the intervention strategy after behavior recognition, the current behavior label, emotion label, selected intervention strategy, and short-term behavioral and emotional changes after intervention are recorded. The recorded information is used to form a training sample quadruple and stored in the database. Performance dimensions of the pet under different types of behavior and emotional states are extracted from the database. These performance dimensions include performance stability, reassurance acceptance, and stimulus tolerance. Each performance dimension is normalized using statistics to form a local personality vector. The local personality vector is used as an auxiliary input and embedded into the behavior recognition model, emotion recognition model, and intervention strategy generation model. Long-term behavioral observation data (such as behavior frequency, duration, and emotional response curves) are used to extract features from the pet. Extraction includes, but is not limited to, the following indicators: standard deviation of barking frequency (reflecting sensitivity to emotional fluctuations); behavioral response delay time (measuring the speed of training acceptance); and behavioral avoidance rate after a specific stimulus (assessing stimulus sensitivity). These features are combined into a pet personality feature vector and stored in an individual profile database.

[0081] In practical implementation, such as dealing with excessive barking and anxious behavior caused by the doorbell ringing, if the owner has installed the system of this invention at home and has established DouDou's personality label through preliminary data collection (the personality trait vector shows that he is emotionally sensitive, has poor tolerance to stimulation, and is highly dependent on the owner's voice).

[0082] When the doorbell rang, DouDou suddenly barked wildly and ran quickly through the living room. The system's acoustic sensors recorded its continuous, high-intensity barking, the inertial measurement unit recorded the dramatic changes in acceleration, and the position module identified its high-frequency switching movement trajectory between the doorway and the living room.

[0083] The system converts acoustic data into Mel-frequency cepstral coefficients (MFCC) tensors to extract barking frequency burst indices and intensity fluctuations; motion signals are converted into acceleration variation sequences and angular velocity data to extract gait rhythms and high-frequency tremor signals; and position data is used to generate trajectory offset tensors. These three types of tensors are time-aligned and concatenated into a coupled tensor.

[0084] The system uses a pre-trained behavior recognition model to identify "excessive barking + aimless running" and outputs the behavior label as "behavior type B3".

[0085] Emotional characteristic parameters were extracted (the average duration of barking was longer than normal, the rhythm fluctuated more, the tail wagging amplitude was small but the tremor value was high), and it was determined to be "anxiety state E2".

[0086] The user's personality tags indicate that DouDou is slow to respond to electrical stimulation but recovers from emotional fluctuations slowly, and prefers to be soothed by its owner's voice.

[0087] Based on behavior "B3" + emotion "E2" + personality label, the system selects a feedback scheme from the behavior-emotion-feedback matrix: "playing the owner's voice clip + gentle vibration prompt + delayed snack release", forming a soothing multimodal intervention combination.

[0088] Considering that DouDou had previously exhibited avoidance behavior in response to high-intensity vibrations, the system selected vibration level L1 (lowest). After playing the owner's voice to soothe DouDou for 10 seconds, if the barking stopped, a treat was released via the feeder within 5 seconds.

[0089] After the intervention was implemented, the system monitored that the sound signal decreased by 90%, the acceleration weakened, and the spatial dwell time increased. The emotion recognition module determined that the emotion had recovered to "calm E5". The system calculated the feedback response score to be 92 points (out of 100), recorded it as a positive response case and wrote it into the "feedback-response-personality" historical model.

[0090] This embodiment effectively suppresses excessive barking behavior in pets caused by external stimuli by integrating behavior recognition, emotion perception, and personality matching. It reduces emotional fluctuations using non-stimulating methods, thereby improving the sustainability of training and the pet's compliance.

[0091] This invention also provides an embodiment of a multimodal intelligent dog training system for correcting pet behavior, which, based on the aforementioned multimodal intelligent dog training method for correcting pet behavior, includes the following modules:

[0092] The sensing and acquisition module is used to collect the pet's acoustic signals, motion signals, and location signals;

[0093] Edge recognition module, including behavior recognition model and emotion recognition model;

[0094] The intervention decision-making module includes a behavior-emotion-feedback mapping unit and an intensity regulation unit;

[0095] The execution control module is used to implement intervention strategies for pets;

[0096] The data feedback module is used to record data and continuously optimize behavior recognition, emotion recognition, and intervention strategies based on the recorded data.

[0097] In this embodiment, the training process of the behavior recognition model includes: collecting sound modal data, motion modal data, and location modal data and unifying them to a time frame; constructing a trimodal tensor; using a model with a kernel nesting mechanism; training a set of perturbation response kernels; inputting the tensor field; mapping it to a high-dimensional embedding space; using a soft geodesic kernel classifier to construct behavior probabilities; using cross-entropy loss combined with tensor regularization terms for training; and using the trained model for online inference.

[0098] Multimodal perception (including acoustic signals, motion signals, and trajectory data) is used for tensor modeling, combined with a deep learning behavior recognition model to identify specific pet behaviors such as excessive barking, running around, and crossing boundaries in real time, significantly improving the accuracy and timeliness of behavior detection. A comprehensive assessment of the pet's emotional state, such as anxiety, fear, and excitement, is conducted to avoid blindly applying stimulation under negative emotions and prevent stress reactions. Statistical modeling of historical pet behavior data and feedback response records dynamically generates "personality tags," including multiple dimensions such as emotional susceptibility, feedback delay, and reward preference, allowing each pet to be matched with the most suitable intervention plan based on its personalized profile. Unlike traditional single-stimulus training methods, this invention constructs a "stimulus-soothing" dual-channel feedback system. Based on the combined behavioral and emotional tags and feedback strategy matrix, it automatically selects methods such as sound soothing, feeding rewards, vibration, and electrical stimulation, achieving a non-invasive priority training principle and improving training effectiveness and pet compliance. By recording behavioral improvement and emotional recovery trends after each feedback, calculating feedback response scores, and dynamically adjusting intervention strategies, a closed-loop control logic of "behavior-feedback-response-readjustment" is formed to ensure the effectiveness and safety of long-term intervention.

[0099] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-modal based intelligent pet behavior correction dog training method, characterized in that, The method comprises the following steps: Step S1, acquiring multi-modal sensing data of the pet, the multi-modal sensing data comprising acoustic signals, motion signals and trajectory data; Step S2, time aligning, feature extracting and tensorizing the multi-modal sensing data, constructing acoustic feature tensors, motion feature tensors and position feature tensors, and splicing the three to form a unified coupled tensor; Step S3, inputting the coupled tensor into a pre-trained behavior recognition model to identify the behavior type and output a behavior label; Step S4, acquiring physiological feature parameters of the pet, and extracting emotion-related parameters from the acoustic signals and motion signals, and inputting the emotion-related parameters into an emotion recognition module to make the emotion recognition module output an emotion label; Step S5, statistically modeling based on historical behavior data, training response data and emotional response data of the pet to form a pet personality label, and updating the personality label in a sliding window manner; Step S6, inputting the behavior label, emotion label and personality label into an intervention strategy generation model and outputting an adaptive intervention strategy, the intervention strategy comprising a calming strategy and a stimulating strategy; Step S7, adjusting feedback intensity parameters of the intervention strategy according to historical feedback records, calculating feedback convergence indexes and emotion recovery indexes, and dynamically correcting feedback levels and delay strategies of the intervention strategy; Step S8, monitoring the response state of the pet's emotion and behavior after the intervention strategy is executed, calculating a feedback response score, and automatically switching the feedback type or enhancing the intervention intensity if the score does not reach the expectation; In the step S4, the step of acquiring the emotion label by the emotion recognition module comprises: performing time series difference and frequency domain jitter rate analysis on the acoustic signals to extract frequency fluctuation rate, peak shift amplitude and high frequency burst ratio; identifying continuous barking segments, and calculating the average duration and maximum duration of each bark; extracting linear acceleration direction change and rhythm fluctuation of the pet from the motion signals, and calculating a pace rhythm consistency index; analyzing the high frequency vibration component of the angular velocity in a small range to determine whether there is a fear or stress reaction; combining the emotion parameters extracted from the acoustic signals and the motion signals into a set of emotion feature vectors, and outputting a corresponding emotion label according to the emotion feature vectors.

2. The multi-modal based intelligent pet behavior correction dog training method of claim 1, wherein: In the step S2, the step of constructing the acoustic feature tensor comprises: frame dividing and frequency domain analyzing the original sound, a frequency domain transformation unit being used for frequency domain transformation processing of each frame, a plurality of acoustic feature vectors being extracted based on mel frequency cepstral coefficients, short time energy and spectral centroid, and the plurality of acoustic features being combined to form an acoustic feature tensor along with the evolution of the time frames; The step of constructing the motion feature tensor comprises: acquiring linear acceleration information of the pet, an angular velocity measurement unit being used for acquiring angular velocity information of the pet in each direction, and the acceleration information and the angular velocity information being aligned and fused according to a unified time window, and the fused motion feature sequence being constructed as a motion feature tensor; The step of constructing the position feature tensor comprises: real-time collecting the current coordinates of the pet, analyzing the moving trajectory, deviation mode and space boundary state of the pet in combination with the time stamp, and constructing the trajectory sequence as a position feature tensor.

3. The multi-modal based intelligent pet behavior correction dog training method of claim 1, wherein: In step S3, the training method of the behavior recognition model is as follows: using a model with a kernel nesting mechanism, training a set of perturbation response kernels, inputting a tensor field, mapping to a high-dimensional embedding space, constructing a behavior probability using a soft geodesic kernel classifier, training using a cross-entropy loss combined with a tensor regularization term, and using the trained model for online inference.

4. The multi-modal based intelligent pet behavior correction dog training method of claim 1, wherein: In step S5, the feedback response data is extracted based on the long-term behavior history of the pet, and the step of extracting the personality characteristics to generate the personality label includes: collecting the performance records of each pet in different behavior situations and marking the behavior characteristics exhibited by the pet; constructing a personality characteristic vector according to the indexes of barking intensity, attack tendency, feedback response delay, and reward acceptance frequency in the long-term behavior record; and mapping the personality characteristic vector and the current emotional characteristic vector to generate the personality label.

5. The multi-modal based intelligent pet behavior correction dog training method of claim 1, wherein: In step S6, the intervention strategy includes at least: sound prompt, voice praise, vibration stimulation, electric stimulation, feeding reward, playing the owner's voice, and calming light; each intervention strategy is matched with parameters including: intensity level, duration, delay time window, and feedback mode; for each combination of behavior label and emotional label, an initial intervention strategy candidate combination is generated based on preset rules; the intervention strategy generation model is a behavior-emotion joint scoring matrix, which is used for joint judgment of the current behavior recognition result and the emotional recognition result.

6. The multi-modal based intelligent pet behavior correction dog training method of claim 5, wherein: In step S7, the behavior convergence index and the emotional recovery index of the intervention strategy are dynamically calculated according to the behavior convergence speed and the emotional fluctuation trend, so as to correct the feedback level and the delay strategy of the intervention strategy.

7. The multi-modal based intelligent pet behavior correction dog training method of claim 2, wherein: In step S8, feedback data is recorded after the intervention strategy is executed, the feedback data includes strategy type, response duration, emotional change rate, and behavior improvement amplitude; a feedback response scoring model is constructed according to the feedback data, which is used to evaluate the effect of the intervention strategy; if the score is lower than the preset threshold, the intervention strategy is automatically adjusted.

8. The multi-modal based intelligent pet behavior correction dog training method of claim 2, wherein: In step S8, after the behavior recognition, the intervention strategy is executed, the current behavior label, emotional label, selected intervention strategy, and behavior and emotional changes in a short time after the intervention are recorded; the recorded information is formed into a training sample four-tuple and stored in a database; the performance dimensions of the pet in different types of behavior and emotional states are extracted from the database, the performance dimensions include performance stability, pacification acceptance degree, and stimulation tolerance degree; each performance dimension is normalized by statistical quantity to form a local personality vector; the local personality vector is embedded into the behavior recognition model, the emotional recognition model, and the intervention strategy generation model as an auxiliary input.

9. A multi-modal based intelligent pet behavior correction training system based on the multi-modal based intelligent pet behavior correction training method of any one of claims 1-8. The following modules are included: A perception acquisition module for acquiring pet acoustic signals, motion signals, and position signals; An edge recognition module including a behavior recognition model and an emotional recognition model; An intervention decision module including a behavior-emotion-feedback mapping unit and an intensity adjustment unit; An execution control module for executing the intervention strategy on the pet; A data feedback module for recording data and continuously optimizing behavior recognition, emotional recognition, and intervention strategy based on the recorded data.

Images