Pet interaction device and method of operation thereof

Abstract

The application discloses a pet interaction device and a working method thereof, which breaks the mechanical working mode of traditional devices relying on fixed timing and preset programs, can accurately judge the real needs of the pet, and combines the environmental cleanliness parameters to make autonomous decisions and seamless switching among the four modes of movement play, snack feeding, emotional pacification and environmental cleaning, thereby ensuring that each interaction is highly consistent with the physiological and psychological state of the pet at the moment, effectively avoiding pet stress or ineffective companionship caused by improper mode, and further ensuring the safety and continuity of the interaction process through the closed-loop continuous monitoring and safety interruption mechanism, thereby significantly improving the welfare level of the pet and the use experience of the owner.

Description

Technical Field

[0001] This invention relates to the field of pet supplies technology, specifically to a pet interaction device and its working method. Background Technology

[0002] Pet interaction devices are smart devices that can automatically play with, feed, soothe, and clean the environment of pets, aiming to alleviate the loneliness of pets when they are alone and reduce the care burden on owners.

[0003] As described in the published patent CN114505870B, "A Control Method, Chip, and Pet-Playing Robot for Pets," pets, as our companions, are a source of happiness and health. Interacting with pets can enrich the lives of the elderly, help young people reduce stress and improve work efficiency, and cultivate children's sense of responsibility and social skills. In conclusion, keeping pets can make our lives healthier and more fulfilling. Dogs are naturally curious and have a large range of activity; they need to run around, hunt, and play with other dogs every day to grow up healthily.

[0004] Existing pet interaction devices typically rely on timers or preset programs to perform actions such as throwing treats, projecting laser beams, or initiating cleaning at fixed times and in fixed ways. However, these devices cannot recognize or understand a pet's real-time emotions, behavioral intentions, or physical state. Their operating modes are unchanging and cannot be adaptively adjusted according to the pet's actual needs. For example, when a pet is anxious or needs rest, the device may still activate an intense play program, which not only fails to provide effective companionship but may also cause stress to the pet. Therefore, how to enable pet interaction devices to break free from mechanical preset programs and dynamically switch operating modes based on the pet's real-time state, thereby providing interaction that truly meets the pet's current needs, has become an urgent technical problem to be solved. Summary of the Invention

[0005] To overcome the shortcomings mentioned above, the present invention aims to provide a technical solution that can solve the above problems.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for operating a pet interactive device, characterized by comprising the following steps:

[0008] S100: The visual acquisition unit set on the pet interaction device captures images of the pet in real time and extracts the pet's skeletal movement data, facial expression data and identity feature data from the images.

[0009] S200, based on the skeletal motion data, facial expression data and identity feature data, a multi-task recognition model is used to determine the pet's identity, immediate emotion type and behavioral intention, and historical interaction preference parameters corresponding to the identity are retrieved;

[0010] S300: Based on the instantaneous emotion type, behavioral intention, and historical interaction preference parameters, combined with the environmental cleanliness parameters obtained by the environmental sensor, a mode selection instruction is generated.

[0011] S400, execute the mode selection instruction to put the pet interaction device into the corresponding working mode;

[0012] S500: During the execution of any working mode, continuously obtain the pet's real-time status through S100 and S200. When the change in the immediate emotion type and / or behavioral intention is detected to meet the preset mode switching conditions, interrupt the current working mode and return to S300 to redetermine and switch to the matching working mode.

[0013] Furthermore, step S100 includes the following steps:

[0014] S110 performs frame-by-frame preprocessing on the raw image stream captured by the visual acquisition unit.

[0015] S120: Input the preprocessed image into the instance segmentation model, segment out the pixel-level mask area of ​​each pet in the picture and assign instance labels, and establish a stable tracking trajectory for each individual pet by using cross-frame data association.

[0016] S130: For the mask area of ​​each pet instance in each frame, the coordinates of multiple key points are extracted using the animal skeleton key point detection network, and the joint motion velocity, acceleration and angle are calculated based on the key point coordinate sequence of consecutive frames of the same pet individual to generate skeleton motion data.

[0017] S140, based on the head key point position in the key points, cut out the head sub-region from the mask area of ​​the same pet instance, detect the contour points of the eyes, mouth and ears through the facial key point positioning model, and calculate the eye opening degree, mouth opening degree and ear posture angle, and combine them into facial expression data.

[0018] S150, The pet identity feature extraction network is used to globally encode the mask region of each pet instance to generate a high-dimensional identity feature vector as the identity feature data, which is used to determine the pet's identity in subsequent steps.

[0019] Furthermore, step S200 includes the following steps:

[0020] S210, perform similarity matching between the identity feature data and the pre-stored identity feature database to determine the pet's identity identifier. If it is a new pet, create a new identity identifier and configure default preference parameters.

[0021] S220, perform frame-level alignment and normalization processing on the skeleton motion data and facial expression data according to the time window, and fuse them to form a multimodal temporal feature tensor;

[0022] S230, the multimodal temporal feature tensor is input into a pre-trained multi-task recognition model, the model including a shared spatiotemporal coding network, an emotion classification head and an intention classification head, and outputs the pet's instantaneous emotion type and behavioral intention;

[0023] S240, based on the identity identifier, retrieve the pet's historical interaction preference parameters from the preference storage module. The historical interaction preference parameters are a multi-dimensional vector that is dynamically updated based on the pet's past interaction feedback.

[0024] Furthermore, the mode selection instruction corresponds to one of the first working mode, the second working mode, the third working mode, and the fourth working mode, wherein the first working mode corresponds to the need for exercise and play, the second working mode corresponds to the need for feeding, the third working mode corresponds to the need for emotional comfort, and the fourth working mode corresponds to the need for rest and environmental maintenance.

[0025] Furthermore, step S300 includes the following steps:

[0026] The S310 generates a quantitative cleanliness index and determines the cleanliness level through its built-in optical dust sensor and vision-based floor stain detection.

[0027] S320, based on the immediate emotion type and behavioral intention, extract the corresponding basic demand tendency weights from the preset mapping table, and use the time-related preference items in the historical interaction preference parameters to make weighted corrections to form a basic demand intensity vector, which includes the intensity of sports and play, the intensity of eating, the intensity of emotional comfort and the intensity of rest.

[0028] S330, adjust the basic requirement intensity vector according to the cleanliness level. When the cleanliness level is lower than the preset threshold, increase the rest intensity and suppress the intensity of movement and play and the intensity of eating, while marking local clean areas that can be used for interaction;

[0029] Furthermore, when the first working mode is selected, the laser projection unit is activated to project dynamic light spots according to the bionic prey movement trajectory generated based on the historical interaction preference parameters, and simultaneously plays the first audio that simulates the sound of the prey.

[0030] When the second working mode is selected, the ejector snack delivery unit is activated, the snack type and throwing direction are determined according to the historical interaction preference parameters, and the preset owner summoning voice is played.

[0031] When the third working mode is selected, a real-time audio connection is established with the owner's terminal through the communication unit, enabling the owner to have a remote conversation with the pet or play pre-stored soothing voice messages from the owner.

[0032] When the fourth working mode is selected, unnecessary interactive modules are turned off, and the automatic cleaning unit is activated to clean the floor.

[0033] Furthermore, step S400 includes the following steps:

[0034] S410, parse the mode selection instruction, extract the target working mode number and associated control parameters, the control parameters include local clean area mask, soothing volume, snack type and projection direction, etc.

[0035] When the first working mode is selected, the S420 generates a bionic prey movement trajectory based on the prey type and movement preference in the historical interaction preference parameters, limits the dynamic light spot projection to a safe and clean ground area, and plays simulated sounds that match the prey type simultaneously. During the projection process, the light spot movement speed and turning frequency are dynamically adjusted according to the pet's real-time tracking success rate and mood.

[0036] S430: When the second working mode is selected, the type and amount of snacks are selected based on historical interaction preference parameters. The snacks are launched to a safe and clean landing area in front of the pet through the ejection mechanism. Before launching, a call voice based on the owner's voiceprint is played, and the amount of food fed is accumulated. Once the daily recommended limit is exceeded, the mode is automatically disabled until the next day.

[0037] S440, when the third working mode is selected, attempts to establish a real-time audio connection with the owner's terminal to achieve remote dialogue. If the connection cannot be established, it plays pre-soothing audio that matches the current immediate mood. When it detects that the pet's mood has turned calm and it intends to rest, it automatically fades out the audio and switches to the fourth working mode.

[0038] When the fourth working mode is selected, the S450 turns off unnecessary interactive modules, plans the cleaning path and starts the automatic cleaning unit. During the cleaning process, it tracks the pet's location in real time. When the pet approaches or becomes stressed, it pauses the cleaning and guides the pet away. If the cleanliness is not up to standard after the cleaning is completed, it will perform supplementary cleaning.

[0039] Furthermore, step S500 includes the following steps:

[0040] S510, during the execution of any work mode, continuously run steps S100 and S200 to construct a temporal state stream from the output instantaneous emotion type, behavioral intention, identity identifier and confidence level, and calculate the change in emotion and intention.

[0041] S520, determine whether the preset mode switching conditions are met according to the time-series state stream. The mode switching conditions include emotion type conversion and confidence level exceeding the confirmation threshold, behavioral intention not matching the needs met by the current mode, environmental cleanliness suddenly dropping to an unsafe level, new pet entering or the current mode continuously timeout and emotion not improving, and each condition must be continuously met within the number of consecutive confirmation frames.

[0042] S530, when the mode switching conditions are met, executes a safety interruption according to the current working mode, namely, the first mode turns off the laser, the second mode cancels the unexecuted ejection, the third mode smoothly fades out the audio, and the fourth mode pauses cleaning and retracts moving parts, and saves the interaction context at the time of the interruption.

[0043] S540, return the interruption context and the latest pet status to step S300 to re-determine and generate a new mode selection instruction, thereby achieving seamless mode switching.

[0044] A pet interaction device includes an autonomous moving chassis, a cleaning module, a treat throwing module, a laser projection module, a visual acquisition module, and a voice interaction module. The interaction device operates using the method described above.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] This invention can autonomously decide and seamlessly switch between multiple working modes based on accurate judgment of the pet's actual needs and combined with environmental cleanliness parameters. This ensures that every interaction is highly appropriate to the pet's current physiological and psychological state, effectively avoiding pet stress or ineffective companionship caused by inappropriate modes. The closed-loop continuous monitoring and safety interruption mechanism further guarantee the safety and continuity of the interaction process, enabling the device to truly realize the intelligent transformation from "timed execution" to "on-demand response," significantly improving the pet's welfare and the owner's user experience. Attached Figure Description

[0047] Figure 1 This is a flowchart of the operation process of S100-S600 in this invention.

[0048] Figure 2 This is a tree diagram of the mechanical structure in one embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Please see Figure 1-2 A method for operating a pet interactive device includes the following steps:

[0051] S100: The visual acquisition unit set on the pet interaction device captures images of the pet in real time and extracts the pet's skeletal movement data, facial expression data and identity feature data from the images.

[0052] S200, based on the skeletal motion data, facial expression data and identity feature data, a multi-task recognition model is used to determine the pet's identity, immediate emotion type and behavioral intention, and historical interaction preference parameters corresponding to the identity are retrieved;

[0053] S300, based on the instantaneous emotion type, behavioral intention and historical interaction preference parameters, combined with the environmental cleanliness parameters obtained by the environmental sensor, a mode selection instruction is generated. The mode selection instruction corresponds to one of the first working mode, the second working mode, the third working mode and the fourth working mode. The first working mode corresponds to the need for exercise and play, the second working mode corresponds to the need for food intake, the third working mode corresponds to the need for emotional comfort, and the fourth working mode corresponds to the need for rest and environmental maintenance.

[0054] S400, Execute the mode selection instruction to put the pet interaction device into the corresponding working mode, wherein:

[0055] When the first working mode is selected, the laser projection unit is activated to project dynamic light spots according to the bionic prey movement trajectory generated based on the historical interaction preference parameters, and simultaneously plays the first audio that simulates the sound of prey.

[0056] When the second working mode is selected, the ejector snack delivery unit is activated, the snack type and throwing direction are determined according to the historical interaction preference parameters, and the preset owner summoning voice is played.

[0057] When the third working mode is selected, a real-time audio connection is established with the owner's terminal through the communication unit, enabling the owner to have a remote conversation with the pet or play pre-stored soothing voice messages from the owner.

[0058] When the fourth working mode is selected, unnecessary interactive modules are turned off, and the automatic cleaning unit is activated to clean the floor.

[0059] S500: During the execution of any working mode, continuously obtain the pet's real-time status through S100 and S200. When the change in the immediate emotion type and / or behavioral intention is detected to meet the preset mode switching conditions, interrupt the current working mode and return to S300 to redetermine and switch to the matching working mode.

[0060] The visual acquisition unit captures the pet's skeletal movement, facial expressions, and identity features in real time. With the help of a multi-task recognition model, it simultaneously analyzes the pet's identity, immediate emotional type, and behavioral intentions. In a single process, it achieves non-invasive and accurate perception of the pet's individual psychological state and action purpose, completely breaking through the limitations of traditional devices that can only mechanically execute according to fixed time and preset programs.

[0061] Building upon this foundation, historical interaction preference parameters and environmental cleanliness parameters are further incorporated into the decision-making process. This ensures that the generation and switching of four working modes—play, feeding, emotional comfort, and environmental cleaning—are entirely driven by the pet's actual needs, rather than being pre-set by humans. When a pet shows an intention to play, a dynamic light spot mimicking the movement trajectory of a prey is generated based on its historical preferences, and a matching simulated prey sound is played simultaneously. The difficulty of the light spot can be adaptively adjusted based on the real-time tracking success rate to ensure the continued attractiveness and sense of accomplishment during play. When a pet has a need to eat, the type of treat and the throwing direction are selected based on preferences, and a call voice based on the owner's voiceprint is played to create a natural feeling of the owner being present while feeding the pet. When a pet is anxious or needs comforting, a real-time voice connection with the owner's terminal can be automatically established or a preset comforting voice from the owner can be played to provide remote emotional support. When a pet needs to rest or the environmental cleanliness is insufficient, unnecessary interaction modules are turned off and the cleaning unit is activated to sweep the floor. During the cleaning process, the unit can track and avoid the pet in real time to prevent stress.

[0062] The closed loop formed by this method, which consists of "real-time perception - fusion understanding - dynamic decision-making - adaptive execution - continuous feedback", combined with anti-shake confirmation of state changes before mode switching and safety interruption strategies for different modes, not only ensures the timeliness and continuity of the interaction process, but also fundamentally avoids the deterioration of pets' emotions or safety hazards caused by inappropriate modes.

[0063] Thus, the pet interaction device has been truly upgraded from "passive timed execution" to "active on-demand response", significantly improving the pet's psychological well-being, physical health, and emotional satisfaction when alone, and greatly reducing the owner's daily care and mental burden.

[0064] In summary, this invention can autonomously decide and seamlessly switch between multiple working modes based on accurate judgment of the pet's actual needs and combined with environmental cleanliness parameters. This ensures that every interaction is highly aligned with the pet's current physiological and psychological state, effectively avoiding pet stress or ineffective companionship caused by inappropriate modes. The closed-loop continuous monitoring and safety interruption mechanism further guarantee the safety and continuity of the interaction process, enabling the device to truly achieve an intelligent transformation from "timed execution" to "on-demand response," significantly improving the pet's welfare and the owner's user experience.

[0065] In this embodiment of the invention, step S100 includes the following steps:

[0066] S110, perform frame-by-frame preprocessing on the raw image stream captured by the visual acquisition unit. The preprocessing includes Gaussian filtering for noise reduction and adaptive histogram equalization, and timestamp each frame to form a continuous image sequence.

[0067] S120: Input the preprocessed image into the instance segmentation model, segment out the pixel-level mask area of ​​each pet in the picture and assign instance labels, and establish a stable tracking trajectory for each individual pet by using cross-frame data association.

[0068] S130: For the mask area of ​​each pet instance in each frame, the coordinates of multiple key points are extracted using the animal skeleton key point detection network, and the joint motion velocity, acceleration and angle are calculated based on the key point coordinate sequence of consecutive frames of the same pet individual to generate skeleton motion data.

[0069] S140, based on the head key point position in the key points, cut out the head sub-region from the mask area of ​​the same pet instance, detect the contour points of the eyes, mouth and ears through the facial key point positioning model, and calculate the eye opening degree, mouth opening degree and ear posture angle, and combine them into facial expression data.

[0070] S150, The pet identity feature extraction network is used to globally encode the mask region of each pet instance to generate a high-dimensional identity feature vector as the identity feature data, which is used to determine the pet's identity in subsequent steps.

[0071] In S110, image stream preprocessing and time synchronization are performed: the visual acquisition unit acquires scene image streams containing pets at a fixed frequency, and performs Gaussian filtering for noise reduction and adaptive histogram equalization on each frame of the image in sequence to eliminate ambient lighting changes and sensor noise; at the same time, timestamps are marked on each frame to ensure the temporal consistency of multiple frames of data and form a continuous and clean image sequence.

[0072] In S120, multi-pet instance segmentation and individual tracking: The preprocessed image is input into an instance segmentation model based on a deep convolutional neural network. This model is trained to distinguish between background, human body, pets and other objects, and outputs a pixel-level mask region for each pet in the image, and assigns an instance label to each mask; Combined with the instance information of the previous frame, Kalman filtering and Hungarian algorithm are used to associate the instance labels across frames, and a stable tracking trajectory is established for each individual pet to ensure that the identity remains consistent when pets are crossed or briefly occluded;

[0073] In S130, skeletal keypoint detection and temporal motion data construction are performed as follows: For the mask region corresponding to each pet instance in each frame, a pre-trained animal skeletal keypoint detection network is used to extract the coordinates of a preset number of keypoints (e.g., 18-25), covering the limb joints, spine, tail, and head; based on the keypoint coordinate sequence of the same pet individual in multiple consecutive frames, the instantaneous motion velocity, acceleration, joint angle, and overall displacement direction of the joints are calculated, and these time-varying parameters are encapsulated into structured skeletal motion data to characterize the dynamic behavior features of the pet.

[0074] In S140, facial organ localization and expression parameterization are performed as follows: Within the mask area of ​​the same pet instance, a head sub-region is cropped based on the head keypoint position in the skeletal keypoints; within this sub-region, a facial keypoint localization model is run to accurately detect the contour points of organs such as eyes, mouth, and ears; based on these contour points, the eye opening degree (ratio of the distance between the upper and lower eyelids to the distance between the corners of the eyes), mouth opening degree (ratio of the distance between the upper and lower lips to the mouth width), and the rotation angle and orientation of the ears relative to the head are calculated; these continuously changing geometric parameters are fused with the local texture features of the facial region (extracted through lightweight feature descriptors) to form facial expression data;

[0075] In S150, global identity feature extraction and identity data output: For the mask area of ​​each pet instance, a separate pet identity feature extraction network is used for global encoding. This network is trained through metric learning and can map the overall visual features of the same pet, such as appearance, body proportion, fur color and texture, into a high-dimensional identity feature vector. This vector is directly used as the identity feature data and is continuously updated with each frame. It is then used in the subsequent step S200 to match the pet with the pre-stored identity feature library during multi-task recognition to finally determine the pet's identity.

[0076] In this embodiment of the invention, step S200 includes the following steps:

[0077] S210, perform similarity matching between the identity feature data and the pre-stored identity feature database to determine the pet's identity identifier. If it is a new pet, create a new identity identifier and configure default preference parameters.

[0078] S220, perform frame-level alignment and normalization processing on the skeleton motion data and facial expression data according to the time window, and fuse them to form a multimodal temporal feature tensor;

[0079] S230, the multimodal temporal feature tensor is input into a pre-trained multi-task recognition model, the model including a shared spatiotemporal coding network, an emotion classification head and an intention classification head, and outputs the pet's instantaneous emotion type and behavioral intention;

[0080] S240, based on the identity identifier, retrieve the pet's historical interaction preference parameters from the preference storage module. The historical interaction preference parameters are a multi-dimensional vector that is dynamically updated based on the pet's past interaction feedback.

[0081] In step S210, identity feature matching and identity identification are performed: the high-dimensional identity feature vector of the current pet instance generated in step S150 is compared with the registered pet features stored in the device's local or cloud identity database to calculate the similarity, using cosine similarity as the metric; if the maximum similarity is higher than a preset threshold (e.g., 0.92), the corresponding registration name or number is used as the pet's identity identifier; if all similarities are lower than the threshold, it is determined to be a new pet, a new identity identifier is automatically created and registered, and a default interaction preference parameter table is assigned to the identity.

[0082] In S220, multimodal temporal feature alignment and fusion are performed: the skeleton motion data generated in step S130 and the facial expression data generated in step S140 are extracted with the same time window (e.g., a 2-second sliding window), and their time series are aligned at the frame level; the skeleton motion data sequence is normalized and aligned to the same coordinate system (relative to the pet's torso center of gravity), and the facial expression data sequence is smoothed and interpolated for missing values; the two processed features are concatenated in the time dimension to form a multimodal fusion feature tensor, which serves as the input for subsequent joint recognition.

[0083] In S230, emotion and behavioral intention are jointly inferred: the multimodal fusion feature tensor is input into a pre-trained multi-task recognition model; this model is designed to share a spatiotemporal graph convolutional or Transformer encoding layer to capture the spatiotemporal dependencies of pet behavior; above the shared encoding layer, an emotion classification head (outputting the probability of various emotions such as excitement, calmness, anxiety, boredom, and fear) and an intention classification head (outputting the probability of various intentions such as playing, foraging, seeking comfort, resting, and excretion) are branched out; the category with the highest probability in each classification head is taken as the current immediate emotion type and behavioral intention, and the confidence of both is output; this multi-task model, trained through a joint loss function, can decouple psychological state and action purpose from dynamic posture and micro-expressions;

[0084] In S240, historical interaction preference parameters are retrieved: based on the identity identifier determined in S210, the pet's unique historical interaction preference parameters are retrieved from the user profile and preference storage module; these parameters are multi-dimensional vectors, including at least: preferences for the movement pattern of the laser spot (fast / slow, random / realistic), the preference weight for different treats, the sensitivity to the owner's voice tone, and the tolerance for interaction time and duration; these parameters are updated through reinforcement learning after each interaction based on the pet's participation feedback (such as continuous tracking of the laser spot, rapid eating, improved mood, etc.) to ensure continuous adaptation; if S210 identifies a new pet, the default preference parameter set determined by prior knowledge of species / breed / age is retrieved.

[0085] In this embodiment of the invention, step S300 includes the following steps:

[0086] The S310 generates a quantitative cleanliness index and determines the cleanliness level through its built-in optical dust sensor and vision-based floor stain detection.

[0087] S320, based on the immediate emotion type and behavioral intention, extract the corresponding basic demand tendency weights from the preset mapping table, and use the time-related preference items in the historical interaction preference parameters to make weighted corrections to form a basic demand intensity vector, which includes the intensity of sports and play, the intensity of eating, the intensity of emotional comfort and the intensity of rest.

[0088] S330, the basic requirement intensity vector is corrected according to the cleanliness level. When the cleanliness level is lower than the preset threshold, the rest intensity is increased and the intensity of movement and play and the intensity of eating are suppressed. At the same time, local clean areas that can be used for interaction are marked.

[0089] S340, select the mode corresponding to the highest value in the corrected intensity vector as a candidate, generate the mode selection instruction under the premise that the candidate intensity exceeds the minimum trigger threshold and there is no need to force the switch to the emotional soothing mode due to the pet's high arousal emotion, and determine the final mode according to the preset strategy of prioritizing physiological needs when the intensity is equal.

[0090] In S310, the environmental cleanliness parameter is quantified in real time: the environmental sensor group built into the pet interaction device is called. This sensor group includes at least: an optical dust sensor for detecting the density of tiny particles and hair on the ground, and a stain coverage rate obtained by analyzing the image of the ground area captured from the visual acquisition unit through a stain segmentation model; the two types of outputs are combined to calculate the cleanliness index between 0 and 1, and the cleanliness level is determined into three levels: "clean", "normal" or "dirty" according to the preset segmentation threshold. This cleanliness level is used for subsequent decision correction.

[0091] In S320, the basic need intensity assessment is based on the state: according to the immediate emotion type and behavioral intention obtained in step S230, the corresponding basic need tendency weight is retrieved from the pre-set multidimensional mapping table; for example, the emotion "excitement" and the intention "play" correspond to a high weight for physical play; the emotion "anxiety" and the intention "seeking comfort" correspond to a high weight for emotional comfort; further, the time- and period-related preference items in the historical interaction preference parameters retrieved in step S240 (such as the pet being active in the current time period) are introduced, and the basic weights and preference vectors are weighted and summed to generate a normalized basic need intensity vector: [physical play intensity, feeding intensity, emotional comfort intensity, rest intensity]; this vector represents the urgency of the pet's need for each mode when environmental constraints are not considered;

[0092] In S330, the intensity of demand under environmental cleanliness constraints is corrected: the basic demand intensity is dynamically corrected using the cleanliness level obtained in S310, with the principle of ensuring pet health and interaction effects: if the cleanliness level is "dirty", the demand value of the fourth working mode corresponding to "rest intensity" is forcibly increased to the preset maximum value, and the "exercise and play intensity" and "feeding intensity" are multiplied by an inhibition factor less than 1 (such as 0.1) to prevent throwing treats or chasing light spots on dirty ground, and to avoid pets accidentally ingesting dirt or dust; if the cleanliness level is "normal", the original intensity is maintained, but exercise and play interactions are set to be performed only in local areas judged by the ground visual detection to be free of liquid stains (the information of this area is transmitted to S400 for execution); if the cleanliness level is "clean", no inhibition correction is made;

[0093] In S340, conflict resolution and mode selection instruction generation: The four modified demand intensity values ​​are sorted in descending order, and the working mode corresponding to the highest intensity value is selected as the candidate mode. To prevent frequent mode switching, the following logic is introduced to confirm the generation of the final mode selection instruction: First, check whether the intensity value of the candidate mode exceeds the preset minimum trigger threshold (e.g., 0.3). If it does not exceed it, the current mode remains unchanged to avoid over-response. Second, when the candidate mode is the fourth working mode (cleaning) and the pet's current emotion is a high arousal state such as "excitement" or "anxiety", it is determined that direct cleaning may cause harm to the pet. In the event of stress, the system will forcibly switch the candidate mode to the third working mode (emotional soothing), prioritizing voice soothing or remote dialogue to stabilize the emotion. Once the S500 detects that the emotion has calmed down, the cleaning instruction will be regenerated. Finally, if multiple candidate intensity values ​​are tied for the highest, a unique mode will be determined based on a preset strategy chain that prioritizes physiological needs over psychological needs and safety, such as "rest / excretion needs" > "feeding needs" > "emotional soothing needs" > "exercise and play needs". After determination, a mode selection instruction containing the mode number and associated control parameters (such as cleaning area mask, soothing volume, etc.) will be generated.

[0094] In this embodiment of the invention, step S400 includes the following steps:

[0095] S410, parse the mode selection instruction, extract the target working mode number and associated control parameters, the control parameters include local clean area mask, soothing volume, snack type and projection direction, etc.

[0096] When the first working mode is selected, the S420 generates a bionic prey movement trajectory based on the prey type and movement preference in the historical interaction preference parameters, limits the dynamic light spot projection to a safe and clean ground area, and plays simulated sounds that match the prey type simultaneously. During the projection process, the light spot movement speed and turning frequency are dynamically adjusted according to the pet's real-time tracking success rate and mood.

[0097] S430: When the second working mode is selected, the type and amount of snacks are selected based on historical interaction preference parameters. The snacks are launched to a safe and clean landing area in front of the pet through the ejection mechanism. Before launching, a call voice based on the owner's voiceprint is played, and the amount of food fed is accumulated. Once the daily recommended limit is exceeded, the mode is automatically disabled until the next day.

[0098] S440: When the third working mode is selected, an attempt is made to establish a real-time audio connection with the owner's terminal to enable remote dialogue. If the connection fails, a pre-soothing audio matching the pet's current mood is played. When the pet's mood becomes calm and it intends to rest, the audio is automatically faded out and the system switches to the fourth working mode. S450: When the fourth working mode is selected, unnecessary interaction modules are turned off, a cleaning path is planned, and the automatic cleaning unit is started. The pet's location is tracked in real time during the cleaning process. When the pet approaches or becomes stressed, the cleaning is paused and the pet is guided away. If the cleanliness is not up to standard after cleaning, supplementary cleaning is performed.

[0099] In S410, mode command parsing and control parameter extraction: the mode selection command is received and parsed, and the target working mode number and associated control parameters are extracted from it; the associated control parameters include at least: the local clean area mask generated by step S330, the volume level of the soothing audio, the type of treat and the projection direction vector, the initial value of the interaction difficulty of the laser projection, etc.; at the same time, the task scheduler of the pet interaction device is locked according to the command, and preparation is made to enter the corresponding mode;

[0100] In S420, the execution logic of the first working mode is as follows: When the target working mode is the first working mode, the following interactive operations are performed: The pet's chasing speed preference, turning frequency preference, and prey type preference (such as insects or rodents) are read from the historical interaction preference parameters; a bionic trajectory generator is invoked to generate a dynamic light spot movement trajectory sequence in real time, simulating the prey's escape behavior. The trajectory design includes behavioral segments such as random turns, acceleration sprints, and concealed pauses; the coordinates of the trajectory sequence are mapped onto the two-dimensional galvanometer control parameters of the laser projection unit, and the projection range is limited to the local clean area. Within a clean area mask, there is a safe ground area free of fragile items and reflective surfaces. During projection, the S100 continuously monitors the pet's tracking success rate and real-time emotions. If the tracking fails multiple times (e.g., more than 3 times) and the pet's emotions turn to "anxiety" or "boredom," the movement speed of the light spot is automatically reduced, and the frequency of directional changes is decreased to dynamically adapt to the pet's current mobility. The audio playback unit is called simultaneously to play simulated sounds that match the selected prey type in a loop. The playback sequence of the sounds is aligned with the pauses and accelerations of the light spot, and the volume is adaptively adjusted by the ambient background noise level.

[0101] In S430, the second working mode execution logic is as follows: When the target working mode is the second working mode, the following feeding interaction operation is performed: Based on the historical interaction preference parameters, the most suitable snack type and the number of pellets fed at one time are determined; the rotary selector and metering valve of the multi-cavity snack compartment are controlled to separate the corresponding snack pellets and send them into the pneumatic or mechanical ejection cavity; based on the pet's current spatial position coordinates, orientation, and interaction distance preference in the preference parameters, the pitch angle, azimuth angle, and initial launch velocity of the ejection mechanism are calculated, and control commands are generated to ensure that the snack is launched to a preset flat area in front of the pet. The system ensures an unobstructed and clean landing area to avoid hitting the pet's body or face. 0.5 seconds before the ejection command is issued, a call voice based on the owner's voiceprint or pre-recorded is played through the speaker. The voice includes the pet's name and feeding instructions. The system records the feeding time and the amount of treats, and compares this with the total amount fed that day. If the accumulated amount reaches the breed-specific daily recommended intake limit, the second working mode is automatically marked as disabled after the current mode ends, and will remain disabled until the next day. Even if the S300 generates a feeding demand again, it will be forcibly switched to the third or fourth working mode.

[0102] In S440, the third working mode execution logic is as follows: When the target working mode is the third working mode, the following emotional soothing and remote communication operations are performed: A status query request is sent to the preset owner terminal through the communication unit to detect the network accessibility of the terminal and whether the owner currently allows the connection; if allowed, a low-latency full-duplex real-time audio stream channel is established to transmit the pet's voice and ambient sound collected by the device's microphone to the owner terminal, and the owner's voice is played out in real time through the device's speaker; if the owner terminal cannot connect or rejects the request, a pre-soothing audio segment matching the current immediate emotional type is retrieved from the local storage (such as playing slow-paced, low-pitched comforting words and soft music for "anxiety"), and played through the speaker, with random intervals added during playback to simulate a natural dialogue feel; during the audio interaction, the pet's emotions and behavioral intentions are monitored, and once the emotion is detected to turn to "calm" and the intention to turn to "rest", the audio fades out, and a mode selection instruction pointing to the fourth working mode is actively generated to realize automatic environmental maintenance after soothing is completed;

[0103] In S450, the fourth working mode execution logic is as follows: When the target working mode is the fourth working mode, the following rest environment maintenance and cleaning operations are performed: Standby or hibernation commands are sent to the laser projection unit, snack ejection unit, and audio interaction unit to reduce power consumption and prevent accidental triggering that could disturb the pet's rest; based on the floor cleanliness index and local clean area mask generated by S310, the path planning module calculates a cleaning path covering all dirty areas and starts the automatic cleaning unit, sequentially driving the side brush, roller brush, and vacuum fan to collect and remove debris; the cleaning area prioritizes areas other than the pet's current location; during cleaning, the visual acquisition unit tracks the pet's position and posture in real time; when the pet is detected actively approaching the operating cleaning components or exhibiting fear or aggression, all moving parts are immediately paused, and a soothing voice or guiding light spot is temporarily activated to guide the pet to a safe area; cleaning resumes after the pet has moved to a safe distance and its emotions have recovered; after cleaning is completed, the floor cleanliness index is recalculated; if the set cleanliness threshold is still not reached, a supplementary cleaning path is automatically generated and the process is repeated once.

[0104] In this embodiment of the invention, step S500 includes the following steps:

[0105] S510, during the execution of any work mode, continuously run steps S100 and S200 to construct a temporal state stream from the output instantaneous emotion type, behavioral intention, identity identifier and confidence level, and calculate the change in emotion and intention.

[0106] S520, determine whether the preset mode switching conditions are met according to the time-series state stream. The mode switching conditions include emotion type conversion and confidence level exceeding the confirmation threshold, behavioral intention not matching the needs met by the current mode, environmental cleanliness suddenly dropping to an unsafe level, new pet entering or the current mode continuously timeout and emotion not improving, and each condition must be continuously met within the number of consecutive confirmation frames.

[0107] S530, when the mode switching conditions are met, executes a safety interruption according to the current working mode, namely, the first mode turns off the laser, the second mode cancels the unexecuted ejection, the third mode smoothly fades out the audio, and the fourth mode pauses cleaning and retracts moving parts, and saves the interaction context at the time of the interruption.

[0108] S540, return the interruption context and the latest pet status to step S300 to re-determine and generate a new mode selection instruction, so as to achieve seamless mode switching;

[0109] In S510, continuous acquisition and change quantification of multimodal state flow: During the execution of any working mode, steps S100 and S200 are kept running uninterrupted, continuously acquiring the instantaneous emotion type, behavioral intent, identity identifier, and its confidence score output for each frame or each fixed time interval (e.g., per second); the above state information is used to construct a state flow queue in chronological order, and a sliding window is used to calculate the rate of change of the emotion confidence vector, the consistency of the intent category, and the change identifier of the identity identifier within the window; when a change or addition of the identity identifier is detected, it means that a new pet has entered the interaction space;

[0110] In S520, the comprehensive judgment of preset mode switching conditions is as follows: Based on the state stream continuously generated by S510, it checks whether any of the following preset mode switching conditions are triggered in sequence, and introduces an anti-shake confirmation mechanism. It is only determined to be a valid trigger after the condition continuously meets the preset confirmation frame number or time threshold: The emotion type changes, and the average confidence of the new emotion in the current window exceeds the preset confirmation threshold (e.g., 0.7). The new emotion does not match the preset emotion adaptation range of the current working mode. The behavior intention category changes, and the physiological or psychological needs corresponding to the new intention do not match the needs type satisfied by the current working mode. For example, the current mode is soothing, but the intention changes to playing. The environmental cleanliness parameter changes abruptly through real-time calculation by S310, from "clean" or "normal" to "dirty", making the currently executed sports play or feeding mode no longer safe. A new pet individual is detected entering the screen, and the new pet's emotion or intention indicates that interaction is needed. The working mode of the multi-pet scene needs to be re-coordinated. The execution time of the current working mode exceeds the maximum single duration limit of the mode in the historical interaction preference parameters, and the pet's emotion does not improve as expected. Interaction fatigue is determined.

[0111] In S530, the current mode's safety interruption and context saving are as follows: Once the switching condition is determined to be validly triggered, a differentiated safety interruption process is executed according to the currently executing working mode type to protect the pet's safety and the device's integrity: If it is the first working mode, the laser emission of the laser projection unit is immediately turned off, the dynamic light spot is terminated, and the playback of simulated prey sounds is stopped; if it is the second working mode, the ejection command that has not yet been executed is canceled, the thrown treats are not retrieved, and the cumulative feeding amount at the time of this feeding interruption is recorded; if it is the third working mode, the soothing audio is smoothly faded out or an interruption prompt tone is sent to the owner's terminal to avoid sudden interruption causing confusion to the pet or abrupt communication with the owner; if it is the fourth working mode, all moving parts of the automatic cleaning unit are paused, the side brush is retracted, and the cleaning path breakpoint and completed areas are recorded; at the same time, a snapshot of the interactive context at the time of interruption is saved, including at least: the interruption reason, the pet's final state at the time of interruption, environmental parameters, and mode execution progress markers, forming an interruption context package;

[0112] In S540, the return to re-decision and seamless switching execution: the interruption context packet generated by S530 is used as input, carrying information such as the current pet's latest status and environmental cleanliness level, and the process returns to and re-executes step S300; when S300 is executed, it will give priority to the switching reason in the interruption context packet to avoid re-decision to enter the mode that was just interrupted (for example, forcibly excluding the sports and play mode before the cleanliness is restored); after S300 generates a new mode selection instruction, it is directly executed by S400 to complete the closed-loop seamless switching from the original mode to the new mode.

[0113] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for operating a pet interactive device, characterized in that, Includes the following steps: S100: The visual acquisition unit set on the pet interaction device captures images of the pet in real time and extracts the pet's skeletal movement data, facial expression data and identity feature data from the images. S200, based on the skeletal motion data, facial expression data and identity feature data, a multi-task recognition model is used to determine the pet's identity, immediate emotion type and behavioral intention, and historical interaction preference parameters corresponding to the identity are retrieved; S300: Based on the instantaneous emotion type, behavioral intention, and historical interaction preference parameters, combined with the environmental cleanliness parameters obtained by the environmental sensor, a mode selection instruction is generated. S400, execute the mode selection instruction to put the pet interaction device into the corresponding working mode; S500: During the execution of any working mode, continuously obtain the pet's real-time status through S100 and S200. When the change in the immediate emotion type and / or behavioral intention is detected to meet the preset mode switching conditions, interrupt the current working mode and return to S300 to redetermine and switch to the matching working mode.

2. The method of operating a pet interaction device according to claim 1, characterized in that, Step S100 includes the following steps: S110 performs frame-by-frame preprocessing on the raw image stream captured by the visual acquisition unit. S120: Input the preprocessed image into the instance segmentation model, segment out the pixel-level mask area of ​​each pet in the picture and assign instance labels, and establish a stable tracking trajectory for each individual pet by using cross-frame data association. S130: For the mask area of ​​each pet instance in each frame, the coordinates of multiple key points are extracted using the animal skeleton key point detection network, and the joint motion velocity, acceleration and angle are calculated based on the key point coordinate sequence of consecutive frames of the same pet individual to generate skeleton motion data. S140, based on the head key point position in the key points, cut out the head sub-region from the mask area of ​​the same pet instance, detect the contour points of the eyes, mouth and ears through the facial key point positioning model, and calculate the eye opening degree, mouth opening degree and ear posture angle, and combine them into facial expression data. S150, The pet identity feature extraction network is used to globally encode the mask region of each pet instance to generate a high-dimensional identity feature vector as the identity feature data, which is used to determine the pet's identity in subsequent steps.

3. The method of operating a pet interaction device according to claim 2, characterized in that, Step S200 includes the following steps: S210, perform similarity matching between the identity feature data and the pre-stored identity feature database to determine the pet's identity identifier. If it is a new pet, create a new identity identifier and configure default preference parameters. S220, perform frame-level alignment and normalization processing on the skeleton motion data and facial expression data according to the time window, and fuse them to form a multimodal temporal feature tensor; S230, the multimodal temporal feature tensor is input into a pre-trained multi-task recognition model, the model including a shared spatiotemporal coding network, an emotion classification head and an intention classification head, and outputs the pet's instantaneous emotion type and behavioral intention; S240, based on the identity identifier, retrieve the pet's historical interaction preference parameters from the preference storage module. The historical interaction preference parameters are a multi-dimensional vector that is dynamically updated based on the pet's past interaction feedback.

4. The method of operating a pet interaction device according to claim 3, characterized in that, The mode selection instruction corresponds to one of the first working mode, the second working mode, the third working mode, and the fourth working mode. The first working mode corresponds to the need for exercise and play, the second working mode corresponds to the need for feeding, the third working mode corresponds to the need for emotional comfort, and the fourth working mode corresponds to the need for rest and environmental maintenance.

5. The method of operating a pet interaction device according to claim 4, characterized in that, Step S300 includes the following steps: The S310 generates a quantitative cleanliness index and determines the cleanliness level through its built-in optical dust sensor and vision-based floor stain detection. S320, based on the immediate emotion type and behavioral intention, extract the corresponding basic demand tendency weights from the preset mapping table, and use the time-related preference items in the historical interaction preference parameters to make weighted corrections to form a basic demand intensity vector, which includes the intensity of sports and play, the intensity of eating, the intensity of emotional comfort and the intensity of rest. S330, adjust the basic requirement intensity vector according to the cleanliness level. When the cleanliness level is lower than the preset threshold, increase the rest intensity and suppress the intensity of movement and play and the intensity of eating, while marking local clean areas that can be used for interaction; 6. The method of operating a pet interaction device according to claim 5, characterized in that, When the first working mode is selected, the laser projection unit is activated to project dynamic light spots according to the bionic prey movement trajectory generated based on the historical interaction preference parameters, and simultaneously plays the first audio that simulates the sound of prey. When the second working mode is selected, the ejector snack delivery unit is activated, the snack type and throwing direction are determined according to the historical interaction preference parameters, and the preset owner summoning voice is played. When the third working mode is selected, a real-time audio connection is established with the owner's terminal through the communication unit, enabling the owner to have a remote conversation with the pet or play pre-stored soothing voice messages from the owner. When the fourth working mode is selected, unnecessary interactive modules are turned off, and the automatic cleaning unit is activated to clean the floor.

7. The method of operating a pet interaction device according to claim 6, characterized in that, Step S400 includes the following steps: S410, parse the mode selection instruction, extract the target working mode number and associated control parameters, the control parameters include local clean area mask, soothing volume, snack type and projection direction, etc. When the first working mode is selected, the S420 generates a bionic prey movement trajectory based on the prey type and movement preference in the historical interaction preference parameters, limits the dynamic light spot projection to a safe and clean ground area, and plays simulated sounds that match the prey type simultaneously. During the projection process, the light spot movement speed and turning frequency are dynamically adjusted according to the pet's real-time tracking success rate and mood. S430: When the second working mode is selected, the type and amount of snacks are selected based on historical interaction preference parameters. The snacks are launched to a safe and clean landing area in front of the pet through the ejection mechanism. Before launching, a call voice based on the owner's voiceprint is played, and the amount of food fed is accumulated. Once the daily recommended limit is exceeded, the mode is automatically disabled until the next day. S440, when the third working mode is selected, attempts to establish a real-time audio connection with the owner's terminal to achieve remote dialogue. If the connection cannot be established, it plays pre-soothing audio that matches the current immediate mood. When it detects that the pet's mood has turned calm and it intends to rest, it automatically fades out the audio and switches to the fourth working mode. When the fourth working mode is selected, the S450 turns off unnecessary interactive modules, plans the cleaning path and starts the automatic cleaning unit. During the cleaning process, it tracks the pet's location in real time. When the pet approaches or becomes stressed, it pauses the cleaning and guides the pet away. If the cleanliness is not up to standard after the cleaning is completed, it will perform supplementary cleaning.

8. The method of operating a pet interaction device according to claim 7, characterized in that, Step S500 includes the following steps: S510, during the execution of any work mode, continuously run steps S100 and S200 to construct a temporal state stream from the output instantaneous emotion type, behavioral intention, identity identifier and confidence level, and calculate the change in emotion and intention. S520, determine whether the preset mode switching conditions are met according to the time-series state stream. The mode switching conditions include emotion type conversion and confidence level exceeding the confirmation threshold, behavioral intention not matching the needs met by the current mode, environmental cleanliness suddenly dropping to an unsafe level, new pet entering or the current mode continuously timeout and emotion not improving, and each condition must be continuously met within the number of consecutive confirmation frames. S530, when the mode switching conditions are met, executes a safety interruption according to the current working mode, namely, the first mode turns off the laser, the second mode cancels the unexecuted ejection, the third mode smoothly fades out the audio, and the fourth mode pauses cleaning and retracts moving parts, and saves the interaction context at the time of the interruption. S540, return the interruption context and the latest pet status to step S300 to re-determine and generate a new mode selection instruction, thereby achieving seamless mode switching.

9. A pet interactive device, characterized in that, The interactive device includes an autonomous mobile chassis, a cleaning module, a snack throwing module, a laser projection module, a visual acquisition module, and a voice interaction module. The interactive device operates using the method described in any one of claims 1-8.

Images