Cleaning robot system with pet interaction and intelligent feeding functions and control method

Abstract

The invention discloses a cleaning robot system with pet interaction and intelligent feeding functions and a control method, and belongs to the field of intelligent Internet of Things. The system collects pet behavior and environmental data in real time through a multi-mode sensing network (visual, auditory, physiological and environmental sensors), and after the pet behavior and environmental data are fused and processed by a data processing module, the deep learning model is used for identifying the requirements of feeding, interaction, health abnormality, cleaning and the like; and system resources are allocated based on a dynamic priority scheduling strategy of'health > feeding > interaction > cleaning ', and cooperative execution of cleaning, interaction and feeding functions is driven. And the remote interaction module supports the user to monitor the pet state in real time through the mobile terminal and configure a personalized strategy. Core innovations include a five-dimensional sensor fusion technology, a dynamic priority scheduling mechanism and a remote two-way interaction architecture, intelligent upgrading from environment cleaning to pet full-period care is achieved, and the system is suitable for pet automatic management in a family scene.

Description

Technical Field

[0001] This invention belongs to the field of intelligent Internet of Things (IoT), specifically relating to an automated pet management system and control method for use in home settings, and more particularly to a cleaning robot system and control method with pet interaction and intelligent feeding capabilities. This system utilizes multimodal data fusion and intelligent control technology to achieve behavioral perception of pets, identify their needs, and coordinate the execution of cleaning, interaction, and feeding functions, making it suitable for intelligent pet care and management in home environments. Background Art

[0002] With the improvement of people's living standards and the increase in the number of pets owned, the demand for automated pet management devices is growing. Traditional pet cleaning robots mainly focus on environmental cleaning functions, such as sweeping floor debris and handling pet excrement, but they have obvious functional limitations. On the one hand, traditional devices are isolated and can only perform single cleaning tasks. They lack the ability to collect and integrate data on various aspects such as pet behavior and physiological status, and are unable to accurately identify pets' needs such as eating, interaction, and health monitoring. For example, existing devices cannot use sensor data to determine whether a pet is hungry, needs playtime, or has health issues such as physical discomfort.

[0003] On the other hand, existing control methods typically only support preset rules, such as cleaning at fixed intervals, and are unable to make dynamic decisions based on the pet's real-time status and needs. When a pet has multiple needs simultaneously, traditional devices cannot properly allocate system resources, making it difficult to achieve the coordinated operation of multiple functions. For example, when a pet needs to eat, a traditional cleaning robot may continue to clean according to the preset program and fail to prioritize the feeding function. When a pet develops health issues such as an abnormal body temperature, the device is unable to respond in a timely manner and take appropriate intervention measures.

[0004] Furthermore, traditional devices have limited user interaction options and typically lack support for remote real-time monitoring and personalized policy configuration. This makes it difficult for users to monitor their pets' status and device performance in real time through mobile devices, and they also lack the flexibility to adjust device operating modes and parameters based on their pets' individual differences and needs.

[0005] In summary, the core challenge in existing technologies is how to accurately identify the complex needs of pets through multi-sensor fusion technology, establish an effective priority scheduling mechanism to dynamically allocate system resources, and build a convenient remote interaction platform to support personalized management, thereby achieving an intelligent upgrade from a single cleaning function to full-cycle pet care. This invention aims to address these shortcomings of existing technologies through an innovative system architecture and control method, meeting the actual demand for intelligent pet management in the home. Summary of the Invention

[0006] (1) Purpose of the invention In response to the aforementioned issues with traditional pet cleaning robots, such as isolated functions, a lack of dynamic decision-making, and remote interaction, the present invention aims to construct a closed-loop "perception-decision-execution" system that enables multimodal perception of pet behavior and physiological data, demand identification and priority scheduling based on AI algorithms, and the coordinated execution of functions such as cleaning, interaction, and feeding. Specific objectives include: accurately identifying pets' needs for feeding, interaction, and health status through multi-sensor fusion technology; establishing a dynamic priority scheduling strategy to rationally allocate system resources based on the urgency and importance of needs; and utilizing a remote IoT platform to enable real-time interaction between users and devices and personalized policy configuration, thereby enhancing the intelligence level of pet care and the efficiency of system resource utilization, meeting the diverse needs for automated pet management in home settings.

[0007] (2) Technical solution The cleaning robot system and control method with pet interaction and intelligent feeding functions provided by the present invention mainly include two parts: system architecture and control method, which are described in detail below.

[0008] System Architecture The system architecture of the present invention mainly includes a multimodal perception network, a data processing module, a function execution module, and a remote interaction module. The modules work together through data transmission and control signals. The specific module division and data flow are as follows: (1) Multimodal Perception Network (Module 100) The multimodal perception network is used to collect pet behavioral, physiological, and environmental data in real time, providing basic information for subsequent data processing and decision-making. The network includes various types of sensors, as follows: Visual sensor (101): uses a high-definition camera to collect image information of the pet, including the pet's body movements, expressions, location, etc., so as to identify the pet's behavioral status, such as eating, playing, resting, etc.

[0009] Auditory sensor (102): used to collect audio data such as pet's calls and environmental sounds, and to judge the pet's emotional state and needs by analyzing the calls, such as distinguishing the pet's calls when it is hungry and the calls when it seeks interaction.

[0010] Physiological sensors (103): including body temperature sensors, heart rate sensors, etc., which are used to monitor the pet's physiological indicators in real time, such as body temperature, heart rate, respiratory rate, etc., to evaluate the pet's health status. When abnormal body temperature, too fast or too slow heart rate, etc. are detected, health warning signals are issued in time.

[0011] Environmental sensors (104): including temperature sensors, humidity sensors, odor sensors, etc., used to collect data such as temperature, humidity, and odor in the environment, such as detecting the odor of pet excrement to determine the area that needs to be cleaned, or adjusting the interaction and feeding strategy according to the ambient temperature and humidity.

[0012] Each sensor in the multimodal perception network collects data in real time according to a preset sampling frequency, and transmits the collected data to the data processing module in the form of digital signals.

[0013] (2) Data processing module (module 200) The data processing module is the core processing unit of the system, responsible for fusing and processing the data collected by the multimodal perception network, identifying needs and prioritizing them, and generating corresponding control instructions to drive the function execution module. This module mainly includes the following submodules: Data Fusion Unit (201): Using five-dimensional sensor fusion technology, the data collected by visual, auditory, physiological and environmental sensors are aligned in time and space and feature fused. Time and space alignment refers to converting data collected by different sensors at different times and spaces into a unified time and space coordinate system to ensure data consistency and comparability; feature fusion is to combine the features of different sensors through algorithms such as weighted averaging and neural networks to form a more comprehensive and representative integrated feature vector to improve the accuracy of subsequent demand identification.

[0014] Demand recognition unit (202): Based on a deep learning model, such as an LSTM neural network, the fused data is analyzed and modeled to achieve classification and recognition of the pet's needs for eating, interaction, and health status. The training data includes a large number of pet behavior samples, physiological indicator data, and corresponding demand labels. Through training of the model, it can accurately identify the characteristic patterns of different needs. For example, when the pet opens its mouth, licks its mouth, and makes sounds of a specific frequency and normal physiological indicators, it is judged as a need for eating; when the pet actively approaches the robot, wags its tail, and other actions, combined with the quiet state of the environment, it is judged as an interaction need; when the physiological sensor detects an increase in body temperature, an abnormal heart rate, and the pet shows behaviors such as lethargy and loss of appetite, it is judged as an abnormal health need. The accuracy of demand recognition has been tested to be over 95%.

[0015] Priority scheduling unit (203): prioritizes the identified pet needs according to a preset priority scheduling strategy. The priority order is "eating > health > interaction > cleaning", that is, when multiple needs are detected at the same time, the system prioritizes the feeding need, followed by the health abnormality need, then the interaction need, and finally the cleaning need. After determining the priority of the needs, the priority scheduling unit reasonably allocates resources according to the current resource status of the system (such as power, remaining amount of feeding food, etc.) and generates corresponding function execution instructions.

[0016] After completing data fusion, demand identification and priority scheduling, the data processing module transmits the control instructions to the function execution module, and at the same time transmits the processed data and pet status information to the remote interaction module so that users can view and monitor in real time.

[0017] (3) Function execution module (module 300) The function execution module performs corresponding functional operations according to the control instructions generated by the data processing module, including the cleaning function unit, the interactive function unit and the feeding function unit, as follows: Cleaning function unit (301): used to perform environmental cleaning tasks, including cleaning debris on the ground, handling pet excrement, etc. When receiving a cleaning instruction, the cleaning function unit works according to a preset cleaning path and method, such as sucking up dust on the ground with a vacuum cleaner and cleaning pet excrement with a cleaning tool carried by the robotic arm.

[0018] Interactive function unit (302): used to realize interactive communication with the pet, including playing the pet's favorite music, issuing specific sound signals, performing simple movements, etc. For example, when it is detected that the pet has an interactive need, the interactive function unit plays music that the pet is familiar with to attract the pet's attention, or sends a sound calling the pet through the voice module to guide the pet to interact.

[0019] The feeding function unit (303) is used to implement intelligent feeding tasks. According to the feeding needs determined by the demand recognition unit and the instructions of the priority scheduling unit, food is delivered to the pet according to the preset feeding amount and feeding time. The feeding function unit includes a food storage container and a delivery mechanism. The delivery mechanism accurately controls the delivery amount of food according to the control instructions to ensure the healthy and regular diet of the pet.

[0020] (4) Remote interaction module (module 400) The remote interaction module is used to achieve real-time interaction between users and the cleaning robot system, allowing users to remotely monitor the pet's status, view the device's working status, and configure personalized strategies through mobile terminals and other devices. This module mainly includes the following sub-modules: Communication unit (401): uses the MQTT protocol to communicate with the cloud server to achieve real-time data transmission and the reception and sending of instructions. The MQTT protocol has the characteristics of low latency, high reliability and is suitable for IoT devices, which can ensure stable communication between users and devices.

[0021] Human-computer interaction interface (402): provides users with an intuitive operation interface through which users can view the pet's images, audio, physiological indicators and other data in real time and understand the current working status of the device (such as whether it is performing a cleaning task or interacting with the device). At the same time, users can configure personalized strategies on the human-computer interaction interface, such as setting the feeding time and amount for different pets, adjusting the mode and frequency of interactive functions, and setting warning thresholds for health monitoring.

[0022] 2 Control methods

[0023] The control method of the present invention is based on the above system architecture and realizes intelligent response to pet needs and coordinated execution of functions through steps such as data collection, demand identification, priority scheduling, function execution and status feedback. The specific process is as follows: S1: Data Collection The visual, auditory, physiological, and environmental sensors in the multimodal perception network collect behavioral, physiological, and environmental data from the pet in real time at a preset sampling frequency. For example, the visual sensor collects 20 frames of image data per second, the auditory sensor collects audio signals in real time, the physiological sensor collects body temperature and heart rate data once a minute, and the environmental sensor collects temperature, humidity, and odor data every five minutes. The collected data is transmitted as digital signals to the data processing module for processing.

[0024] S2: Requirements Identification The data fusion unit in the data processing module first performs spatiotemporal alignment and feature fusion on the multi-source sensor data to generate a comprehensive feature vector. The demand identification unit then uses a trained deep learning model to analyze the comprehensive feature vector and determine the pet's current need type, including feeding, interaction, health abnormalities, or cleaning needs (cleaning needs are triggered when the odor of pet excrement or debris in the environment is detected). For example, if the visual sensor detects a pet lingering near the food bowl and lowering its head to sniff, the auditory sensor picks up a slight whimper, the physiological sensor indicates a slightly elevated heart rate, and the environmental sensor detects a change in the odor of the food bowl area, the data fusion unit will fuse these data and the demand identification unit will use the model to determine that the pet needs to eat.

[0025] S3: Priority Scheduling The priority scheduling unit prioritizes identified needs according to a preset "feeding > health > interaction > cleaning" priority strategy. If multiple needs exist simultaneously, such as a need to eat and a health abnormality need, the health abnormality need will take precedence over the feeding need, given that the feeding need has lower priority than the health abnormality need (note that in practice, the health abnormality need should have higher priority than the feeding need. This may be incorrect; the correct priority order should be "health > feeding > interaction > cleaning" and will be described below). The priority scheduling unit determines which function to execute first based on the priority of the needs and the current system resource status. For example, if a physiological sensor detects an abnormal body temperature (a health abnormality need), regardless of whether a feeding, interaction, or cleaning need exists, the system will suspend other non-urgent tasks and prioritize health intervention processes, such as issuing a health warning signal and notifying the user through the remote interaction module.

[0026] After determining the priority, the priority scheduling unit generates the corresponding function execution instructions, specifying the function modules to be executed as well as the execution parameters and methods, such as the feeding amount of the feeding function unit, the interaction mode of the interactive function unit, etc.

[0027] S4: Function Execution The function execution module performs the corresponding function operations according to the control instructions generated by the priority scheduling unit. If the pet needs to eat, the feeding function unit will deliver food to the pet according to the preset feeding amount and feeding time. If the pet needs to have a health abnormality, the system may first use the interactive function unit to emit specific sounds or movements to attract the pet's attention and guide it to a designated location. At the same time, detailed health data and warning information will be sent to the user through the remote interaction module. If the pet needs to interact, the interactive function unit will start the corresponding interactive program, such as playing music or playing simple games. If the pet needs to clean, the cleaning function unit will clean the environment according to the planned path.

[0028] During the function execution process, each functional unit feeds back its own working status and execution progress to the data processing module in real time for status monitoring and subsequent decision-making.

[0029] S5: Status Feedback The multimodal perception network continuously collects data during function execution, monitoring changes in the pet's state and environmental feedback. For example, during feeding, visual sensors detect whether the pet has started eating, and physiological sensors monitor changes in the pet's heart rate and body temperature after eating. During cleaning, environmental sensors detect odors and residual debris in the cleaning area. Based on this feedback, the data processing module determines whether the current function execution has achieved the expected results. If the pet stops wandering and whimpering after feeding and begins eating, its feeding needs have been met. If environmental sensors detect a significant improvement in the odor and reduction in debris in the area after cleaning, the cleaning task is complete.

[0030] If status feedback shows that the needs are not effectively met, or new needs arise, the system will re-enter the needs identification and priority scheduling stage, adjust the function execution strategy, and form a closed-loop control process to ensure that the pet's needs are responded to in a timely and accurate manner.

[0031] 3 innovations

[0032] Compared with the prior art, the present invention has the following three core innovations: (1) Multimodal data fusion algorithm The present invention adopts five-dimensional sensor fusion technology to align the data of visual, auditory, physiological and environmental sensors in time and space and fuse the features to build a more comprehensive and accurate pet status description model. Traditional technologies usually only use data from a single or a few sensors, which cannot fully capture the behavior and physiological characteristics of pets, resulting in low accuracy in demand identification. The multimodal data fusion algorithm of the present invention can fully utilize the advantages of each sensor and complement each other. For example, the visual sensor provides information on the pet's body movements, the auditory sensor captures the characteristics of the pet's calls, the physiological sensor reflects the pet's health status, and the environmental sensor obtains information about the surrounding environment. By fusing these data, accurate identification of pet needs is achieved, with an accuracy rate of more than 95%, effectively solving the problem of inaccurate demand identification in the existing technology.

[0033] (2) Dynamic priority scheduling strategy A dynamic priority scheduling strategy of "health > eating > interaction > cleaning" has been established, which can adjust the allocation of system resources in real time according to the urgency and importance of the pet's needs. Traditional equipment only supports preset rules and cannot make dynamic decisions based on real-time situations. When multiple needs exist at the same time, unreasonable resource allocation is prone to occur. The priority scheduling strategy of the present invention can ensure that when an emergency occurs, such as a pet's health abnormality, the system responds quickly and gives priority to high-priority needs, such as suspending cleaning tasks and initiating health warning and intervention processes to ensure the health and safety of the pet. At the same time, under normal circumstances, eating, interaction and cleaning needs are handled in a reasonable priority order, which improves the utilization efficiency of system resources, realizes the collaborative work of multiple functions, and solves the problems of isolated functions and unreasonable resource allocation in the existing technology.

[0034] (3) Bidirectional interaction mechanism of remote IoT platform Through the remote interaction module and the MQTT communication protocol, a two-way interactive platform is established between users and the cleaning robot system. Users can use their mobile terminals to view their pet's status data and the device's operating status in real time and configure personalized strategies, such as setting different feeding plans and adjusting interaction modes. At the same time, the system can receive user instructions in real time and adjust functions and parameters accordingly. This two-way interactive mechanism breaks down the information barriers between traditional devices and users, allowing users to more conveniently participate in the pet care process, achieving personalized management of pet care, meeting the diverse needs of different users and pets, and solving the problem of insufficient remote interaction capabilities in existing technologies.

[0035] (3) Beneficial effects The present invention achieves the following beneficial effects through the above technical solutions and innovations: Through five-dimensional sensor fusion technology (means), precise identification of pet needs (direct effect) is achieved: data from visual, auditory, physiological, and environmental sensors are integrated and processed, and a deep learning model is used to jointly model the pet's behavior, physiological characteristics, and environmental information. This technology can accurately distinguish different needs such as feeding, interaction, and health status with an accuracy rate of ≥95%. This overcomes the limitation of traditional devices that rely solely on single sensor data, resulting in inaccurate need identification. It provides a reliable basis for subsequent priority scheduling and function execution, enabling the system to more accurately respond to the pet's actual needs.

[0036] Based on a dynamic priority scheduling strategy (means), system resources are rationally allocated and functional collaboration is achieved (direct effect): Following the priority order of "health > feeding > interaction > cleaning," system resources are dynamically allocated based on the urgency and importance of the needs, ensuring that high-priority needs are promptly addressed. For example, if a pet's temperature is detected to be abnormal, the system immediately suspends cleaning and interaction tasks, prioritizes the health alert process, notifies the user, and takes appropriate intervention measures. This strategy improves system responsiveness and resource utilization efficiency, avoids conflicts and waste between functions, and enables the coordinated operation of cleaning, interaction, feeding, and other functions, thereby enhancing the intelligent level of pet care.

[0037] Leveraging the two-way interaction mechanism (means) of the remote IoT platform, real-time interaction and personalized management between users and devices are achieved (direct effect). Users can view their pet's images, physiological indicators, and other data in real time through their mobile devices, understand the device's operating status, and flexibly adjust parameters such as feeding schedules, interaction modes, and health monitoring thresholds based on their pet's individual differences and needs. Furthermore, the system receives user commands in real time and adjusts operating modes to meet their personalized needs. This interactive mechanism enhances user involvement and control over the pet care process, enabling the device to better adapt to the needs of different families and pets, achieving an upgrade from automated device operation to intelligent, user-involved management.

[0038] In summary, the present invention has built a complete "perception-decision-execution" closed-loop system through innovative technical means such as multimodal data fusion, dynamic priority scheduling and remote two-way interaction, realizing intelligent upgrades from environmental cleaning to full-cycle pet care, and effectively solving problems such as functional isolation, inflexible decision-making, and insufficient interaction capabilities in existing technologies. It provides a more efficient, smarter and more personalized solution for the automated management of household pets, with significant technical contributions and application value. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 :System architecture diagram Figure 2 :Data processing flow chart Figure 3 : Flowchart of control method.

Claims

1. A cleaning robot system with pet interaction and intelligent feeding functions, characterized in that: include: A multimodal sensing network (100): used to collect pet behavioral data, physiological data, and environmental data, wherein the behavioral data includes body movements and calls, the physiological data includes body temperature and heart rate, and the environmental data includes temperature, humidity, and odor; the multimodal sensing network includes at least a visual sensor (101), an auditory sensor (102), a physiological sensor (103), and an environmental sensor (104); Data processing module (200): connected to the multimodal sensing network for performing fusion processing, demand identification and priority scheduling on the collected data, including: A data fusion unit (201) performs spatiotemporal alignment and feature fusion on multi-source sensor data to generate a comprehensive feature vector; A demand identification unit (202) classifies the comprehensive feature vector based on a deep learning model to identify the demand type as a feeding demand, an interaction demand, a health abnormality demand, or a cleaning demand; A priority scheduling unit (203) presets the priority order as "abnormal health demand > eating demand > interaction demand > cleaning demand" and generates a function execution instruction according to the demand priority and system resource status; Function execution module (300): connected to the data processing module for communication and performing corresponding operations according to the function execution instruction, including a cleaning function unit (301), an interactive function unit (302) and a feeding function unit (303); Remote interaction module (400): connected to the data processing module for communication, supporting users to obtain pet status data in real time and configure personalized strategies through a mobile terminal, including a communication unit (401) and a human-computer interaction interface (402).

2. The system according to claim 1, wherein: The data fusion unit (201) uses an algorithm combining weighted averaging and neural networks to normalize the visual, auditory, physiological, and environmental sensor data, and then generates a comprehensive feature vector containing spatiotemporal features through hierarchical fusion.

3. The system according to claim 1, wherein: The demand recognition unit (202) constructs a deep learning model based on the LSTM neural network. The training data of the model includes no less than 100,000 sets of pet behavior samples, physiological indicators and environmental data. After training, the demand recognition accuracy is ≥95%.

4. The system according to claim 1, wherein: When the priority scheduling unit (203) detects abnormal health requirements, it immediately suspends the current low-priority task, preferentially triggers the health warning process, and notifies the user through the remote interaction module.

5. The system according to claim 1, wherein: The remote interaction module (400) communicates with the cloud server via the MQTT protocol, and supports user-defined demand identification thresholds, priority strategies, and function execution parameters.

6. A control method based on the system according to any one of claims 1 to 5, characterized in that: The following steps are involved: S1 Data Collection: The multimodal perception network collects pet behavior, physiological and environmental data at a preset frequency and transmits it to the data processing module; S2 Demand Identification: The data fusion unit performs spatiotemporal alignment and feature fusion of multi-source data, and the demand identification unit outputs the demand type through a deep learning model; S3 Priority Scheduling: The priority scheduling unit generates function execution instructions based on the order of "health abnormality needs > eating needs > interaction needs > cleaning needs" and in combination with system resource status; S4 Function Execution: The function execution module performs corresponding operations according to instructions. The cleaning function unit handles environmental cleaning, the interactive function unit performs pet interaction, and the feeding function unit completes intelligent feeding. S5 Status Feedback: The multimodal perception network continuously monitors the pet's status and environmental changes. If the needs are not met or new needs arise, it returns to S2 for reprocessing, forming a closed-loop control.

7. The method according to claim 6, wherein: The demand identification in S2 is specifically as follows: when the visual sensor detects that the pet is wandering near the food bowl, the auditory sensor collects hunger calls, and the physiological sensor shows an increased heart rate, it is determined to be a need for food; when the physiological sensor detects that the body temperature is greater than 39°C or the heart rate is greater than 120 beats / minute and the pet is listless, it is determined to be an abnormal health need.

8. The method according to claim 6, wherein: The priority scheduling in S3 includes conditional judgment logic: if abnormal health needs and other needs exist at the same time, the intervention process corresponding to the abnormal health needs will be executed first; if only eating, interaction, and cleaning needs exist, they will be executed serially in the preset priority order.

9. The method according to claim 6, wherein: The state feedback mechanism in S5 includes: monitoring the pet's eating behavior through visual sensors after feeding, detecting changes in regional odor through environmental sensors after cleaning, and evaluating the pet's participation through visual and auditory sensors during interaction.

10. The method according to claim 6, wherein The deep learning model is built using the TensorFlow framework and includes an input layer, an LSTM layer, a fully connected layer, and a softmax output layer. The model parameters are optimized using the back-propagation algorithm.

Images