Control method and device based on multi-mode perception, pet fostering equipment and medium

Abstract

The invention discloses a control method and device based on multi-modal perception, pet fostering equipment and a medium, and the method comprises the steps: obtaining multi-modal data which comprises pet behavior data, environment state data and equipment feedback data; a dynamic priority matrix is constructed based on the multi-modal data, a task queue is generated according to the dynamic priority matrix, the task queue comprises a plurality of foster tasks, and the dynamic priority matrix is used for representing priorities of foster requirements of pets; and controlling the pet fostering equipment to sequentially execute the fostering tasks according to the arrangement sequence in the task queue. The pet fostering equipment can be controlled according to the real-time requirements of the pet so as to meet the requirements of the pet.

Description

Technical Field

[0001] The present invention relates to the technical field of pet rearing, and particularly to a control method, device, pet rearing equipment and medium based on multi-modal perception. Background Art

[0002] Pet rearing equipment (such as automatic feeders, air purifiers) is used for automatically rearing pets, such as feeding pets, purifying air, cleaning, etc. Most of the current pet rearing equipment lacks collaborative control capabilities. For example, the feeder only works based on timing or quantitative rules, and the air purifier only supports timed cleaning, and cannot be dynamically adjusted according to the real-time needs of pets, which is likely to interfere with the rest or feeding of pets. Summary of the Invention

[0003] Embodiments of the present invention provide a control method, device, pet rearing equipment and medium based on multi-modal perception, aiming to solve the problem that the current pet rearing equipment cannot be dynamically adjusted according to the real-time needs of pets.

[0004] In a first aspect, an embodiment of the present invention provides a control method based on multi-modal perception, which is applied to pet rearing equipment. The method includes: Obtaining multi-modal data, where the multi-modal data includes pet behavior data, environmental state data, and device feedback data; Constructing a dynamic priority matrix based on the multi-modal data, and generating a task queue according to the dynamic priority matrix, where the task queue includes multiple rearing tasks, and the dynamic priority matrix is used to represent the priority of the pet's rearing needs; Controlling the pet rearing equipment to sequentially execute the rearing tasks according to the arrangement order in the task queue.

[0005] In a second aspect, an embodiment of the present invention further provides a control device based on multi-modal perception, which is applied to pet rearing equipment. The device includes: A first obtaining unit, configured to obtain multi-modal data, where the multi-modal data includes pet behavior data, environmental state data, and device feedback data; A first constructing unit, configured to construct a dynamic priority matrix based on the multi-modal data, and generate a task queue according to the dynamic priority matrix, where the task queue includes multiple rearing tasks, and the dynamic priority matrix is used to represent the priority of the pet's rearing needs; A first executing unit, configured to control the pet rearing equipment to sequentially execute the rearing tasks according to the arrangement order in the task queue.

[0006] In a third aspect, an embodiment of the present invention further provides a pet rearing device, which includes a memory and a processor connected to the memory. A computer program is stored on the memory, and when the processor executes the computer program, the above method is implemented.

[0007] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above method can be implemented.

[0008] An embodiment of the present invention provides a control method, device, pet rearing device and medium based on multimodal perception. The method includes: obtaining multimodal data, where the multimodal data includes pet behavior data, environmental status data, and device feedback data; constructing a dynamic priority matrix based on the multimodal data, and generating a task queue according to the dynamic priority matrix, where the task queue includes multiple rearing tasks, and the dynamic priority matrix is used to represent the priority of the pet's rearing needs; controlling the pet rearing device to sequentially execute the rearing tasks according to the arrangement order in the task queue. An embodiment of the present invention can obtain multimodal data in real time by acquiring pet behavior data, environmental status data, and device feedback data, then construct a dynamic priority matrix based on the multimodal data, and generate a task queue based on the dynamic priority matrix, so that the pet's needs can be matched with the task priorities of the pet rearing device. Finally, the rearing tasks are sequentially executed according to the arrangement order in the task queue to ensure that the rearing tasks with higher demand levels are preferentially satisfied, realizing dynamic adjustment. Description of the Drawings

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0010] Figure 1 is a schematic flowchart of the control method based on multimodal perception provided by an embodiment of the present invention; Figure 2 is a schematic diagram of the first sub-process of the control method based on multimodal perception provided by an embodiment of the present invention; Figure 3 is a schematic diagram of the second sub-process of the control method based on multimodal perception provided by an embodiment of the present invention; Figure 4 is a schematic diagram of the third sub-process of the control method based on multimodal perception provided by an embodiment of the present invention; Figure 5It is a schematic diagram of the fourth sub - process of the control method based on multi - modal perception provided by an embodiment of the present invention; Figure 6 It is a schematic diagram of the fifth sub - process of the control method based on multi - modal perception provided by an embodiment of the present invention; Figure 7 It is a schematic block diagram of a control device based on multi - modal perception provided by an embodiment of the present invention; Figure 8 It is a schematic block diagram of a pet - raising device provided by an embodiment of the present invention. Detailed implementation manners

[0011] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0012] It should be understood that when used in this specification and the appended claims, the terms "comprises" and "comprising" indicate the presence of the described features, wholes, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, operations, elements, components, and / or their combinations.

[0013] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in this specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms. It should be further understood that the term "and / or" used in this specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the related listed items, and includes these combinations.

[0014] Please refer to Figure 1 , Figure 1 It is a schematic flow diagram of the control method based on multi - modal perception provided by an embodiment of the present invention. The control method based on multi - modal perception in the embodiment of the present invention can be applied to pet - raising devices, such as automatic feeders and pet purifiers, for generating a task queue according to the needs of pets to achieve the purpose of dynamic adjustment. As Figure 1 shown, the method includes steps S100 - S120.

[0015] S100, obtain multi - modal data, where the multi - modal data includes pet behavior data, environmental state data, and device feedback data.

[0016] In the embodiments of the present invention, the multimodal data may include timestamp data, pet behavior data, environmental status data, and device feedback data. Among them, the timestamp data refers to the time points when pet behaviors occur (such as the last feeding time, the start time of an activity), and the time difference (Δt) is calculated to evaluate the urgency of needs. For example, when the time since the last feeding exceeds 5 hours, the priority of the hunger need increases linearly with time. The environmental status data refers to the integration of parameters such as PM2.5 (unit: μg / m³), temperature and humidity (℃ / %RH), odor concentration, etc., which are converted into environmental pressure values. For example, when the temperature > 28℃, the weight of the humidification and cooling needs increases by 20%. The pet behavior data refers to the behavioral actions and sounds of the pet. The behavioral actions include behaviors such as running, jumping, chasing, curling up, etc., and the sound refers to the barking of the pet. The needs of the pet can be confirmed through the behavioral actions and sounds of the pet. The device feedback data refers to the parameters of the device itself, such as the weight of the food bowl, the fan speed, etc.

[0017] The pet behavior data can be collected through a camera and a microphone. For example, the behavioral actions of the pet are captured through the camera, and the barking of the pet is collected through the microphone. The environmental status data can be collected through a camera and various sensors. For example, whether there is vomit is identified through the camera, and whether the odor is abnormal is detected through the sensor to obtain temperature data and humidity data. The device feedback data can be directly read and obtained.

[0018] S110, constructing a dynamic priority matrix based on the multimodal data, and generating a task queue according to the dynamic priority matrix, where the task queue includes multiple feeding tasks, and the dynamic priority matrix is used to represent the priority of the pet's feeding needs.

[0019] In the embodiments of the present invention, the dynamic priority matrix is used to represent the priority of the pet feeding needs. The task queue is an ordered execution sequence of tasks generated based on the dynamic priority matrix and sorted by priority. It can convert the demand decision driven by multimodal data into an ordered scheduling scheme for specific device actions.

[0020] The dynamic priority matrix can be updated in real time according to the changes in the multimodal data, so that the sorting in the task queue can be adjusted immediately. For example, high-priority tasks can preempt low-priority tasks in real time (such as vomit cleaning interrupts the ongoing feeding), ensuring that urgent needs are processed first. It can also disassemble complex tasks into atomic operations (such as pausing purification before feeding), achieve cross-module collaboration (such as purification, feeding, audio function linkage), and record the historical decision effects (such as the demand response accuracy rate), and use reinforcement learning to update the matrix weight allocation to improve the adaptability of the device to individual differences of pets and environmental changes.

[0021] For example, based on multimodal data, the feeding requirement is confirmed to generate a dynamic feeding task. At the same time, when the environmental PM2.5 = 40 μg / m³, a "conventional purification" task is generated. Then, the task queue can include two tasks: dynamic feeding and conventional purification, and the priority of dynamic feeding is higher than that of conventional purification. During execution, dynamic feeding is preferentially executed. When performing dynamic feeding, the purification is first paused, then food is fed, and the purification is resumed after feeding is completed to avoid interference with pet feeding by purification. After completing the dynamic feeding task, the conventional purification task is executed, and the fan is automatically adjusted according to PM2.5.

[0022] In some embodiments, such as in the embodiments of the present invention, as Figure 2 shown, step 110 includes steps S111 - S113.

[0023] S111, confirm the pet's care needs based on the multimodal data, and confirm the basic weight corresponding to the care needs according to a preset grading standard; S112, confirm the adjustment weight of the care needs based on the multimodal data, and confirm the final weight based on the adjustment weight and the basic weight; S113, sort the care needs according to the final weight to obtain the dynamic priority matrix.

[0024] In the embodiments of the present invention, the preset grading standard may include high - priority needs, sub - high - priority needs, medium - priority needs, low - priority needs, and normal - priority needs. High - priority needs refer to abnormal behaviors of the pet such as vomiting and convulsions, which need to be processed immediately. Sub - high - priority needs mean that the pet has a feeding need and needs to be fed in a timely manner. Medium - priority needs refer to the pet's rest need, and noise or purification actions need to be reduced to avoid disturbing the pet's rest. Low - priority needs refer to the pet's playing need, and music can be played to accompany the pet's play. Normal - priority needs mean that the pet has no obvious specific needs, and the pet care device can automatically perform purification and cleaning tasks according to environmental status data.

[0025] Different care tasks correspond to different basic weights. High - priority care tasks have higher basic weights, and low - priority care tasks have lower basic weights. For example, the basic weight of high - priority needs can be set to 100, the basic weight of sub - high - priority needs can be set to 80, the basic weight of medium - priority needs can be set to 50, the basic weight of low - priority needs can be set to 30, and the basic weight of normal - priority needs can be set to 10.

[0026] Adjusting the weight means increasing or decreasing the basic weight according to the changes in multimodal data. For example, if it has been a long time since the pet's last meal, the basic weight of the second-highest priority need can be increased. If the pet has been active for a long time, the basic weight of the medium-priority need can be increased. For instance, the basic weight of the second-highest priority is 80. If it has been a long time since the last meal, the calculated adjusted weight is 7, so the final weight of the second-highest priority need is 87. If the pet has been active for a long time, the calculated adjusted weight is 6, so the final weight of the low-priority need is 56. It can be understood that the basic weight of the high-priority need is 100, corresponding to the pet's urgent needs, such as the pet showing vomiting behavior, convulsion behavior, etc. Therefore, the basic weight of the urgent need is usually not adjusted. Once the pet has an urgent need, it must be dealt with first.

[0027] In some embodiments, such as in the embodiments of the present invention, as Figure 3 shown, step 111 includes steps S1111 - S1115.

[0028] S1111, obtain the pet behavior data, the environmental status data, and the device feedback data, and confirm the rearing needs based on the pet behavior data, the environmental status data, and the device feedback data; S1112, if it is confirmed through the pet behavior data that the pet has vomiting behavior and it is confirmed through the environmental status data that there is an abnormal smell, then confirm that the pet has an urgent need; S1113, if it is confirmed through the pet behavior data that the pet is in front of the food bowl and it is confirmed through the device feedback data that the weight of the food bowl is less than the preset threshold, then confirm that the pet has a feeding need; S1114, if it is confirmed through the pet behavior data that the pet's posture is a curled lying position and the eyes are in a closed state, then confirm that the pet has a rest need; S1115, if it is confirmed through the pet behavior data that the pet has jumping behavior or chasing behavior, then confirm that the pet has a playing need.

[0029] In the embodiments of the present invention, the needs of pets can be confirmed based on pet behavior data, environmental status data, and device feedback data. For example, when a pet exhibits vomiting behavior or convulsive behavior, it can be confirmed that the pet has an urgent need. When the pet is in front of the food bowl and the weight of the food bowl is less than a preset threshold, it is confirmed that the pet has an eating need. When the pet's posture is a curled lying position and its eyes are closed, it is confirmed that the pet has a rest need. When the pet exhibits jumping behavior or chasing behavior, it is confirmed that the pet has a playing need. Specifically, a CNN+SVM cascade model can be used to confirm the needs of pets. For example, pet behavior data, environmental status data, and device feedback data can be obtained. For example, the pet's posture (such as sitting, lying, jumping) can be captured in real time through a camera, and the behavior scenario can be identified through an algorithm (such as wandering in front of the food bowl, vomit recognition). The call signal is collected through a microphone and converted into a frequency-domain feature vector through feature extraction to reflect the pet's emotional state (such as hungry calls, excited calls during play). The remaining amount of the food bowl is monitored through a weight sensor, and the staying duration of the pet in the food bowl or the rest area is detected through a pressure pad to judge the persistence of the need. Parameters such as PM2.5, temperature and humidity, and odor concentration are collected in real time through various sensors.

[0030] After obtaining multi-modal data, timestamp calibration can be performed to add a unified timestamp to pet behavior data, environmental status data, and device feedback data, and data with different sampling rates can be aligned through linear interpolation (such as matching the frequency of the camera at 30fps with that of the temperature and humidity sensor at 1Hz). In addition, a sliding window filter can be used to remove sensor noise (such as resampling when the fluctuation of the weight sensor is >5g).

[0031] After data cleaning, feature extraction can be performed through CNN. Visual images, MFCC feature vectors, and environmental parameters (after normalization) are input into a lightweight CNN (such as MobileNet) to extract cross-modal shared features (such as the visual feature of "the pet approaching the food bowl" is associated with the acoustic feature of "hungry calls"). The fused features output by CNN are input into an SVM classifier so that the SVM classifier classifies the need types (eating / rest / playing / urgent) and the urgent levels (1-5 levels), and outputs a probability distribution (such as the probability of eating need is 88%).

[0032] Demand type mapping rules can be constructed to facilitate the model to accurately identify the needs of pets. For example, for the feeding demand, the trigger conditions are visual recognition of "standing / bowing the head in front of the food bowl" (confidence level ≥ 80%) + the remaining amount of the weight sensor < the threshold (such as 15g) + the staying duration > 30 seconds, or auditory recognition of "hungry barking". For the rest demand, the trigger conditions are visual recognition of "curled lying posture + eyes closed" (lasting for 5 minutes, confidence level ≥ 90%) + environmental noise < 40dB, or the pressure pad detects continuous staying > 10 minutes. For the play demand, the trigger conditions are visual recognition of "jumping / chasing behavior" (lasting for 1 minute, confidence level ≥ 93%) + auditory recognition of "excited barking" + environmental temperature < 35°C. For the emergency demand, the trigger conditions are visual recognition of "vomit" (confidence level ≥ 85%) + abnormal odor sensor (such as a sudden increase in the concentration of volatile organic compounds), or the pressure pad detects abnormal twitching movements of the pet.

[0033] In some embodiments, for example, in the embodiments of the present invention, as Figure 4 shown, step 111 further includes steps S1116 - S1120.

[0034] S1116, obtain all the rearing demands and confirm the category of each rearing demand; S1117, if the rearing demand is the emergency demand, confirm the basic weight of the emergency demand as the first weight according to the preset grading standard; S1118, if the rearing demand is the feeding demand, confirm the basic weight of the feeding demand as the second weight according to the preset grading standard; S1119, if the rearing demand is the rest demand, confirm the basic weight of the rest demand as the third weight according to the preset grading standard; S1120, if the rearing demand is the play demand, confirm the basic weight of the play demand as the fourth weight according to the preset grading standard.

[0035] In the embodiments of the present invention, the first weight to the fourth weight are all empirical values. For example, the first weight can be 100, the second weight can be 80, the third weight can be 50, the fourth weight can be 30. Additionally, when the pet has no obvious specific needs, the conventional task weight of the pet rearing device is 10.

[0036] For example, if it is recognized that the pet stands in front of the food bowl with its head lowered (the model confidence level is 92%), the weight sensor shows a remaining amount of 8g (< 15g threshold, confidence level 100%), and the environmental temperature is 28°C (accelerating metabolism and enhancing the rationality of feeding), then the SVM classifies it as "feeding demand", and the corresponding basic weight is 80 points.

[0037] If it is recognized that the pet's eyes are closed and it is in a curled lying position (lasting for 5 minutes with a confidence level of 90%), the environmental temperature and humidity are 22°C / 38% (the humidity is lower than the target of 45%), and the pressure pad detects that the pet has stayed continuously for 6 minutes, then it is determined that there is a rest need, and the basic weight is 50 points.

[0038] In some embodiments, for example, in the embodiments of the present invention, as Figure 5 shown, step 121 further includes steps S1121 - S1124.

[0039] S1121, obtain all the rearing needs and confirm the category of each rearing need; S1122, if the rearing need is the feeding need, obtain the temperature factor, activity factor, and weight factor, and confirm the adjustment weight of the feeding need based on the temperature factor, activity factor, and weight factor; S1123, if the rearing need is the rest need, obtain the humidity factor and age factor, and confirm the adjustment weight of the rest need based on the humidity factor and age factor; S1124, if the rearing need is the playing need, obtain the temperature factor and activity factor, and confirm the adjustment weight of the playing need based on the temperature factor and activity factor.

[0040] In the embodiments of the present invention, for the feeding need, the temperature factor is the environmental temperature, denoted by E1, the activity factor is the pet's daily activity amount, denoted by H1, and the weight factor is the pet's current weight, denoted by H2. When the environmental temperature > 25°C, for every 1°C increase, E1 increases by 0.02 (0.02 is an empirical value and can be adjusted), indicating that high temperature accelerates metabolism and more feeding compensation is needed. For example, when the temperature is 28°C, E1 = (28 - 25)×0.02 = 0.06. When the pet's daily activity amount exceeds the average activity amount, the feeding amount can be increased, and when the pet's weight exceeds the average, the pet's feeding amount is reduced. Let H1 = 1 - a%*0.2, H2 = 1 + b%*0.1, then the final weight = basic weight * (1 + E1*H1*H2). For example, the environmental temperature is 28°C (E = 0.06), the pet's weight is normal (a = 0, weight factor = 1), and the activity amount exceeds the average by 20% (b = 20, activity factor = 1.02), then the final weight = 80 * (1 + 0.06 * 1.02) ≈ 84.89.

[0041] For the rest requirement, the adjustment weight of the rest requirement can be calculated according to the humidity factor E2 and the age factor H3. For example, if the target humidity is 45% and the actual humidity is S%, then the humidity gap = 45 - S. When the gap > 5%, for each 1% increase, E2 increases by 0.03 (when the humidity is insufficient, the adjustment priority of the rest environment is improved). For example, when the actual humidity is 38%, the gap = 7%, and E2 = 7×0.03 = 0.21. The age factor H3 for pets older than 8 years old can be set to 1.1, the age factor for pets younger than 1 year old is 1.05, and the age factor for those in between is 1. Then the final weight = basic weight * (1 + E2 * H3). For example, when the actual humidity is 38% (E = 0.21) and the pet is an 8-year-old senior dog (H3 = 1.1), then the final weight = 50 * (1 + 0.21 * 1.1) ≈ 61.55.

[0042] For the play weight, its adjustment weight can be calculated according to the temperature factor E1 and the activity level factor H1. When the ambient temperature > 30°C, for each 1°C increase, E increases by 0.04 (when it is hot, cooling and humidifying are required for playing). When the daily activity level is lower than the average, then H = 1 - c% * 0.05 (when the activity level is low, the demand for play stimulation is reduced). If it is higher than the average, then H = 1 (no adjustment). Then the final weight = basic weight * (1 + E1 * H1). For example, when the ambient temperature is 32°C (E = (32 - 30)×0.04 = 0.08) and the pet's activity level is higher than the average (H = 1), then the final weight = 30 * (0.08 * 1) = 24.

[0043] S120, control the pet rearing device to sequentially execute the rearing tasks according to the arrangement order in the task queue.

[0044] In the embodiment of the present invention, the task queue may include multiple rearing tasks, such as emergency tasks, feeding tasks, rest tasks, play tasks, and routine cleaning tasks. For each task, when executing, the task can be split into atomic operations that cannot be further divided. For example, the feeding task is split into pausing the purifier - calculating the feeding amount and starting the feeding motor - resuming the operation of the purifier. That is, when executing a task, first obtain the first task at the forefront from the task queue in order, then split the first task into multiple subtasks, and execute them sequentially to complete the first task. After completing the first task, continue to obtain the first task at the forefront from the task queue, and repeat the above process until all tasks in the task queue are completed.

[0045] An execution mechanism can be configured for the task queue. For example, high-priority tasks can interrupt the execution of low-priority tasks. For instance, when the feeding task (second-highest priority, weight 88 points) is being executed, if vomiting (high priority, weight 100 points) is detected, the feeding will be immediately paused and an emergency cleaning task will be inserted. Dependency relationships are set. For example, "restoring the purifier" must be executed after "feeding is completed" to ensure the correctness of the action sequence through a state machine. After the execution layer completes a task, the feedback layer evaluates the effect (such as the degree of food intake). If an abnormality is found (such as the remaining rate of continuous feeding > 20%), the basic weight of subsequent tasks will be automatically adjusted (such as reducing the feeding amount by 5%); when new requirements are generated, the weights are recalculated and inserted into the queue, triggering dynamic updates to the queue.

[0046] The following uses several specific examples to illustrate the control logic of pet-rearing equipment under different requirements.

[0047] Scenario 1: Emergency requirements The camera captures vomit on the ground and detects foreign objects on the ground (classified as vomit, confidence level 85%) through visual recognition. Priority determination: Trigger the highest-priority task (safety and cleaning), immediately turn off the purifier fan to prevent the spread of pollutants; start the emergency cleaning mode: The built-in vacuuming module (power increased to 200W) sweeps the vomit along the preset path; the ultraviolet disinfection lamp irradiates the contaminated area for 3 minutes after cleaning; push an alarm message (including accident photos and processing logs) to the user APP.

[0048] Scenario 2: Play requirements The camera recognizes that the pet is playing, and the microphone detects the pet's barking. The ambient noise > 40dB, and the temperature and humidity sensor shows a temperature of 32°C and a humidity of 40%. When the camera recognizes that the pet is in a jumping or chasing behavior (duration > 1 minute) and the ambient noise > 40dB, trigger the "play environment self-regulation mode" with a confidence level of 93%. Purifier control: Automatically control the fan speed according to the PM2.5 value; and automatically enhance the negative ion release. If the pet is detected to be approaching, set the "gentle mode" and the wind speed drops by 50%. Humidification control: The target humidity is increased to 60%, and humidification and cooling are turned on; if it exceeds the target value, the humidification is turned off. Play background music: Select the sound type with a pet response rate > 80% (such as bird chirping, sound of flowing water) according to historical data, and automatically reduce the fan speed to < 45dB of background noise during playback; after the audio playback ends, the fan speed gradually returns to the normal level. Audio interaction: Provide an interaction mode to interact with the pet simply (such as voice response). Audio preference update: Record the reaction duration of the pet to different sounds, and update the playlist priority through cluster analysis.

[0049] Scenario 3: Routine requirements When the camera recognizes that there are no pets within the room range (or through manual switching), it triggers the regular automatic purification mode with a confidence level of 90%. Purifier control: Automatically control the fan speed according to the PM2.5 value; and automatically turn on the negative ion function. Humidification control: Start humidification with a target of 45%; turn off humidification if the target value is exceeded.

[0050] In some embodiments, such as in the embodiments of the present invention, the method further includes the following steps: If the rearing requirement is the feeding requirement, then confirm the feeding amount according to a preset feeding amount formula, where the preset feeding amount formula is: Q = [Q base (1 + k·Δt)−C·Wremain]p; Q is the feeding amount, Q base is the reference feeding amount, Δt is the time difference from the last feeding, Wremain is the remaining amount in the food bowl, k and C are adaptive coefficients, and p is the food intake compensation coefficient.

[0051] In the embodiments of the present invention, assume the reference feeding amount Q base =50, the time difference from the last feeding Δt = 5h (from 10 am to 3 pm), the remaining amount in the food bowl Wremain = 8g, the adaptive coefficient k = 0.1, C = 0.2, and the food intake compensation coefficient p = 1.1 (due to the environmental temperature of 28°C, the metabolism accelerates and compensation is required). Then, substituting into the preset feeding amount formula, we can get Q = 80.74g. Through the preset feeding amount formula, the feeding amount can increase linearly with the hunger time, avoiding overhunger of the pet, and the feeding can also be reduced according to the remaining food amount to prevent waste.

[0052] In some embodiments, such as in the embodiments of the present invention, as Figure 6 shown, the multi-modal perception-based control method further includes steps S130 - S140.

[0053] S130, if a new rearing requirement is detected, then confirm the basic weight and adjustment weight of the new rearing requirement to obtain the final weight of the new rearing requirement; S140, adjust the dynamic priority matrix based on the final weight of the new rearing requirement to generate a new dynamic priority matrix.

[0054] In the embodiments of the present invention, the pet rearing device obtains multi-modal data in real time. If a new rearing requirement is generated in the multi-modal data, the final weight of the new rearing requirement can be calculated, and the dynamic priority matrix can be adjusted based on the final weight to obtain a new dynamic priority matrix, and then a new task queue can be generated to realize real-time update of the task queue.

[0055] The control method based on multi-modal perception disclosed by the present invention can switch the connection state of the compressor according to the target frequency, and adjust the magnitude of the bus voltage according to the weak magnetic angle, which can improve the energy efficiency in the low-frequency band while ensuring the stability in the high-frequency band.

[0056] Figure 7 It is a schematic block diagram of a control device 200 based on multi-modal perception provided by an embodiment of the present invention. As Figure 7 shown, corresponding to the above control method based on multi-modal perception, the present invention also provides a control device 200 based on multi-modal perception. The control device 200 based on multi-modal perception includes units for executing the above control method based on multi-modal perception. Specifically, please refer to Figure 7 , the control device 200 based on multi-modal perception includes a first acquisition unit 201, a first construction unit 202, and a first execution unit 203.

[0057] Among them, the first acquisition unit 201 is used to acquire multi-modal data, where the multi-modal data includes pet behavior data, environmental state data, and device feedback data; The first construction unit 202 is used to construct a dynamic priority matrix based on the multi-modal data, and generate a task queue according to the dynamic priority matrix, where the task queue includes a plurality of rearing tasks, and the dynamic priority matrix is used to represent the priority of the pet's rearing needs; The first execution unit 203 is used to control the pet rearing device to sequentially execute the rearing tasks according to the arrangement order in the task queue.

[0058] In some embodiments, such as this embodiment, the first construction unit 202 further includes a first confirmation unit, a second confirmation unit, and a first sorting unit.

[0059] Among them, the first confirmation unit is used to confirm the pet's rearing needs according to the multi-modal data, and confirm the basic weight corresponding to the rearing needs according to a preset grading standard; The second confirmation unit is used to confirm the adjustment weight of the rearing needs according to the multi-modal data, and confirm the final weight based on the adjustment weight and the basic weight; The first sorting unit is used to sort the rearing needs according to the final weight to obtain the dynamic priority matrix.

[0060] In some embodiments, such as this embodiment, the first confirmation unit further includes a second acquisition unit, a third confirmation unit, a fourth confirmation unit, a fifth confirmation unit, and a sixth confirmation unit.

[0061] Among them, the second acquisition unit is configured to acquire the pet behavior data, the environmental status data, and the device feedback data, and confirm the rearing needs based on the pet behavior data, the environmental status data, and the device feedback data; The third confirmation unit is configured to confirm that the pet has an urgent need if it is confirmed through the pet behavior data that the pet has a vomiting behavior and it is confirmed through the environmental status data that there is an abnormal smell; The fourth confirmation unit is configured to confirm that the pet has an eating need if it is confirmed through the pet behavior data that the pet is in front of the food bowl and it is confirmed through the device feedback data that the weight of the food bowl is less than a preset threshold; The fifth confirmation unit is configured to confirm that the pet has a rest need if it is confirmed through the pet behavior data that the posture of the pet is a curled lying posture and the eyes are in a closed state; The sixth confirmation unit is configured to confirm that the pet has a playing need if it is confirmed through the pet behavior data that the pet has a jumping behavior or a chasing behavior.

[0062] In some embodiments, such as this embodiment, the first confirmation unit further includes a third acquisition unit, a seventh confirmation unit, an eighth confirmation unit, a ninth confirmation unit, and a tenth confirmation unit.

[0063] Among them, the third acquisition unit is configured to acquire all the rearing needs and confirm the category of each rearing need; The seventh confirmation unit is configured to, if the rearing need is the urgent need, confirm that the basic weight of the urgent need is the first weight according to a preset grading standard; The eighth confirmation unit is configured to, if the rearing need is the eating need, confirm that the basic weight of the eating need is the second weight according to a preset grading standard; The ninth confirmation unit is configured to, if the rearing need is the rest need, confirm that the basic weight of the rest need is the third weight according to a preset grading standard; The tenth confirmation unit is configured to, if the rearing need is the playing need, confirm that the basic weight of the playing need is the fourth weight according to a preset grading standard.

[0064] In some embodiments, such as this embodiment, the second confirmation unit further includes a fourth acquisition unit, a fifth acquisition unit, a sixth acquisition unit, and a seventh acquisition unit.

[0065] Among them, the fourth acquisition unit is configured to acquire all the rearing needs and confirm the category of each rearing need; A fifth acquisition unit, configured to, if the care need is the feeding need, acquire a temperature factor, an activity factor, and a weight factor, and confirm an adjustment weight of the feeding need based on the temperature factor, the activity factor, and the weight factor; A sixth acquisition unit, configured to, if the care need is the rest need, acquire a humidity factor and an age factor, and confirm an adjustment weight of the rest need based on the humidity factor and the age factor; A seventh acquisition unit, configured to, if the care need is the play need, acquire the temperature factor and the activity factor, and confirm an adjustment weight of the play need based on the temperature factor and the activity factor.

[0066] In some embodiments, such as this embodiment, the multi-modal perception-based control device 200 further includes a calculation unit.

[0067] Wherein, the calculation unit is configured to, if the care need is the feeding need, confirm a feeding amount according to a preset feeding amount formula, where the preset feeding amount formula is: Q = [Q base (1 + k·Δt)−C·Wremain]p; Q is the feeding amount, Q base is the reference feeding amount, Δt is the time difference from the last feeding, Wremain is the remaining amount in the food bowl, k and C are adaptive coefficients, and p is a food intake compensation coefficient.

[0068] In some embodiments, such as this embodiment, the multi-modal perception-based control device 200 further includes a detection unit and a generation unit.

[0069] Wherein, the detection unit is configured to, if a new care need is detected, confirm a basic weight and an adjustment weight of the new care need to obtain a final weight of the new care need; The generation unit is configured to adjust the dynamic priority matrix based on the final weight of the new care need to generate a new dynamic priority matrix.

[0070] It should be noted that those skilled in the art can clearly understand that the specific implementation processes of the above multi-modal perception-based control device and each unit can refer to the corresponding descriptions in the foregoing method embodiments. For the convenience and brevity of description, they will not be elaborated here.

[0071] The above multi-modal perception-based control device can be implemented in the form of a computer program, and the computer program can run on a Figure 8 pet care device as shown.

[0072] Please refer toFigure 8 , Figure 8 is a schematic block diagram of a pet rearing device provided by an embodiment of the present application. It can be a terminal or a server. Among them, the terminal can be an electronic device with a communication function such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a personal digital assistant, and a wearable device. The server can be an independent server or a server cluster composed of multiple servers.

[0073] Refer to Figure 8 , the pet rearing device 300 includes a processor 302, a memory, and an interface 305 connected through a system bus 301. Among them, the memory can include a non-volatile storage medium 303 and an internal memory 304.

[0074] The non-volatile storage medium 303 can store an operating system 3031 and a computer program 3032. When the computer program 3032 is executed, the processor 302 can be made to execute a control method based on multi-modal perception.

[0075] The processor 302 is used to provide computing and control capabilities to support the operation of the entire pet rearing device 300.

[0076] The internal memory 304 provides an environment for the operation of the computer program 3032 in the non-volatile storage medium 303. When the computer program 3032 is executed by the processor 302, the processor 302 can be made to execute a control method based on multi-modal perception.

[0077] The interface 305 is used to communicate with other devices. Those skilled in the art can understand that Figure 8 the structure shown in

[0078] It should be understood that in the embodiments of the present application, the processor 302 may be a central processing unit (CPU), and the processor 302 may also be other general-purpose processors, digital signal processors (FSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0079] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a storage medium, and the storage medium is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0080] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program. When the computer program is executed by a processor, any embodiment of the above control method based on multi-modal perception is implemented.

[0081] The storage medium may be various computer-readable storage media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disc that can store program codes.

[0082] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0083] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0084] The steps in the method embodiments of the present invention can be adjusted, combined, and deleted according to actual needs. The units in the device embodiments of the present invention can be combined, divided, and deleted according to actual needs. In addition, each functional unit in various embodiments of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0085] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a pet rearing device to execute all or part of the steps of the methods described in various embodiments of the present invention.

[0086] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0087] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, provided that these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these changes and modifications.

[0088] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A control method based on multi-modal perception, characterized in that, Applied to a pet rearing device, the method includes: Obtain multimodal data, where the multimodal data includes pet behavior data, environmental status data, and device feedback data; Construct a dynamic priority matrix based on the multimodal data, and generate a task queue according to the dynamic priority matrix, where the task queue includes multiple rearing tasks, and the dynamic priority matrix is used to represent the priority of the pet's rearing needs; Control the pet rearing device to sequentially execute the rearing tasks according to the arrangement order in the task queue.

2. The method according to claim 1, wherein The step of constructing a dynamic priority matrix based on the multimodal data includes: Confirm the pet's rearing needs according to the multimodal data, and confirm the basic weight corresponding to the rearing needs according to a preset grading standard; Confirm the adjustment weight of the rearing needs according to the multimodal data, and confirm the final weight based on the adjustment weight and the basic weight; Sort the rearing needs according to the final weight to obtain the dynamic priority matrix.

3. The method according to claim 2, characterized in that, The step of confirming the pet's rearing needs according to the multimodal data includes: Obtain the pet behavior data, the environmental status data, and the device feedback data, and confirm the rearing needs based on the pet behavior data, the environmental status data, and the device feedback data; If it is confirmed through the pet behavior data that the pet has a vomiting behavior and it is confirmed through the environmental status data that there is an abnormal smell, then confirm that the pet has an urgent need; If it is confirmed through the pet behavior data that the pet is in front of the food bowl and it is confirmed through the device feedback data that the weight of the food bowl is less than a preset threshold, then confirm that the pet has an eating need; If it is confirmed through the pet behavior data that the pet's posture is a curled lying position and the eyes are in a closed state, then confirm that the pet has a rest need; If it is confirmed through the pet behavior data that the pet has a jumping behavior or a chasing behavior, then confirm that the pet has a playing need.

4. The method according to claim 3, characterized in that, The step of confirming the basic weight corresponding to the rearing needs according to a preset grading standard includes: Obtain all the rearing needs, and confirm the category of each rearing need; If the rearing need is the urgent need, then confirm the basic weight of the urgent need as the first weight according to a preset grading standard; If the rearing need is the eating need, then confirm the basic weight of the eating need as the second weight according to a preset grading standard; If the rearing need is the rest need, then confirm the basic weight of the rest need as the third weight according to a preset grading standard; If the rearing need is the playing need, then confirm the basic weight of the playing need as the fourth weight according to a preset grading standard.

5. The method according to claim 3, wherein The step of confirming the adjustment weight of the rearing needs according to the multimodal data includes: Obtain all the rearing needs, and confirm the category of each rearing need; If the rearing need is the eating need, then obtain a temperature factor, an activity level factor, and a weight factor, and confirm the adjustment weight of the eating need based on the temperature factor, the activity level factor, and the weight factor; If the care requirement is the rest requirement, obtain the humidity factor and the age factor, and confirm the adjustment weight of the rest requirement based on the humidity factor and the age factor; If the care requirement is the play requirement, obtain the temperature factor and the activity level factor, and confirm the adjustment weight of the play requirement based on the temperature factor and the activity level factor.

6. The method according to claim 3, wherein The method further includes: If the care requirement is the feeding requirement, confirm the feeding amount according to a preset feeding amount formula, where the preset feeding amount formula is: Q = [Q base (1 + k·Δt)−C·Wremain]p; Q is the feeding amount, Q base is the reference feeding amount, Δt is the time difference from the last feeding, Wremain is the remaining amount in the food bowl, k and C are adaptive coefficients, and p is the food intake compensation coefficient.

7. The method according to claim 2, characterized in that, The method further includes: If a new care requirement is detected, confirm the basic weight and the adjustment weight of the new care requirement to obtain the final weight of the new care requirement; Adjust the dynamic priority matrix based on the final weight of the new care requirement to generate a new dynamic priority matrix.

8. A control device based on multimodal perception, characterized in that, Applied to a pet care device, the device includes: A first acquisition unit, configured to acquire multimodal data, where the multimodal data includes pet behavior data, environmental state data, and device feedback data; A first construction unit, configured to construct a dynamic priority matrix based on the multimodal data and generate a task queue according to the dynamic priority matrix, where the task queue includes multiple care tasks, and the dynamic priority matrix is used to represent the priority of the pet's care requirements; A first execution unit, configured to control the pet care device to sequentially execute the care tasks according to the arrangement order in the task queue.

9. A pet rearing device, characterized in that, The pet care device includes a memory and a processor connected to the memory; the memory is used to store a computer program; the processor is used to run the computer program stored in the memory to execute the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1-7 can be implemented.

Images