Stove detection method and device, inspection robot and storage medium

Abstract

The embodiment of the invention provides a cooker detection method and device, an inspection robot and a storage medium, and the method comprises the steps: responding to the movement of the inspection robot to a cooker, and controlling a detection system to collect an induction information set of the cooker; wherein the sensing information set comprises at least two kinds of sensing information; determining a first detection result of the kitchen range based on the induction information set of the kitchen range by using the detection model; under the condition that the target detection result of the stove is the first target detection result, the inspection robot is controlled to execute target operation; wherein the target detection result is determined based on the first detection result, and the first target detection result represents that the cooker has potential safety hazards.

Description

Technical Field

[0001] The present disclosure relates to, but is not limited to, the field of robotics, and particularly relates to a detection method and device for a cooking appliance, an inspection robot, and a storage medium. Background Art

[0002] In the related art, during the process of home inspection by an inspection robot, safety hazards of cooking appliances, water usage, electricity usage, etc. in the kitchen are usually detected. Currently, the cooking appliances are mainly detected by temperature, which has problems such as a high misjudgment rate and poor compatibility. Summary of the Invention

[0003] Embodiments of the present disclosure provide a detection method and device for a cooking appliance, an inspection robot, a storage medium, and a computer program product.

[0004] The technical solution of the embodiments of the present disclosure is implemented as follows:

[0005] Embodiments of the present disclosure provide a detection method for a cooking appliance, which is applied to an inspection robot with a detection system, and includes:

[0006] In response to the inspection robot moving to the cooking appliance, controlling the detection system to collect an induction information set of the cooking appliance; wherein, the induction information set includes at least two types of induction information;

[0007] Using a detection model, based on the induction information set of the cooking appliance, determining a first detection result of the cooking appliance;

[0008] In the case that the target detection result of the cooking appliance is a first target detection result, controlling the inspection robot to perform a target operation; wherein, the target detection result is determined based on the first detection result, and the first target detection result indicates that the cooking appliance has a safety hazard.

[0009] Embodiments of the present disclosure provide a detection device for a cooking appliance, which is applied to an inspection robot with a detection system, and includes:

[0010] An acquisition module, configured to, in response to the inspection robot moving to the cooking appliance, control the detection system to collect an induction information set of the cooking appliance; wherein, the induction information set includes at least two types of induction information;

[0011] A determination module, configured to use a detection model to determine a first detection result of the cooking appliance based on the induction information set of the cooking appliance;

[0012] An operation module, configured to, in the case that the target detection result of the cooking appliance is a first target detection result, control the inspection robot to perform a target operation; wherein, the target detection result is determined based on the first detection result, and the first target detection result indicates that the cooking appliance has a safety hazard.

[0013] An embodiment of the present disclosure provides an inspection robot, which includes a processor and a memory. The memory stores a computer program that can run on the processor. When the processor executes the computer program, the above method is implemented.

[0014] An embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above method is implemented.

[0015] An embodiment of the present disclosure provides a computer program product. The computer program product includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, the above method is implemented.

[0016] In the embodiment of the present disclosure, first, the detection result of the cooking appliance is comprehensively determined by various sensing information collected by the detection system in the inspection robot. Compared with the related art where only temperature is used to determine the detection result of the cooking appliance, the possibilities of missed judgment and misjudgment are reduced, and the accuracy and reliability of the detection result are improved. At the same time, since the inspection robot can collect various sensing information, the applicability and compatibility of the inspection robot are improved. Second, the detection model is used to detect the cooking appliance, realizing the perception and understanding of multiple modal data and improving the detection efficiency. Finally, when the detection result indicates that there is a safety hazard in the cooking appliance, the safety risk of the cooking appliance is reduced through the target operation, thereby improving the safety guarantee of the kitchen.

[0017] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0019] Figure 1 Schematic diagram of the implementation process of a method for detecting a cooking appliance provided by an embodiment of the present disclosure Figure 1 ;

[0020] Figure 2 Schematic diagram of the implementation process of a method for detecting a cooking appliance provided by an embodiment of the present disclosure Figure 2 ;

[0021] Figure 3 Schematic diagram of the implementation process of a method for detecting a cooking appliance provided by an embodiment of the present disclosure Figure 3 ;

[0022] Figure 4 Schematic diagram of the implementation process of a method for detecting a cooking appliance provided by an embodiment of the present disclosure Figure 4 ;

[0023] Figure 5 A schematic diagram of the composition structure of a detection device for a cooking appliance provided by an embodiment of the present disclosure;

[0024] Figure 6 A schematic diagram of the composition structure of a patrol robot provided by an embodiment of the present disclosure. Specific embodiments

[0025] In order to make the objectives, technical solutions, and advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be construed as limiting the present disclosure. All other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present disclosure.

[0026] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0027] In the following description, the terms "first / second / third" are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present disclosure belongs. The terms used herein are only for the purpose of describing the embodiments of the present disclosure and are not intended to limit the present disclosure.

[0029] Next, the technical solutions in the embodiments of the present disclosure will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present disclosure.

[0030] Figure 1 A schematic diagram of the implementation process of a detection method for a cooking appliance provided by an embodiment of the present disclosure Figure 1 , which is applied to a patrol robot having a detection system. As Figure 1 shown, the detection method includes steps S11 to S13, where:

[0031] Step S11, in response to the patrol robot moving to the cooking appliance, controlling the detection system to collect an induction information set of the cooking appliance; wherein, the induction information set includes at least two types of induction information.

[0032] Here, the patrol robot can be used for home patrol. In some embodiments, the patrol robot includes a robot for taking care of special objects such as the elderly and children at home. During implementation, the patrol robot can perform mobile patrols at home. For example, it can patrol indoor areas such as the kitchen, bedroom, and living room, or outdoor areas such as the terrace.

[0033] The detection system can be any suitable system capable of realizing information detection. The detection system can include but is not limited to an infrared acquisition unit, a visual acquisition unit, a temperature acquisition unit, etc. The infrared detection unit can be any suitable unit capable of realizing infrared information acquisition. The visual acquisition unit can be any suitable unit capable of performing visual information acquisition, such as a camera, a camera, etc. The temperature acquisition unit can be any suitable unit capable of realizing temperature acquisition, such as a temperature sensor, etc. The above-mentioned suitable units refer to units whose specifications, accuracy, acquisition or processing capabilities, etc. match the current usage scenario. The usage scenario can be determined based on information such as the purpose of the patrol robot and the care object.

[0034] The sensed information set can include at least two of but is not limited to infrared information, visual information, temperature information, etc. For example, the sensed information set includes infrared information and visual information. Another example is that the sensed information set includes infrared information, visual information, and temperature information. The visual information can be any suitable visual information, such as an image, a video, etc. The suitable visual information refers to visual information that is adapted to the current usage scenario, patrol area, etc. In some embodiments, different usage scenarios can correspond to the same or different visual information. For example, in usage scenario A1, the visual information can refer to an image; in usage scenario A2, the visual information can be a video; in usage scenario A3, the visual information can be an image and / or a video. In some embodiments, different patrol areas in the same usage scenario can correspond to the same or different visual information. For example, in a certain usage scenario, the visual information of patrol area B1 can be an image, the visual information of patrol area B2 can be a video, and the visual information of patrol area B3 can be an image + video.

[0035] During implementation, those skilled in the art can independently set the relationship among the usage scenario, patrol area, and visual information according to actual needs.

[0036] The sensed information set is obtained based on methods such as a collection instruction, a collection period, etc. For example, the detection system receives a collection instruction sent by the patrol robot and collects the sensed information set according to the collection instruction. Another example is that the detection system periodically collects the sensed information set according to the collection period.

[0037] Step S12: Use the detection model to determine the first detection result of the cooking appliance based on the sensed information set of the cooking appliance.

[0038] Here, the detection model can be any suitable model capable of implementing this function. For example, a stove fire recognition model, a large language model, a vision-language model, etc. The stove fire recognition model is obtained by pre-training with a training sample set. Through this stove fire recognition model, it is possible to determine whether there is a cooking appliance, the position of the stove fire of the cooking appliance, the intensity of the stove fire, etc.

[0039] In some embodiments, the same or different detection models can be used for different sensing information. For example, for visual information, infrared information, etc., a large language model, a vision-language model, etc. can be used as the detection model. For another example, for temperature information, infrared information, etc., the stove fire recognition model can be used as the detection model.

[0040] The first detection result is used to characterize whether the cooking appliance is turned off. The first detection result can include, but is not limited to, a detection result indicating that the cooking appliance is not turned off, a detection result indicating that the cooking appliance is turned off, etc.

[0041] The determination method of the first detection result can be any suitable method.

[0042] In some embodiments, the sensing information set can be input into the detection model to obtain the first detection result. The detection model has the ability to fuse, perceive, and understand multi-modal data.

[0043] In some embodiments, the first detection result can be determined according to the corresponding detection results determined by the detection model based on each sensing information.

[0044] For example, when the sensing information set includes visual information and infrared information, the first detection result can be comprehensively determined according to the visual detection result determined by the detection model based on the visual information and the infrared detection result determined by the detection model based on the infrared information. Among them, the visual detection result (including other visual detection results mentioned later) can include, but is not limited to, whether there is a cooking utensil on the cooking appliance, the opening degree of the valve of the cooking appliance, etc. The infrared detection result (including other infrared detection results mentioned later) can include, but is not limited to, the position of the stove fire of the cooking appliance, the intensity of the stove fire, etc. For example, when the visual detection result indicates that the valve of the cooking appliance is not closed or the infrared detection result indicates that the intensity of the stove fire exceeds the set intensity threshold, the detection result indicating that the cooking appliance is not turned off is used as the first detection result; otherwise, the detection result indicating that the cooking appliance is turned off is used as the first detection result.

[0045] Step S13, when the target detection result of the cooking appliance is the first target detection result, control the patrol robot to perform the target operation; wherein, the target detection result is determined based on the first detection result, and the first target detection result indicates that there is a safety hazard with the cooking appliance.

[0046] Here, the target detection result may include, but is not limited to, a first target detection result indicating a safety hazard of the cooking appliance, a second target detection result indicating no safety hazard of the cooking appliance, etc.

[0047] The target detection result can be determined in any suitable way. For example, the target detection result is determined according to the first detection result. For instance, when the first detection result indicates that the cooking appliance is not turned off, the first target detection result is taken as the target detection result; otherwise, the second target detection result is taken as the target detection result. Another example is that the target detection result is determined according to the first detection result and the second detection result, where the second detection result indicates whether there is someone within the operation range corresponding to the cooking appliance. The operation range can be any suitable range, such as in front of the cooking appliance, inside the kitchen, within a distance threshold from the cooking appliance, etc. The distance threshold can be any suitable value, such as 1 meter, 0.5 meter, etc. For example, when the first detection result indicates that the cooking appliance is not turned off and the second detection result indicates that there is no one within the operation range corresponding to the cooking appliance, the first target detection result is taken as the target detection result. Another example is that when the first detection result indicates that the cooking appliance is turned off or the second detection result indicates that there is someone within the operation range corresponding to the cooking appliance, the second target detection result is taken as the target detection result. Still another example is that when the first detection result indicates that the cooking appliance is not turned off and the second detection result indicates that the person within the operation range corresponding to the cooking appliance does not have the ability to operate the cooking appliance, the first target detection result is taken as the target detection result.

[0048] The target operation may include, but is not limited to, at least one of a voice prompt operation, a cooking appliance shutdown operation, a remote assistance operation, etc. The cooking appliance shutdown operation refers to closing the valve of the cooking appliance or cutting off the gas supply to the cooking appliance. The remote assistance operation may include, but is not limited to, making a call, sending a message, etc.

[0049] The target operation can be determined in any suitable way. In some embodiments, the target operation can be determined according to the urgency level determined by the target detection result. The urgency level may include, but is not limited to, general, urgent, very urgent, etc. In implementation, different urgency levels may correspond to the same or different target operations.

[0050] For example, when the urgency level is general, the voice prompt operation can be taken as the target operation; when the urgency level is urgent, the voice prompt operation and the cooking appliance shutdown operation can be taken as the target operation; when the urgency level is very urgent, the voice prompt operation, the cooking appliance shutdown operation, and the remote assistance operation can all be taken as the target operation.

[0051] In some embodiments, the target operation can be determined based on the information about people leaving the cooking appliance. The information about people leaving can include, but is not limited to, the leaving time, the person leaving, etc. For example, when the information about people leaving meets the leaving condition, the voice prompt operation is taken as the target operation; when the information about people leaving does not meet the leaving condition, the operation of turning off the cooking appliance and / or the remote help operation is taken as the target operation. The leaving condition can be any suitable condition and can be set according to the information about people leaving. For example, when the information about people leaving includes the leaving time, the leaving condition can be not exceeding the set leaving duration threshold, and the leaving duration threshold can be any suitable duration, such as 30 seconds, 1 minute, 5 minutes, etc. Another example is that when the information about people leaving includes the person leaving, the leaving condition can be that the person leaving has the ability to operate the cooking appliance.

[0052] In some embodiments, the inspection robot further includes a prompt module, and the prompt module can be any suitable module capable of performing voice prompts. For example, an alarm, a speaker, etc. During implementation, the voice prompt operation can be performed by controlling the prompt module, and the voice prompt operation is used to remind the user to turn off the cooking appliance.

[0053] In some embodiments, the inspection robot further includes an operating component, and the operating component can be any suitable component capable of turning off the cooking appliance. For example, a manipulator, an actuator connected to the end of the robotic arm. The actuator can include, but is not limited to, a suction cup, a gripper, etc. The actuator is connected to different operating mechanisms to achieve different operations, where the operating mechanism can be any suitable tool, device, etc. During implementation, the operation of turning off the cooking appliance can be performed by the operating component. In some embodiments, the inspection robot can include a manipulator and / or an actuator.

[0054] In the embodiments of the present disclosure, first, the detection result of the cooking appliance is comprehensively determined through various sensing information collected by the detection system in the inspection robot. Compared with the related art where only temperature is used to determine the detection result of the cooking appliance, the possibilities of missed judgment and misjudgment are reduced, and the accuracy and reliability of the detection result are improved. At the same time, since the inspection robot can collect various sensing information, the applicability and compatibility of the inspection robot are improved; second, the detection model is used to detect the cooking appliance, realizing the perception and understanding of multi-modal data and improving the detection efficiency; finally, when the detection result indicates that there is a safety hazard in the cooking appliance, the safety risk of the cooking appliance is reduced through the target operation, thereby improving the safety guarantee of the kitchen.

[0055] In some embodiments, the "controlling the inspection robot to perform the target operation" in step S13 includes steps S131 to S134, where:

[0056] Step S131, determine the target operation.

[0057] Here, the target operation includes at least one of the following: voice prompt operation, stove shutdown operation, and remote assistance operation. The determination method of the target operation can be any suitable method.

[0058] In some embodiments, the target operation can be determined according to the urgency level determined by the target detection result. Different urgency levels can correspond to the same or different target operations.

[0059] In some embodiments, the target operation can be determined according to the information about the person leaving the stove.

[0060] In some embodiments, in the case where the operator does not respond or is unable to respond, the remote assistance operation can be used as the target operation. In some embodiments, in the case where the inspection robot is unable to perform the stove shutdown operation, the remote assistance operation can be used as the target operation.

[0061] In some embodiments, step S131 includes step S1311 and / or step S1312, where:

[0062] Step S1311: Use the voice prompt operation, stove shutdown operation, and / or remote assistance operation as the target operation.

[0063] Here, at least one of the voice prompt operation, stove shutdown operation, remote assistance operation, etc. can be used as the target operation according to the urgency level, configuration information, etc. Among them, the configuration information includes the target operation.

[0064] Step S1312: When the information about the person leaving the stove meets the leaving condition, use the voice prompt operation as the target operation; when the information about the person leaving the stove does not meet the leaving condition, use the stove shutdown operation and / or remote assistance operation as the target operation.

[0065] Here, the information about the person leaving the stove can include, but is not limited to, the leaving time, the person leaving, etc.

[0066] The leaving condition can be any suitable condition, and the leaving condition can be set according to the information about the person leaving the stove. For example, when the information about the person leaving the stove includes the leaving time, the leaving condition can be not exceeding the set leaving duration threshold. Another example is that when the information about the person leaving the stove includes the person leaving, the leaving condition can be that the person leaving has the ability to operate the stove.

[0067] In the embodiments of the present disclosure, the target operation is determined in multiple ways such as the urgency level, configuration information, and information about the person leaving the stove, which improves the accuracy and flexibility of the target operation.

[0068] Step S132: When the target operation includes a voice prompt operation, control the prompt module of the inspection robot to perform a voice reminder operation.

[0069] Here, the inspection robot further includes a prompt module, which can be any suitable module capable of performing voice prompts. For example, an alarm, a speaker, etc. In implementation, the prompt module can perform a voice reminder operation according to the voice operation instruction issued by the inspection robot.

[0070] Step S133: When the target operation includes an operation of closing the cooking appliance, based on the valve of the cooking appliance, adjust the operating component of the inspection robot, and control the operating component of the inspection robot to perform the operation of closing the cooking appliance.

[0071] Here, the inspection robot further includes an operating component, which can be any suitable component capable of closing the cooking appliance. For example, a manipulator, an actuator connected to the end of the robotic arm. The actuator is connected to different operating mechanisms to achieve different operations, where the operating mechanism can be any suitable tool, device, etc.

[0072] Different valves can correspond to the same or different operating components. The adjustment of the operating component can include but is not limited to the adjustment of the opening and closing degree of the manipulator, the adjustment of the operating mechanism grasped by the manipulator, the adjustment of the operating mechanism in the actuator, etc. In implementation, by adjusting the operating component to adapt to the valve of the cooking appliance, it is convenient to close the valve of the cooking appliance, thereby realizing the operation of closing the valve. In some implementation manners, the operating component can perform the operation of closing the cooking appliance according to the control instruction issued by the inspection robot.

[0073] Step S134: When the target operation includes a remote assistance operation, control the inspection robot to perform a remote assistance operation.

[0074] Here, the remote assistance operation can include but is not limited to making a call, sending a message, etc. For example, the inspection robot can make a call, send a message, etc. through a mobile phone. Another example is that the inspection robot can remotely send a message through an application.

[0075] In the implementation manner of the present disclosure, by the inspection robot performing the determined target operation, the safety risk of the cooking appliance is reduced, thereby improving the safety guarantee of the kitchen.

[0076] In some implementation manners, the "based on the valve of the cooking appliance, adjust the operating component of the inspection robot" in step S133 includes steps S1331 to S1333, where:

[0077] Step S1331: Determine the operating component of the inspection robot.

[0078] Here, the operating component of the inspection robot includes one of the following: a manipulator, an actuator connected to the end of the robotic arm.

[0079] In the case where the inspection robot only includes a manipulator, the manipulator is used as the operating component.

[0080] In the case where the inspection robot only includes an actuator, the actuator is used as the operating component.

[0081] In the case where the inspection robot includes a manipulator and an actuator, the determination method of the operating component can be any suitable method.

[0082] In some embodiments, the operating component can be determined according to the usage order of the manipulator and the actuator. For example, if the manipulator was used as the operating component last time, then this time the actuator can be used as the operating component.

[0083] In some embodiments, the operating component can be determined according to the object to be operated. The object to be operated can be any suitable object that can be operated. For example, components such as valves that directly control the opening or closing of the stove, tools that can be used to open or close the stove, or auxiliary appliances for repairing the stove, or other objects that can be operated by the robot. In some embodiments, different objects to be operated can correspond to the same or different operating components. For example, for the object to be operated X1, the manipulator can be used as the operating component; for the object to be operated X2, the actuator can be used as the operating component.

[0084] In some embodiments, the operating component can be determined according to the usage scenario, inspection area, etc. In some embodiments, different usage scenarios can correspond to the same or different operating components. For example, in usage scenario A1, the manipulator can be used as the operating component; in usage scenario A2, the actuator can be used as the operating component. In some embodiments, different inspection areas can correspond to the same or different operating components. For example, in a certain usage scenario, if it is inspection area B1 or inspection area B2, the manipulator can be used as the operating component; if it is inspection area B3, the actuator can be used as the operating component.

[0085] Step S1332: In the case where the operating component of the inspection robot includes a manipulator, based on the valve of the stove, adjust the opening and closing degree of the manipulator of the inspection robot, or, based on the valve of the stove, determine a first operating mechanism from at least one operating mechanism, and control the manipulator to grasp the first operating mechanism.

[0086] Here, the opening and closing degree of the manipulator is adjusted according to the specifications of the valve so that the manipulator can grasp the valve. In some embodiments, the specifications of the valve can be determined in real time through the visual information in the sensing information set. In some embodiments, the specifications of the valve are pre-entered.

[0087] The operating mechanism can be any suitable tool, device, etc. The first operating mechanism is a tool adapted to the valve, and the valve can be closed through the adapted tool.

[0088] Step S1333, when the operating component of the inspection robot includes an actuator, based on the valve of the cooking appliance, determine a second operating mechanism from at least one operating mechanism, and adjust the current operating mechanism of the actuator to the second operating mechanism.

[0089] Here, the second operating mechanism is a tool adapted to the valve. The second operating mechanism and the first operating mechanism can be the same or different. In some embodiments, the actuator is connected to the operating mechanism to perform corresponding operations.

[0090] In the embodiments of the present disclosure, the operating component is adjusted according to the valve of the cooking appliance to improve the accuracy and efficiency of the valve closing operation.

[0091] In some embodiments, when the target operation includes the operation of closing the cooking appliance, the method further includes steps S141 to S143, where:

[0092] Step S141, control the detection system to collect the second visual information of the cooking appliance.

[0093] Here, the second visual information can be any suitable visual information, such as images, videos, etc. In implementation, the detection system can collect the second visual information according to the collection instruction issued by the inspection robot.

[0094] Step S142, based on the second visual information, determine the operation result of the cooking appliance.

[0095] Here, the operation result of the cooking appliance can include but is not limited to a first operation result, a second operation result, etc. The first operation result indicates that the valve of the cooking appliance has not been closed, and the second operation result indicates that the valve of the cooking appliance has been closed.

[0096] The operation result can be determined in any suitable way. In some embodiments, the operation result can be determined according to the similarity between the second visual information and the target visual information, and the target visual information can be visual information including the valve being closed. For example, if the similarity between the second visual information and the target visual information exceeds the similarity threshold, the second operation result is taken as the operation result; otherwise, the first operation result is taken as the operation result. In some embodiments, the second visual information can be input into a pre-trained neural network model to obtain the operation result, and the neural network model can be any suitable model capable of implementing this function. In some embodiments, the second visual information can be input into a large model or a vision-language model to obtain the operation result.

[0097] Step S143, when the operation result of the cooking appliance indicates that the valve of the cooking appliance is not closed, control the inspection robot to perform the operation of closing the cooking appliance again.

[0098] Here, if the valve is not closed, at this time, the operation of closing the cooking appliance can be performed again, and the specific implementation of this operation of closing the cooking appliance can refer to the foregoing step S133. In some embodiments, the operation of closing the cooking appliance can be performed multiple times until the valve of the cooking appliance is closed, or the operation of closing the cooking appliance can be performed a preset number of times. After performing the operation of closing the cooking appliance once, steps S141 to S142 can be performed again.

[0099] In the embodiments of the present disclosure, after performing the operation of closing the cooking appliance, the operation result of the cooking appliance is determined according to the collected second visual information to ensure that the valve of the cooking appliance is closed, thereby improving safety.

[0100] In some embodiments, the method further includes step S151 and step S152, where:

[0101] Step S151, when the first detection result indicates that the cooking appliance is not closed, take the first target detection result as the target detection result of the cooking appliance.

[0102] Here, when the cooking appliance is not closed, it indicates that there is a safety hazard for the cooking appliance. At this time, the first target detection result indicating that there is a safety hazard for the cooking appliance can be taken as the target detection result.

[0103] Step S152, when the first detection result indicates that the cooking appliance is closed, take the second target detection result as the target detection result of the cooking appliance.

[0104] Here, when the cooking appliance is closed, it indicates that there is no safety hazard for the cooking appliance. At this time, the second target detection result indicating that there is no safety hazard for the cooking appliance can be taken as the target detection result.

[0105] In the embodiments of the present disclosure, the target detection result of the cooking appliance is determined according to the first detection result, which improves the accuracy of the target detection result and thus enhances the safety.

[0106] Figure 2 Schematic implementation process of a detection method for a cooking appliance provided by an embodiment of the present disclosure Figure 2 which is applied to a patrol robot with a detection system, such as Figure 2 shown, the detection method includes steps S21 to S25, where:

[0107] Step S21, in response to the patrol robot moving to the cooking appliance, control the detection system to collect the induction information set of the cooking appliance; wherein, the induction information set includes infrared information and first visual information.

[0108] Here, the first visual information can be any suitable visual information, for example, images, videos, etc. The manner in which the detection system collects the induction information set can refer to the specific implementation manner of the foregoing step S11.

[0109] Step S22, use the detection model to determine the visual detection result of the cooking appliance based on the first visual information collected by the detection system.

[0110] Here, the detection model can be any suitable model capable of implementing the visual information detection. For example, large models, vision-language models, etc. The visual detection result includes at least one of the following: whether there is a cooking utensil on the cooking appliance, the opening degree of the valve of the cooking appliance.

[0111] The manner of determining the visual detection result can be any suitable manner. In some embodiments, the first visual information can be input into the detection model to obtain the visual detection result. In some embodiments, the first visual information can be subjected to a first preprocessing to obtain third visual information, and the third visual information can be input into the detection model to obtain the visual detection result. The first preprocessing can include, but is not limited to, denoising processing, splicing processing, adjustment processing, etc. Splicing processing refers to splicing multiple visual information. Adjustment processing can refer to adjusting the position, size, etc. of the cooking appliance in the visual information.

[0112] Step S23, use the detection model to determine the infrared detection result of the cooking appliance based on the infrared information collected by the detection system.

[0113] Here, the detection model can be any suitable model capable of implementing the infrared information detection. For example, a stove fire recognition model, large models, vision-language models, etc. The infrared detection result includes at least one of the following: the position of the stove fire of the cooking appliance, the intensity of the stove fire.

[0114] The determination method of the infrared detection result can be any suitable method. In some embodiments, the infrared information can be input into a detection model to obtain the infrared detection result. In some embodiments, the infrared information can be secondarily preprocessed to obtain first infrared information, and the first infrared information can be input into the detection model to obtain the infrared detection result. The secondary preprocessing can include, but is not limited to, denoising processing, splicing processing, etc. Splicing processing refers to splicing multiple infrared information.

[0115] Step S24: Based on the visual detection result of the cooking appliance and the infrared detection result of the cooking appliance, determine the first detection result of the cooking appliance.

[0116] Here, the first detection result can include, but is not limited to, the detection result indicating that the cooking appliance is not turned off, the detection result indicating that the cooking appliance is turned off, etc. The determination method of the first detection result can be any suitable method. In some embodiments, a correspondence relationship can be established in advance among multiple visual detection results, multiple infrared detection results, and multiple first detection results. Then, according to this correspondence relationship, the first detection result that matches both the visual detection result and the infrared detection result can be obtained. In some embodiments, the visual detection result and the infrared detection result can be input into a large model / vision-language model to obtain the first detection result.

[0117] In some embodiments, this step S24 includes step S241 and step S242, where:

[0118] Step S241: If the visual detection result of the cooking appliance indicates that the valve of the cooking appliance is not turned off or the infrared detection result of the cooking appliance indicates that the intensity of the stove fire exceeds the set intensity threshold, then use the detection result indicating that the cooking appliance is not turned off as the first detection result of the cooking appliance.

[0119] Here, the intensity threshold can be any suitable threshold. During implementation, if the valve of the cooking appliance is not turned off or the intensity of the stove fire exceeds this intensity threshold, it is considered that the cooking appliance is not turned off. Then, the detection result indicating that the cooking appliance is not turned off can be used as the first detection result.

[0120] Step S242: If the visual detection result of the cooking appliance indicates that the valve of the cooking appliance is turned off and the infrared detection result of the cooking appliance indicates that the intensity of the stove fire does not exceed the set intensity threshold, then use the detection result indicating that the cooking appliance is turned off as the first detection result of the cooking appliance.

[0121] Here, if the valve of the cooking appliance is turned off and the intensity of the stove fire does not exceed this intensity threshold, it is considered that the cooking appliance is turned off. Then, the detection result indicating that the cooking appliance is turned off can be used as the first detection result.

[0122] In the embodiments of the present disclosure, a first detection result indicating whether the cooking appliance is turned off is comprehensively determined based on the valve of the cooking appliance and the intensity of the cooking flame, so as to improve the accuracy and reliability of the first detection result and reduce the possibility of misjudgment.

[0123] Step S25: When the target detection result of the cooking appliance is the first target detection result, control the patrol robot to perform a target operation; wherein, the target detection result is determined based on the first detection result, and the first target detection result indicates that there is a safety hazard with the cooking appliance.

[0124] Here, this step S25 corresponds to the foregoing step S13. When implemented, reference may be made to the specific implementation manner of the foregoing step S13.

[0125] In the embodiments of the present disclosure, a first detection result indicating whether the cooking appliance is turned off is comprehensively determined based on the visual detection result corresponding to the visual information and the infrared detection result corresponding to the infrared information. This not only improves the accuracy and reliability of the first detection result and reduces the possibility of misjudgment, but also shortens the detection time and improves the detection efficiency because a detection model is used to detect the information.

[0126] In some embodiments, this step S22 includes step S221 and step S222, where:

[0127] Step S221: Determine the first prompt information corresponding to the cooking appliance.

[0128] Here, the first prompt information may be any suitable prompt information. The first prompt information is adapted to the visual detection result. For example, the first prompt information may include, but is not limited to, the position of the cooking appliance, whether there is a cooking container on the cooking appliance, the opening degree of the valve of the cooking appliance, etc.

[0129] The first prompt information can be determined in any suitable manner. In some embodiments, the first prompt information can be determined based on at least one of the type of the cooking appliance, the historical prompt information set, the first visual information, the visual detection area corresponding to the first visual information, etc. The type of the cooking appliance may include, but is not limited to, gas stoves, coal gas stoves, etc., and different types may correspond to the same or different first prompt information. The historical prompt information set includes at least one historical first prompt information. When implemented, a certain historical first prompt information may be used as the first prompt information, or at least two historical first prompt information may be combined to obtain the first prompt information. The visual detection area may be any area in the cooking appliance, and different first visual information may correspond to the same or different visual detection areas.

[0130] Step S222: Input the first visual information and the first prompt information collected by the detection system into the detection model to obtain the visual detection result of the cooking appliance; wherein, the detection model includes one of the following: the first large text-image model, the first vision-language model.

[0131] Here, the first large text-image model can be any suitable large text-image model. The large text-image model (including the first large text-image model and the large text-image models mentioned hereinafter) is a deep learning model that combines two modalities of information, i.e., image and text. For example, the first large text-image model can be ChatGPT. The first vision-language model can be any suitable vision-language model (VLM). A VLM (including the first vision-language model and other vision-language models mentioned hereinafter) is a multi-model, generative artificial intelligence model that can understand and process videos, images, and text. For example, the first vision-language model can be DeepSeek, CLIP, etc.

[0132] In the embodiments of the present disclosure, a large text-image model, a vision-language model, etc. are used to determine the visual detection result according to the prompt information and the visual information, which shortens the determination duration of the visual detection result. At the same time, compared with using a trained model for visual detection, since the present disclosure does not require preparing a training data set and a training model, the cost is reduced, and the possibility of misjudgment and limitations caused by using the model for visual detection is reduced, achieving the purpose of improving the reliability, accuracy, and flexibility of visual detection.

[0133] In some embodiments, this step S221 includes at least one of steps S2211 to S2214, wherein:

[0134] Step S2211: Determine the first prompt information based on the type of the cooking appliance.

[0135] Here, the determination method of the type of the cooking appliance can be any suitable method. For example, the type of the cooking appliance is pre-entered. Or, for example, the type of the cooking appliance is determined based on the first visual information.

[0136] Different types can correspond to the same or different first prompt information. In some embodiments, a correspondence relationship between multiple types and multiple first prompt information can be established in advance. According to this correspondence relationship, the first prompt information corresponding to the type of the cooking appliance can be obtained. In some embodiments, the type of the cooking appliance can be input into the prompt generation model to obtain the first prompt information. The prompt generation model can be any suitable neural network model with the function of generating prompt information.

[0137] Step S2212: Determine the first prompt information based on the historical prompt information set of the cooking appliance.

[0138] Here, the historical prompt information set includes at least one historical first prompt information. The determination method of this first prompt information can be any suitable method. For example, randomly select a certain historical first prompt information as this first prompt information. Another example is to select a certain historical first prompt information as this first prompt information according to the usage order. Still another example is to select a specified historical first prompt information as this first prompt information. Yet another example is to merge multiple historical first prompt information to obtain this first prompt information.

[0139] Step S2213: Determine the first prompt information based on the first visual information of the cooking appliance.

[0140] Here, different first visual information can correspond to the same or different first prompt information. In some embodiments, a correspondence relationship between multiple visual information and multiple first prompt information can be established in advance. According to this correspondence relationship, the first prompt information corresponding to this visual information can be obtained. In some embodiments, this first visual information can be input into a prompt generation model to generate this first prompt information.

[0141] Step S2214: Determine the first prompt information based on the visual detection area corresponding to the first visual information of the cooking appliance.

[0142] Here, the visual detection area can be any area in the cooking appliance. Different first visual information can correspond to the same or different visual detection areas. Different visual detection areas can correspond to the same or different first prompt information. In some embodiments, a correspondence relationship between multiple visual detection areas and multiple first prompt information can be established in advance. According to this correspondence relationship, the first prompt information corresponding to this visual detection area can be obtained. In some embodiments, this visual detection area can be input into a prompt generation model to obtain this first prompt information.

[0143] In the embodiments of the present disclosure, the first prompt information is determined in multiple ways such as the type of the cooking appliance, the historical prompt information set, the first visual information, and the visual detection area, which improves the flexibility and comprehensiveness of the first prompt information.

[0144] In some embodiments, the first visual information collected by the detection system includes a first visual image from a first perspective and a second visual image from a second perspective; this step S22 includes steps S261 to S263, where:

[0145] Step S261: Use a detection model to determine the first visual detection result of the cooking appliance based on the first visual image.

[0146] Here, the first perspective and the second perspective are different. In some embodiments, the detection system includes a first camera located at the head of the inspection robot and a second camera located on the arm of the inspection robot. Then, the first visual image can be captured by the first camera, and the second visual image can be captured by the second camera. The capture time of the first visual image and the capture time of the second visual image can be the same or different. The visual image (including the first visual image and the second visual image) at least includes the cooking stove.

[0147] The detection model can include, but is not limited to, a first text-image large model, a first vision-language model, etc.

[0148] The first visual detection result includes at least one of the following: whether a cooking utensil is placed on the cooking stove, the opening degree of the valve of the cooking stove. In implementation, the first visual image and the first prompt information corresponding to the first visual image are input into the detection model, and then the first visual detection result can be obtained. The determination method of the first prompt information corresponding to the first visual image can refer to the specific implementation manner of the foregoing step S221.

[0149] Step S262: Use the detection model to determine the second visual detection result of the cooking stove based on the second visual image.

[0150] Here, the detection model can include, but is not limited to, a first text-image large model, a first vision-language model, etc. The second visual detection result includes at least one of the following: whether a cooking utensil is placed on the cooking stove, the opening degree of the valve of the cooking stove. In implementation, the second visual image and the first prompt information corresponding to the second visual image are input into the detection model, and then the second visual detection result can be obtained. The determination method of the first prompt information corresponding to the second visual image can refer to the specific implementation manner of the foregoing step S221.

[0151] Step S263: Determine the visual detection result of the cooking stove based on the first visual detection result and the second visual detection result.

[0152] Here, the determination method of the visual detection result can be any suitable method. For example, when the first visual detection result and the second visual detection result are the same, the first visual detection result or the second visual detection result is used as the visual detection result. For another example, when the first visual detection result and the second visual detection result are different, the first visual detection result is used as the visual detection result, or the first visual detection result is adjusted based on the second visual detection result, and the adjusted first visual detection result is used as the visual detection result.

[0153] In the embodiments of the present disclosure, the visual detection result of the cooking stove is comprehensively determined according to the visual detection results determined from the visual images of multiple perspectives, which improves the accuracy and reliability of the visual detection result.

[0154] In some embodiments, the first visual information collected by the detection system includes a first visual image from a first perspective and a second visual image from a second perspective; this step S22 includes steps S271 to S272, where:

[0155] Step S271: Based on the first visual image and the second visual image, determine the target visual image.

[0156] Here, the target visual image can be determined in any suitable way. For example, according to the quality, clarity, integrity, etc. of the first visual image and the second visual image, the first visual image or the second visual image can be used as the target visual image. For another example, the first visual image and the second visual image can be stitched together to form the target visual image. Still for another example, the cooking appliance in the second visual image can be used to adjust the cooking appliance in the first visual image to obtain the target visual image.

[0157] Step S272: Use the detection model to determine the visual detection result of the cooking appliance based on the target visual image.

[0158] Here, the detection model can include but is not limited to text and image large models, vision-language models, etc. When implemented, the specific implementation of determining the visual detection result can refer to the specific implementation manners of the aforementioned step S261 or step S262.

[0159] In the embodiments of the present disclosure, using the target visual image determined according to the visual images from multiple perspectives by the detection model to comprehensively determine the visual detection result of the cooking appliance improves the accuracy and reliability of the visual detection result.

[0160] In some embodiments, this step S271 includes step S2711 and / or step S2712, where:

[0161] Step S2711: Use the cooking appliance in the second visual image to adjust the cooking appliance in the first visual image to obtain the target visual image.

[0162] Here, the first visual image from the first perspective is optimized by the second visual image from the second perspective to obtain the target visual image.

[0163] Step S2712: Based on the first visual image and the second visual image, generate a panoramic image and use the panoramic image as the target visual image.

[0164] Here, the panoramic image can at least include a first visual image and a second visual image. The generation of the panoramic image can be in any suitable manner. For example, pre-processing, stitching, fusion, post-processing, etc. are performed on these two visual images. The pre-processing can include, but is not limited to, denoising, adjusting contrast, etc., to improve the accuracy and effect of subsequent image processing through pre-processing. The post-processing can include, but is not limited to, color correction, sharpening, etc., to improve the quality of the panoramic image. Another example is to generate the panoramic image based on these two visual images through a generative model, and the generative model can be any suitable model with the function of generating panoramic images.

[0165] In the embodiments of the present disclosure, the target visual image is determined in various ways such as adjusting the first visual image with the second visual image and generating a panoramic image based on the first visual image and the second visual image, which not only improves the accuracy of the target visual image, but also enhances the flexibility and comprehensiveness of the target visual image.

[0166] In some embodiments, the detection model includes a stove fire recognition model, and the stove fire recognition model is obtained after being trained using a training sample set; the step S23 includes step S231, where:

[0167] Step S231: Use the stove fire recognition model to identify the infrared information collected by the detection system to obtain the infrared detection result of the stove.

[0168] Here, the stove fire recognition model can be any suitable neural network model with the function of detecting infrared information. During implementation, inputting the infrared information into the stove fire recognition model can obtain the infrared detection result.

[0169] In the embodiments of the present disclosure, using the stove fire recognition model to determine the infrared detection result based on the infrared information shortens the determination duration of the infrared detection result and improves the reliability and accuracy of the infrared detection result.

[0170] In some embodiments, the step S23 includes step S281 and step S282, where:

[0171] Step S281: Determine the second prompt information corresponding to the stove.

[0172] Here, the second prompt information can be any suitable prompt information. The second prompt information is adapted to the infrared detection result. For example, the second prompt information can include, but is not limited to, the stove fire position, the intensity of the stove fire, etc.

[0173] The determination method of the second prompt information can be any suitable method. In some embodiments, the second prompt information can be determined based on at least one of the type of the cooking appliance, the second historical prompt information set, the infrared information, the infrared detection area corresponding to the infrared information, etc. The second historical prompt information set includes at least one historical second prompt information. The infrared detection area can be any area in the cooking appliance, and different infrared information can correspond to the same or different infrared detection areas. The determination process of the second prompt information is similar to that of the first prompt information. When implemented, the determination process of the second prompt information can refer to the determination process of the first prompt information in the foregoing step S221.

[0174] Step S282: Input the infrared information and the second prompt information collected by the detection system into the detection model to obtain the infrared detection result of the cooking appliance; wherein, the detection model includes one of the following: the second large text-image model, the second vision-language model.

[0175] Here, the second large text-image model can be any suitable large text-image model. For example, the second large text-image model can be ChatGPT. The second large text-image model and the first large text-image model can be the same or different. The second vision-language model can be any suitable VLM. For example, the second vision-language model can be DeepSeek, CLIP, etc. The second vision-language model and the first vision-language model can be the same or different.

[0176] In the embodiments of the present disclosure, using large text-image models, vision-language models, etc. to determine the infrared detection result according to the prompt information and the infrared information shortens the determination duration of the infrared detection result. At the same time, compared with using a trained model for infrared detection, since the present disclosure does not require preparing a training data set and a training model, the cost is reduced, and the possibility of misjudgment and limitations caused by using the model for infrared detection is reduced, achieving the purpose of improving the reliability, accuracy, and flexibility of infrared detection.

[0177] Figure 3 Schematic of the implementation process of a detection method for a cooking appliance provided by an embodiment of the present disclosure Figure 3 , applied to an inspection robot with a detection system, as Figure 3 shown, the detection method includes steps S31 to S35, wherein:

[0178] Step S31: In response to the inspection robot moving to the cooking appliance, control the detection system to collect the induction information set of the cooking appliance; wherein, the induction information set includes at least two types of induction information.

[0179] Here, step S31 corresponds to the foregoing step S11 or step S21. When implemented, the specific implementation manners of the foregoing step S11 or step S22 can be referred to.

[0180] Step S32: Using the detection model, based on the induction information set of the cooking appliance, determine the first detection result of the cooking appliance.

[0181] Here, this step S32 corresponds to the aforementioned step S12 or steps S22 to S24. When implemented, the specific implementation manners of the aforementioned step S12 or steps S22 to S24 can be referred to.

[0182] Step S33: Based on the information about the person leaving the cooking appliance, determine the second detection result; wherein, the second detection result indicates whether there is a person within the operation range corresponding to the cooking appliance.

[0183] Here, the information about the person leaving can include but is not limited to the leaving time, the person leaving, etc.

[0184] The determination method of this second detection result can be any suitable method.

[0185] In some implementation manners, the information about the person leaving can be input into the detection model for the person leaving to obtain this second detection result. Among them, the detection model for the person leaving can be any suitable neural network model that can implement the detection function of the information about the person leaving.

[0186] In some implementation manners, a corresponding relationship between various information about the person leaving and the second detection result can be established in advance, and the second detection result of the cooking appliance can be obtained according to the information about the person leaving of the cooking appliance.

[0187] In some implementation manners, the information about the person leaving can be compared with the first condition for the person leaving to obtain this second detection result. The first condition for the person leaving can be any suitable condition, and the first condition for the person leaving can be set according to the information about the person leaving.

[0188] For example, when the information about the person leaving includes the leaving time, the first condition for the person leaving can be not exceeding the set threshold of the leaving duration. Then, when the leaving time meets the first condition for the person leaving, the detection result indicating that there is a person within the operation range corresponding to the cooking appliance is used as this second detection result. Otherwise, the detection result indicating that there is no person within the operation range corresponding to the cooking appliance is used as this second detection result.

[0189] For another example, when the information about the person leaving includes the person leaving, the first condition for the person leaving can be that the person leaving has the ability to operate the cooking appliance. Then, when the person leaving meets the first condition for the person leaving, the detection result indicating that there is a person within the operation range corresponding to the cooking appliance is used as this second detection result. Otherwise, the detection result indicating that there is no person within the operation range corresponding to the cooking appliance is used as this second detection result.

[0190] For another example, when the leaving person information includes the leaving time and the person leaving, the first leaving person condition may be that the leaving time does not exceed the leaving duration threshold and the person leaving has the ability to operate the cooking appliance. Then, when the leaving person information meets the first leaving person condition, the detection result indicating that there is someone within the operation range corresponding to the cooking appliance is used as the second detection result; otherwise, the detection result indicating that there is no one within the operation range corresponding to the cooking appliance is used as the second detection result.

[0191] Step S34: Based on the first detection result and the second detection result, determine the target detection result of the cooking appliance.

[0192] Here, the target detection result may include but is not limited to the first target detection result, the second target detection result, etc. The determination method of the target detection result can be any suitable method. In some embodiments, a correspondence relationship between multiple first detection results, multiple second detection results, and multiple target detection results can be established in advance. According to this correspondence relationship, the target detection result that matches both the first detection result and the second detection result can be obtained. In some embodiments, when the first detection result indicates that the cooking appliance is closed, the second target detection result can be used as the target detection result; when the first detection result indicates that the cooking appliance is not closed, the target detection result is determined according to the second detection result. In some embodiments, when the second detection result indicates that there is someone within the operation range corresponding to the cooking appliance, the second target detection result can be used as the target detection result; when the second detection result indicates that there is no one within the operation range corresponding to the cooking appliance, the target detection result is determined according to the first detection result.

[0193] Step S35: When the target detection result of the cooking appliance is the first target detection result, control the patrol robot to perform the target operation; wherein, the first target detection result indicates that there is a safety hazard with the cooking appliance.

[0194] Here, step S35 corresponds to the aforementioned step S13. When implemented, reference can be made to the specific implementation manner of the aforementioned step S13.

[0195] In the embodiments of the present disclosure, the target detection result of the cooking appliance is comprehensively determined according to the induction information set and the leaving person information of the cooking appliance, which improves the accuracy of the target detection result, reduces the possibility of misjudgment, and thus improves the safety guarantee of the kitchen.

[0196] In some embodiments, step S34 includes step S341 and step S342, where:

[0197] Step S341: When the first detection result indicates that the cooking appliance is not closed, based on the second detection result, determine the target detection result of the cooking appliance.

[0198] Here, if the cooking appliance is not turned off, it indicates that there may be a safety hazard with the cooking appliance. At this time, the target detection result can be further determined based on the second detection result.

[0199] The way to determine the target detection result can be any suitable way.

[0200] In some embodiments, a corresponding relationship between each second detection result and each target detection result can be established in advance. According to this corresponding relationship, the target detection result corresponding to the second detection result can be obtained.

[0201] In some embodiments, "determining the target detection result of the cooking appliance based on the second detection result" in step S341 includes step S3411 and step S3412, where:

[0202] Step S3411: When the second detection result indicates that there is no one within the operating range corresponding to the cooking appliance or the second detection result indicates that the person within the operating range corresponding to the cooking appliance does not have the ability to operate the cooking appliance, use the first target detection result as the target detection result of the cooking appliance.

[0203] Here, when the cooking appliance is not turned off, if there is no one within the operating range corresponding to the cooking appliance or the person within this operating range cannot operate the cooking appliance, at this time, it indicates that there is a safety hazard with the cooking appliance, and the first target detection result needs to be used as the target detection result.

[0204] Step S3412: When the second detection result indicates that the person within the operating range corresponding to the cooking appliance has the ability to operate the cooking appliance, use the second target detection result as the target detection result of the cooking appliance.

[0205] Here, when the cooking appliance is not turned off, if there is someone within the operating range corresponding to the cooking appliance and the person within this operating range can operate the cooking appliance, at this time, it indicates that there is no safety hazard with the cooking appliance, and the second target detection result needs to be used as the target detection result.

[0206] In the embodiments of the present disclosure, when the cooking appliance is not turned off, the target detection result is further determined based on the second detection result to improve the accuracy of the target detection result.

[0207] Step S342: When the first detection result indicates that the cooking appliance is turned off, use the second target detection result as the target detection result of the cooking appliance.

[0208] Here, if the cooking appliance is turned off, it indicates that there is no safety hazard with the cooking appliance. At this time, the second target detection result can be used as the target detection result.

[0209] In the embodiments of the present disclosure, the target detection result is comprehensively determined based on the first detection result and the second detection result, improving the accuracy of the target detection result.

[0210] The following takes a gas stove as an example to illustrate the detection method provided by the embodiments of the present disclosure.

[0211] In the related art, during the home inspection process, the inspection robot usually detects potential safety hazards of the gas stove in the kitchen. Currently, the gas stove is mainly detected by the collected temperature information, which has problems such as a high false positive rate and poor compatibility.

[0212] The embodiments of the present disclosure provide a detection method for a cooking appliance. First, the detection result of the cooking appliance is comprehensively determined by various sensing information collected by the detection system in the inspection robot. Compared with the related art that only uses temperature to determine the detection result of the cooking appliance, the possibilities of missed detection and false detection are reduced, and the accuracy and reliability of the detection result are improved. At the same time, since the inspection robot can collect various sensing information, the applicability and compatibility of the inspection robot are improved. Secondly, a detection model is used to detect the cooking appliance, realizing the perception and understanding of multi-modal data and improving the detection efficiency. Finally, when the detection result indicates that there are potential safety hazards in the cooking appliance, the safety risk of the cooking appliance is reduced through target operations, thereby improving the safety guarantee of the kitchen.

[0213] Figure 4 The implementation process of a detection method for a cooking appliance provided by the embodiments of the present disclosure is schematically shown Figure 4 , as Figure 4 shown, the detection method includes steps S401 to S409, where:

[0214] Step S401, in response to the inspection robot moving to the gas stove, control the detection system to collect the infrared information of the gas stove, the first visual image of the first perspective, and the second visual image of the second perspective;

[0215] Step S402, use the text and image large model to respectively determine the first visual detection result corresponding to the first visual image and the second visual detection result corresponding to the second visual image;

[0216] Step S403, determine the visual detection result of the gas stove according to the first visual detection result and the second visual detection result;

[0217] Step S404, use the trained stove fire recognition model to determine the infrared detection result corresponding to the infrared information;

[0218] Step S405, determine the first detection result of the gas stove according to the visual detection result and the infrared detection result;

[0219] Step S406, determine the second detection result of the gas stove according to the information of people leaving;

[0220] Step S407: Determine the target detection result of the gas stove according to the first detection result and the second detection result;

[0221] Step S408: When the target detection result indicates that there is a safety hazard with the gas stove, use the voice reminder operation, the stove turning-off operation, and / or the remote assistance operation as the target operation, and control the inspection robot to execute the target operation;

[0222] Step S409: If the target operation includes the stove turning-off operation, control the detection system to collect the second visual information of the gas stove, determine the operation result of the gas stove according to the second visual information, and when the operation result indicates that the valve of the gas stove is not closed, control the inspection robot to execute the stove turning-off operation again.

[0223] Based on the above embodiments, an embodiment of the present disclosure provides a detection device for a stove, Figure 5 which is a schematic structural diagram of a detection device for a stove provided by an embodiment of the present disclosure. As Figure 5 shown, the detection device 50 includes a collection module 51, a determination module 52, and an operation module 53, where:

[0224] The collection module 51 is configured to control the detection system to collect the induction information set of the stove in response to the inspection robot moving to the stove; wherein, the induction information set includes at least two types of induction information;

[0225] The determination module 52 is configured to use the detection model to determine the first detection result of the stove based on the induction information set of the stove;

[0226] The operation module 53 is configured to control the inspection robot to execute the target operation when the target detection result of the stove is the first target detection result; wherein, the target detection result is determined based on the first detection result, and the first target detection result indicates that there is a safety hazard with the stove.

[0227] In some embodiments, the induction information set includes infrared information and first visual information; the determination module 52 is further configured to: use the detection model to determine the visual detection result of the stove based on the first visual information collected by the detection system; wherein, the visual detection result of the stove includes at least one of the following: whether there is a cooking utensil on the stove, the opening degree of the valve of the stove; use the detection model to determine the infrared detection result of the stove based on the infrared information collected by the detection system; wherein, the infrared detection result includes at least one of the following: the position of the stove fire, the intensity of the stove fire; determine the first detection result of the stove based on the visual detection result of the stove and the infrared detection result of the stove.

[0228] In some embodiments, the determination module 52 is further configured to: determine a first prompt message corresponding to the cooking appliance; input the first visual information and the first prompt message collected by the detection system into a detection model to obtain a visual detection result of the cooking appliance; wherein the detection model includes one of the following: a first text-image large model, a first vision-language model.

[0229] In some embodiments, the determination module 52 is further configured to perform at least one of the following: determine the first prompt message based on the type of the cooking appliance; determine the first prompt message based on the historical prompt message set of the cooking appliance; determine the first prompt message based on the first visual information of the cooking appliance; determine the first prompt message based on the visual detection area corresponding to the first visual information of the cooking appliance.

[0230] In some embodiments, the first visual information collected by the detection system includes a first visual image from a first perspective and a second visual image from a second perspective; the determination module 52 is further configured to: use the detection model to determine a first visual detection result of the cooking appliance based on the first visual image; use the detection model to determine a second visual detection result of the cooking appliance based on the second visual image; determine the visual detection result of the cooking appliance based on the first visual detection result and the second visual detection result.

[0231] In some embodiments, the first visual information collected by the detection system includes a first visual image from a first perspective and a second visual image from a second perspective; the determination module 52 is further configured to: determine a target visual image based on the first visual image and the second visual image; use the detection model to determine the visual detection result of the cooking appliance based on the target visual image.

[0232] In some embodiments, the determination module 52 is further configured to perform at least one of the following: use the cooking appliance in the second visual image to adjust the cooking appliance in the first visual image to obtain a target visual image; generate a panoramic image based on the first visual image and the second visual image, and use the panoramic image as the target visual image.

[0233] In some embodiments, the detection model includes a stove fire recognition model, and the stove fire recognition model is obtained by training using a training sample set; the determination module 52 is further configured to: use the stove fire recognition model to identify the infrared information collected by the detection system to obtain an infrared detection result of the cooking appliance.

[0234] In some embodiments, the determination module 52 is further configured to: determine a second prompt message corresponding to the cooking appliance; input the infrared information and the second prompt message collected by the detection system into the detection model to obtain an infrared detection result of the cooking appliance; wherein the detection model includes one of the following: a second text-image large model, a second vision-language model.

[0235] In some embodiments, the determination module 52 is further configured to: if the visual detection result of the cooking appliance indicates that the valve of the cooking appliance is not closed or the infrared detection result of the cooking appliance indicates that the intensity of the cooking fire exceeds a set intensity threshold, then use the detection result indicating that the cooking appliance is not closed as the first detection result of the cooking appliance; if the visual detection result of the cooking appliance indicates that the valve of the cooking appliance is closed and the infrared detection result of the cooking appliance indicates that the intensity of the cooking fire does not exceed the set intensity threshold, then use the detection result indicating that the cooking appliance is closed as the first detection result of the cooking appliance.

[0236] In some embodiments, the apparatus further includes a first determination module, and the first determination module is configured to: if the first detection result indicates that the cooking appliance is not closed, use the first target detection result as the target detection result of the cooking appliance; if the first detection result indicates that the cooking appliance is closed, use the second target detection result as the target detection result of the cooking appliance; wherein, the second target detection result indicates that there is no safety hazard for the cooking appliance.

[0237] In some embodiments, the apparatus further includes a second determination module, and the second determination module is configured to: determine a second detection result of the cooking appliance based on the information about whether there is a person away from the cooking appliance; wherein, the second detection result indicates whether there is a person within the operation range corresponding to the cooking appliance; determine the target detection result of the cooking appliance based on the first detection result and the second detection result.

[0238] In some embodiments, the second determination module is further configured to: if the first detection result indicates that the cooking appliance is not closed, determine the target detection result of the cooking appliance based on the second detection result; if the first detection result indicates that the cooking appliance is closed, use the second target detection result as the target detection result of the cooking appliance.

[0239] In some embodiments, the second determination module is further configured to: if the second detection result indicates that there is no person within the operation range corresponding to the cooking appliance or the second detection result indicates that the person within the operation range corresponding to the cooking appliance does not have the ability to operate the cooking appliance, use the first target detection result as the target detection result of the cooking appliance; if the second detection result indicates that the person within the operation range corresponding to the cooking appliance has the ability to operate the cooking appliance, use the second target detection result as the target detection result of the cooking appliance.

[0240] In some embodiments, the operation module 53 is further configured to: determine a target operation; wherein, the target operation includes at least one of the following: a voice prompt operation, a cooking appliance closing operation, a remote assistance operation; if the target operation includes a voice prompt operation, control the prompt module of the patrol robot to perform a voice reminder operation; if the target operation includes a cooking appliance closing operation, based on the valve of the cooking appliance, adjust the operation component of the patrol robot, and control the operation component of the patrol robot to perform the cooking appliance closing operation; if the target operation includes a remote assistance operation, control the patrol robot to perform a remote assistance operation.

[0241] In some embodiments, the operation module 53 is further configured to: use the voice prompt operation, the stove shutdown operation, and / or the remote assistance operation as target operations.

[0242] In some embodiments, the operation module 53 is further configured to: use the voice prompt operation as a target operation when the person-absence information of the stove meets the person-absence condition; and use the stove shutdown operation and / or the remote assistance operation as target operations when the person-absence information of the stove does not meet the person-absence condition.

[0243] In some embodiments, the operation module 53 is further configured to: determine the operating component of the inspection robot; wherein, the operating component of the inspection robot includes one of the following: a manipulator, an actuator connected to the end of the robotic arm of the inspection robot; when the operating component of the inspection robot includes a manipulator, based on the valve of the stove, adjust the opening degree of the manipulator of the inspection robot, or, based on the valve of the stove, determine a first operating mechanism from at least one operating mechanism, and control the manipulator to grab the first operating mechanism; when the operating component of the inspection robot includes an actuator, based on the valve of the stove, determine a second operating mechanism from at least one operating mechanism, and adjust the current operating mechanism of the actuator to the second operating mechanism.

[0244] In some embodiments, when the target operation includes the stove shutdown operation, the acquisition module 51 is further configured to: control the detection system to acquire the second visual information of the stove; the determination module 52 is further configured to: determine the operation result of the stove based on the second visual information; the operation module 53 is further configured to: when the operation result of the stove indicates that the valve of the stove is not closed, control the inspection robot to perform the stove shutdown operation again.

[0245] The description of the above device embodiments is similar to the description of the above method embodiments, and has beneficial effects similar to those of the method embodiments. For the technical details not disclosed in the device embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure for understanding.

[0246] It should be noted that in the embodiments of the present disclosure, if the above method is implemented in the form of software function modules and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present disclosure, in essence or the part that contributes to the related art, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present disclosure. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), magnetic disks, or optical discs that can store program codes. In this way, the embodiments of the present disclosure are not limited to any specific combination of hardware and software.

[0247] The embodiments of the present disclosure provide a patrol robot, including a memory and a processor. The memory stores a computer program that can run on the processor, and when the processor executes the computer program, the above method is implemented.

[0248] The embodiments of the present disclosure provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above method is implemented. The computer-readable storage medium can be transient or non-transient.

[0249] The embodiments of the present disclosure provide a computer program product. The computer program product includes a non-transient computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be specifically implemented in a manner of hardware, software, or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0250] It should be noted that Figure 6 is a schematic diagram of the hardware entity of a patrol robot provided in the embodiments of the present disclosure. As Figure 6 shown, the hardware entity of the patrol robot 600 includes: a processor 601, a communication interface 602, and a memory 603, where:

[0251] The processor 601 generally controls the overall operation of the patrol robot 600.

[0252] The communication interface 602 can enable the patrol robot to communicate with other terminals or servers through a network.

[0253] The memory 603 is configured to store instructions and applications executable by the processor 601, and can also cache data to be processed or already processed by the processor 601 and each module in the inspection robot 600 (for example, image data, audio data, voice communication data, and video communication data), and can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM). Data transmission can be performed between the processor 601, the communication interface 602, and the memory 603 through the bus 604.

[0254] It should be noted here that the descriptions of the above inspection robot, storage medium, and program product embodiments are similar to the descriptions of the above method embodiments and have similar beneficial effects to the method embodiments. For the technical details not disclosed in the inspection robot, storage medium, and program product embodiments of the present disclosure, please refer to the descriptions of the method embodiments of the present disclosure for understanding.

[0255] It should be understood that the phrase "one embodiment" or "an embodiment" mentioned throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the present disclosure. Therefore, the appearances of the phrase "in one embodiment" or "in an embodiment" throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present disclosure, the magnitudes of the serial numbers of the above processes do not mean the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present disclosure. The serial numbers of the embodiments of the present disclosure above are only for description and do not represent the advantages or disadvantages of the embodiments.

[0256] It should be noted that in this article, the term "comprising", "including" or any other variation thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0257] In several embodiments provided by the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed with each other can be through some interfaces. The indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0258] The units described above as separate components may or may not be physically separated. The components shown as units may or may not be physical units. They can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0259] In addition, each functional unit in the embodiments of the present disclosure can be all integrated in a processing unit, or each unit can be separately a unit, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0260] Those of ordinary skill in the art can understand that all or part of the steps to implement the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments. The foregoing storage medium includes various media that can store program codes, such as removable storage devices, read-only memory (ROM), magnetic disks, or optical discs.

[0261] Alternatively, if the above-mentioned integrated units of the present disclosure are implemented in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the related technology can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present disclosure. The foregoing storage medium includes various media that can store program codes, such as removable storage devices, ROM, magnetic disks, or optical discs.

[0262] As described above, it is only the implementation manner of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present disclosure can easily think of changes or substitutions, which should all be covered within the protection scope of the present disclosure.

Claims

1. A method for detecting a stove, characterized in that: Applied to an inspection robot having a detection system, the method comprises: In response to the inspection robot moving to the stove, controlling the detection system to collect a sensing information set of the stove; wherein the sensing information set includes at least two types of sensing information; Determining a first detection result of the cooker based on the sensing information set of the cooker by using the detection model; When the target detection result of the cooker is a first target detection result, the inspection robot is controlled to perform a target operation; wherein the target detection result is determined based on the first detection result, and the first target detection result indicates that there is a safety hazard in the cooker.

2. The method according to claim 1, characterized in that The sensing information set includes infrared information and first visual information, and the using of the detection model to determine the first detection result of the cooker based on the sensing information set of the cooker includes: Determine the visual inspection result of the stove by using the inspection model based on the first visual information collected by the inspection system; wherein the visual inspection result of the stove includes at least one of the following: whether cooking utensils are placed on the stove, and the opening degree of the valve of the stove; Using the detection model, based on the infrared information collected by the detection system, an infrared detection result of the stove is determined; wherein the infrared detection result includes at least one of the following: the position of the stove fire of the stove, the intensity of the stove fire; Based on the visual detection result of the cooker and the infrared detection result of the cooker, a first detection result of the cooker is determined.

3. The method according to claim 2, characterized in that The method of using the detection model to determine the visual detection result of the cooker based on the first visual information collected by the detection system includes: determining the first prompt information corresponding to the cooker; inputting the first visual information collected by the detection system and the first prompt information into the detection model to obtain the visual detection result of the cooker; wherein the detection model includes one of the following: a first large picture and text model, a first visual language model; The using the detection model to determine the infrared detection result of the stove based on the infrared information collected by the detection system includes: using a stove fire recognition model of the detection model to recognize the infrared information collected by the detection system to obtain the infrared detection result of the stove, wherein the stove fire recognition model is obtained after training with a training sample set; or determining the second prompt information corresponding to the stove, inputting the infrared information collected by the detection system and the second prompt information into the detection model to obtain the infrared detection result of the stove, wherein the detection model includes one of the following: a second large picture and text model, a second visual language model; The determining the first detection result of the stove based on the visual detection result of the stove and the infrared detection result of the stove includes: if the visual detection result of the stove indicates that the valve of the stove is not closed or the infrared detection result of the stove indicates that the intensity of the stove fire exceeds a set intensity threshold, then the detection result indicating that the stove is not closed is used as the first detection result of the stove; if the visual detection result of the stove indicates that the valve of the stove is closed and the infrared detection result of the stove indicates that the intensity of the stove fire does not exceed the set intensity threshold, then the detection result indicating that the stove is closed is used as the first detection result of the stove.

4. The method according to claim 2 or 3, characterized in that: The first visual information collected by the detection system includes a first visual image from a first perspective and a second visual image from a second perspective; The using the detection model to determine the visual detection result of the cooker based on the first visual information collected by the detection system includes one of the following: Determining a first visual detection result of the cooker based on the first visual image by using the detection model; Determining a second visual detection result of the cooker based on the second visual image by using the detection model; Determining a visual inspection result of the cooker based on the first visual inspection result and the second visual inspection result; determining a target visual image based on the first visual image and the second visual image; The detection model is used to determine a visual detection result of the cooker based on the target visual image.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: When the first detection result indicates that the cooker is not turned off, the first target detection result is used as the target detection result of the cooker; when the first detection result indicates that the cooker is turned off, the second target detection result is used as the target detection result of the cooker; wherein the second target detection result indicates that there is no safety hazard in the cooker; or, Based on the information of the person leaving the stove, a second detection result of the stove is determined; wherein the second detection result indicates whether there is a person in the operation range corresponding to the stove; and based on the first detection result and the second detection result, a target detection result of the stove is determined.

6. The method according to claim 5, characterized in that The determining the target detection result of the cooker based on the first detection result and the second detection result includes: When the first detection result indicates that the cooker is turned off, taking the second target detection result as the target detection result of the cooker; When the first detection result indicates that the stove is not turned off, a target detection result of the stove is determined based on the second detection result; wherein, determining the target detection result of the stove based on the second detection result includes: when the second detection result indicates that there is no person within the operating range corresponding to the stove or the second detection result indicates that the person within the operating range corresponding to the stove does not have the ability to operate the stove, taking the first target detection result as the target detection result of the stove; when the second detection result indicates that the person within the operating range corresponding to the stove has the ability to operate the stove, taking the second target detection result as the target detection result of the stove.

7. The method according to any one of claims 1 to 6, characterized in that The controlling the inspection robot to perform a target operation comprises: Determine the target operation; wherein the target operation includes at least one of the following: a voice prompt operation, a stove-off operation, and a remote help operation, and the determining the target operation includes: taking the voice prompt operation, the stove-off operation, and / or the remote help operation as the target operation; or, when the stove-off information satisfies the leave-the-person condition, taking the voice prompt operation as the target operation, and when the stove-off information does not satisfy the leave-the-person condition, taking the stove-off operation and / or the remote help operation as the target operation; In a case where the target operation includes the voice prompt operation, controlling the prompt module of the inspection robot to perform the voice prompt operation; In a case where the target operation includes the remote help operation, controlling the inspection robot to perform the remote help operation; In the case where the target operation includes the stove-closing operation, based on the valve of the stove, the operating component of the inspection robot is adjusted to control the operating component of the inspection robot to perform the stove-closing operation; wherein, the adjusting the operating component of the inspection robot based on the valve of the stove comprises: determining the operating component of the inspection robot; wherein, the operating component of the inspection robot comprises one of the following: a manipulator, an actuator connected to the end of the manipulator arm of the inspection robot; in the case where the operating component of the inspection robot includes the manipulator, based on the valve of the stove, the opening and closing degree of the manipulator of the inspection robot is adjusted, or, based on the valve of the stove, a first operating mechanism is determined from the at least one operating mechanism, and the manipulator is controlled to grab the first operating mechanism; in the case where the operating component of the inspection robot includes the actuator, based on the valve of the stove, a second operating mechanism is determined from the at least one operating mechanism, and the current operating mechanism of the actuator is adjusted to the second operating mechanism; In the case where the target operation includes the operation of shutting down the cooker, the method further includes: Controlling the detection system to collect second visual information of the cooker; determining an operation result of the cooker based on the second visual information; When the operation result of the stove indicates that the valve of the stove is not closed, the inspection robot is controlled to perform the stove closing operation again.

8. A detection device for a stove, characterized in that: Applied to inspection robots with detection systems, including: a collection module, configured to control the detection system to collect a sensing information set of the cooker in response to the inspection robot moving to the cooker; wherein the sensing information set includes at least two types of sensing information; a determination module, configured to determine a first detection result of the cooker based on a sensing information set of the cooker by using a detection model; An operation module is used to control the inspection robot to perform a target operation when the target detection result of the cooker is a first target detection result; wherein the target detection result is determined based on the first detection result, and the first target detection result indicates that the cooker has a safety hazard.

9. An inspection robot, comprising a processor and a memory, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method described in any one of claims 1 to 7 is implemented.

Images