An auxiliary control method and device for intelligent kitchen appliances

Abstract

The application discloses an auxiliary control method and device of intelligent kitchen appliances, the method comprises the following steps: generating a video call request based on a remote control request sent by a first terminal; sending the video call request to a second terminal; generating a control token if a video call response returned by the second terminal is received; sending the control token to the first terminal; if an auxiliary control instruction carrying the control token sent by the first terminal is received under the condition that the video call between the first terminal and the second terminal is established, performing corresponding operations based on the auxiliary control instruction. In the application, the auxiliary control of the kitchen appliances is realized in the process of the call, the cumbersome account registration and permission allocation process is abandoned, the user can independently complete the remote authorization without learning, the cognitive threshold and time cost of the interaction are significantly reduced, the dynamic nature and controllability of the operation permission are ensured, and the remote kitchen appliance control can be safely and reliably applied to the daily cooking scene.

Description

Technical Field

[0001] This application relates to the field of kitchen appliance control technology, and in particular to an auxiliary control method and device for intelligent kitchen appliances. Background Technology

[0002] With the improvement of people's living standards and the promotion and popularization of technologies such as the Internet, big data, artificial intelligence, and voice interaction, more and more traditional lifestyles are gradually changing, and the use of kitchen appliances is gradually moving towards intelligence. The introduction of remote control functions has greatly improved the convenience of using kitchen appliances, allowing users to monitor and adjust appliances such as gas stoves, ovens, and air conditioners in real time through mobile terminals. This remote interaction capability demonstrates extremely high application value and technological importance when users are away from home handling emergencies or assisting family members in cooking.

[0003] Currently, commonly used remote control solutions for kitchen appliances are typically based on an account-sharing model. Users bind their kitchen appliances to a main account via their mobile phones and then grant other family members remote control permissions as needed. However, the complex account registration, member addition, and menu navigation processes create a high cognitive barrier, resulting in lengthy remote control times. Some users also struggle to independently control the devices, significantly reducing the accessibility of smart home services. Furthermore, the lack of an effective real-time confirmation mechanism makes remote operation particularly vulnerable to serious safety incidents due to misoperation or theft of remote accounts, especially when handling high-risk activities involving gas or high temperatures. This makes the remote authorization model inherently risky. Therefore, remote control of kitchen appliances is often difficult to apply effectively in daily cooking. Summary of the Invention

[0004] To address existing technical problems, this invention provides an auxiliary control method and device for smart kitchen appliances. Through the core concept of authorization via video call, it utilizes the user's familiar video call answering action to issue temporary control permissions after the call is established, enabling auxiliary control of the kitchen appliances during the call. This eliminates the cumbersome account registration and permission allocation process, allowing users to independently complete remote authorization without learning, significantly reducing the cognitive threshold and time cost of interaction. Furthermore, by establishing a temporary token mechanism for video calls, the dynamic and controllable nature of operation permissions is ensured, enabling safe and reliable remote kitchen appliance control in daily cooking scenarios.

[0005] In a first aspect, embodiments of this application provide an auxiliary control method for intelligent kitchen appliances, the method comprising: A video call request is generated based on the remote control request sent by the first terminal; Send a video call request to the second terminal; If a video call response is received from the second terminal, a control token is generated. Send a control token to the first terminal; When a video call is established between the first terminal and the second terminal, if an auxiliary control command carrying a control token is received from the first terminal, the corresponding operation is executed based on the auxiliary control command.

[0006] In one alternative embodiment, the auxiliary control instructions include status query instructions; If an auxiliary control command carrying a control token is received from the first terminal, the corresponding operation is executed based on the auxiliary control command, including: If a status query command carrying a control token is received from the first terminal, the operating status information of the smart kitchen appliance can be obtained. The running status information is sent to the first terminal.

[0007] In one alternative embodiment, the auxiliary control instructions include auxiliary operation instructions; If an auxiliary control command carrying a control token is received from the first terminal, the corresponding operation is executed based on the auxiliary control command, including: If an auxiliary operation instruction carrying a control token is received from the first terminal, the operation risk level is determined based on the auxiliary operation instruction; If the operational risk level is the first risk level, execute the corresponding operation based on the auxiliary operation instruction; or; if the operational risk level is the second risk level, perform a confirmation and verification operation on the auxiliary operation instruction, obtain the verification result, and execute the corresponding operation based on the verification result and the auxiliary operation instruction.

[0008] In one optional embodiment, if the operational risk level is the first risk level, the corresponding operation is performed based on the auxiliary operation instruction, including: If the operational risk level is the highest risk level, auxiliary operation instructions will be sent to the execution module of the smart kitchen appliance. If a successful execution response is received from the execution module based on the auxiliary operation instruction, status change information is sent to the second terminal; the status change information is used to display the status of the execution module after performing the operation on the second terminal.

[0009] In one optional embodiment, the auxiliary operation command is verified to obtain a verification result, including: A confirmation prompt message is sent to the second terminal based on the auxiliary operation instructions; the confirmation prompt message is used to instruct the second user of the second terminal to perform a secondary confirmation operation. If a second confirmation message generated based on a second confirmation operation is received from the second terminal, the confirmation keyword information and confirmation duration information in the second confirmation message are extracted. If both the keyword information and the confirmation duration information meet the confirmation requirements, the verification is considered successful; or, if either the keyword information or the confirmation duration information does not meet the confirmation requirements, the verification is considered unsuccessful.

[0010] In one optional embodiment, the corresponding operation is performed based on the verification result and the auxiliary operation instruction, including: If the verification result is successful, a confirmation message is sent to the first terminal. Send auxiliary operation instructions to the execution module of the smart kitchen appliance; If a successful execution response is received from the execution module based on the auxiliary operation instructions, status change information is sent to the second terminal; Auxiliary operation logs are generated and stored based on secondary confirmation information and auxiliary operation instructions.

[0011] In one optional embodiment, when a video call is established between the first terminal and the second terminal, if an auxiliary control command carrying a control token is received from the first terminal, the corresponding operation is performed based on the auxiliary control command, including: When a video call is established between the first terminal and the second terminal, if an emergency termination command is received from the second terminal, the validity of the control token is terminated; the emergency termination command is triggered by the second user of the second terminal based on the emergency termination control. Send an operation abort notification message to the first terminal; Send an emergency lock command to the execution module of the smart kitchen appliance; the emergency lock command is used to instruct the smart kitchen appliance to enter a preset safety lock state.

[0012] In one optional embodiment, if an auxiliary control command carrying a control token is received from the first terminal, the corresponding operation is performed based on the auxiliary control command, including: Receive kitchen appliance image information collected and sent by the second terminal; The kitchen appliance image information is sent to the first terminal; the kitchen appliance image information is used for image recognition processing based on the first terminal to obtain the control component information; the control component information is used to generate a virtual control component on the first terminal and overlay it on the video call screen of the first terminal. If a virtual operation command carrying a control token is received from the first terminal, triggered by a virtual operation device, the control parameters are determined based on the virtual operation command. Encapsulate control parameters into auxiliary control commands; The auxiliary control commands are sent to the second terminal through the signaling channel of the video call; the auxiliary control commands are used by the second terminal to send to the execution module of the smart kitchen appliance through the local protocol.

[0013] In an optional embodiment, the method further includes: If a successful execution response is received from the execution module based on the auxiliary control command, execution feedback information is sent to the first terminal; the execution feedback signal is used to trigger the first terminal to update the status of the virtual operation device on the video call screen.

[0014] Secondly, embodiments of this application provide an auxiliary control device for intelligent kitchen appliances, the device comprising: The request generation module is used to generate a video call request based on the remote control request sent by the first terminal. The request sending module is used to send video call requests to the second terminal. The token generation module is used to generate a control token if a video call response is received from the second terminal. The token sending module is used to send control tokens to the first terminal; The auxiliary control module is used to perform corresponding operations based on the auxiliary control command sent by the first terminal carrying a control token when a video call is established between the first terminal and the second terminal.

[0015] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The processor loads and executes the at least one instruction, at least one program, code set, or instruction set to implement the auxiliary control method for intelligent kitchen appliances of the first aspect.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the auxiliary control method for the intelligent kitchen appliance of the first aspect.

[0017] Fifthly, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the auxiliary control method for intelligent kitchen appliances according to the first aspect.

[0018] The auxiliary control method and device for smart kitchen appliances provided in this application have the following technical effects: Based on the remote control request sent by the first terminal, a video call request is generated; a video call request is sent to the second terminal; if a video call response is received from the second terminal, a control token is generated; the control token is sent to the first terminal; if a video call is established between the first and second terminals, and an auxiliary control command carrying the control token is received from the first terminal, the corresponding operation is executed based on the auxiliary control command. In this application, through the core concept of video call authorization, and utilizing the user's most familiar video call answering action, temporary control permissions are issued after the call is established, enabling auxiliary control of kitchen appliances during the call. This eliminates the cumbersome account registration and permission allocation process, allowing users to independently complete remote authorization without learning, significantly reducing the cognitive threshold and time cost of interaction; moreover, by establishing a temporary token mechanism for video calls, the dynamic and controllable nature of operation permissions is ensured, enabling remote kitchen appliance control to be safely and reliably applied in daily cooking scenarios. Attached Figure Description

[0019] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application; Figure 2 This is a flowchart illustrating an auxiliary control method for a smart kitchen appliance provided in an embodiment of this application. Figure 1 ; Figure 3 This is a flowchart illustrating an auxiliary control method for a smart kitchen appliance provided in an embodiment of this application. Figure 2 ; Figure 4 This is a flowchart illustrating an auxiliary control method for a smart kitchen appliance provided in an embodiment of this application. Figure 3 ; Figure 5 This is a flowchart illustrating an auxiliary control method for a smart kitchen appliance provided in an embodiment of this application. Figure 4 ; Figure 6 This is a schematic diagram of the structure of an auxiliary control device for a smart kitchen appliance provided in an embodiment of this application; Figure 7 This is a hardware structure block diagram of a server for an auxiliary control method for intelligent kitchen appliances provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0023] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application, including kitchen appliances 101, a first terminal 102 and a second terminal 103, which communicate and interact with each other using communication protocols such as WiFi, 4G / 5G, Bluetooth and Zigbee.

[0024] In one possible embodiment, the first terminal 102 is a remote assistance terminal, specifically a smart device such as a smartphone, tablet, or computer. The device has a dedicated control application (APP) installed, primarily used by remote assistance personnel, family members, and professional cooking instructors. The first terminal 102 has functions such as initiating video calls, viewing device status, operating augmented reality (AR) virtual controls, and issuing auxiliary control commands; it is the command initiator of the entire control system.

[0025] In one possible embodiment, the second terminal 103 is a local controlled terminal, including smart home devices such as smart home screens and smartphones designed for the elderly, primarily for use by local users such as seniors and novice cooks. The second terminal 103 is pre-installed with only a simplified version of the family assistant application, eliminating complex registration, login, and menu switching operations. The interface only features large controls for answering, rejecting, emergency locking, and secondary confirmation, adapting to the usage habits of users with low learning abilities and reducing the user's operating threshold.

[0026] In this embodiment, the kitchen appliance 101 generates a video call request based on a remote control request sent by a first terminal; sends the video call request to a second terminal; if a video call response is received from the second terminal, a control token is generated; the control token is sent to the first terminal; and if an auxiliary control command carrying the control token is received from the first terminal when a video call is established between the first terminal and the second terminal, the corresponding operation is performed based on the auxiliary control command.

[0027] In this application, the core concept of authorization via video call is utilized. By leveraging the user's familiar video call answering action, temporary control permissions are granted after the call is established, enabling auxiliary control of kitchen appliances during the call. This eliminates the cumbersome account registration and permission allocation process, allowing users to independently complete remote authorization without learning, significantly reducing the cognitive threshold and time cost of interaction. Furthermore, by establishing a temporary token mechanism for video calls, the dynamic and controllable nature of operation permissions is ensured, enabling the safe and reliable application of remote kitchen appliance control in daily cooking scenarios.

[0028] The following describes a specific embodiment of an auxiliary control method for intelligent kitchen appliances according to this application. Figure 2 This is a flowchart illustrating an auxiliary control method for a smart kitchen appliance provided in an embodiment of this application. Figure 1 This specification provides method operation steps as shown in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual system or server products, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as shown in the embodiments or drawings... Figure 2 As shown, it may include: S201: Generate a video call request based on the remote control request sent by the first terminal.

[0029] S202: Send a video call request to the second terminal.

[0030] S203: If a video call response is received from the second terminal, a control token is generated.

[0031] S204: Send a control token to the first terminal.

[0032] S205: When a video call is established between the first terminal and the second terminal, if an auxiliary control command carrying a control token is received from the first terminal, the corresponding operation is executed based on the auxiliary control command.

[0033] Figure 3This is a flowchart illustrating an auxiliary control method for a smart kitchen appliance provided in an embodiment of this application. Figure 2 The method may include: S301: Generate a video call request based on the remote control request sent by the first terminal.

[0034] In this embodiment, before remote assisted control, an initial binding step is included: remotely assisting the user, such as a child, to bind the family kitchen appliances on the APP of the first terminal and associate the elderly person's mobile phone number or the controlled device identifier (ID) with their family. The second user, such as an elderly person, does not need to install or log in to a complex APP; their second terminal only needs a pre-installed, minimalist "Family Assistant" application, which only has large "Video Answer" and "Confirm Authorization" buttons.

[0035] In one possible embodiment, when a remote assistance user needs to help a local user operate kitchen appliances, the user can click the remote assistance function button in the control APP of the first terminal.

[0036] After capturing user actions, the first terminal generates a remote control request carrying a unique household identifier, the first terminal device ID, account information, and the serial number of the kitchen appliance to be managed, and uploads the request to the cloud server of the kitchen appliance.

[0037] After receiving the request, the server parses the unique household identifier, matches and binds the corresponding second terminal, and simultaneously generates a unique video call request with an authorization prompt. The prompt is used to inform the second terminal user that answering the call means granting the other party temporary control permissions for kitchen appliances.

[0038] S302: Send a video call request to the second terminal.

[0039] In one possible embodiment, the server generates a completed video call request and pushes it to the successfully matched second terminal via a wireless network.

[0040] The second terminal listens for cloud message pushes in real time while in standby mode. Upon receiving a request, it automatically wakes up the screen and displays the video call interface in full screen. The interface only has two basic operation buttons, "Answer" and "Reject", without any extra complex functions, simplifying the operation for local users to the greatest extent.

[0041] S303: If a video call response is received from the second terminal, a control token is generated.

[0042] In one possible embodiment, if the second terminal user views the video call and authorization prompt and clicks the answer control, the second terminal immediately sends a video answer response back to the cloud server.

[0043] After receiving the response, the cloud server completes two-way verification of the identities of both users, confirming that they belong to the same legitimate family account. Upon successful verification, the cloud generates a temporary control token strongly bound to this video session.

[0044] In this embodiment, the control token has a time-limited feature, with its validity period completely synchronized with the video call period. It can only be used for kitchen appliance control within the current session and does not support reuse across sessions or devices. If the second terminal user rejects the call or fails to respond within the timeout period, the cloud directly rejects the remote assistance request and does not generate any control token.

[0045] S304: Send a control token to the first terminal.

[0046] In one possible embodiment, the cloud server sends the control token, a list of all bound kitchen appliances under the family account, and real-time operating status data of each device to the first terminal. The first terminal verifies the validity of the token's format and validity period. Upon successful verification, a two-way video call link is automatically established, and a simplified version of the kitchen appliance control panel is displayed in a pop-up window to facilitate subsequent assistance operations by the remote user.

[0047] S305: When a video call is established between the first terminal and the second terminal, if an auxiliary control command carrying a control token is received from the first terminal, the corresponding operation is executed based on the auxiliary control command.

[0048] In one possible embodiment, during the duration of a video call session, all auxiliary control commands issued by the first terminal must be accompanied by a temporary control token as an authorization credential.

[0049] After receiving the instruction, the cloud server first verifies the validity of the token. Only after the verification is successful can the corresponding operation be executed according to the instruction type.

[0050] In this application embodiment, the auxiliary control commands cover a variety of types, such as status query, device operation, and parameter adjustment, covering the needs of remote assistance in all scenarios.

[0051] In one optional embodiment, the auxiliary control command includes a status query command, specifically a query for the open / closed status of the stove valve. If an auxiliary control command carrying a control token is received from the first terminal, the corresponding operation is performed based on the auxiliary control command, including: S3051: If a status query command carrying a control token is received from the first terminal, obtain the operating status information of the smart kitchen appliance.

[0052] If the cloud server receives a status query command carrying a valid control token from the first terminal, it will immediately retrieve the real-time operating status information of the corresponding kitchen appliance. The operating status information includes core data such as the device start / stop status, real-time firepower, set temperature, valve opening / closing status, and fault error information.

[0053] S3052: Send the running status information to the first terminal.

[0054] The cloud server synchronously sends the collected operating status information to the first terminal, allowing remote users to intuitively view the operating status of local kitchen appliances and formulate subsequent control plans based on this information, thus achieving remote visual assistance.

[0055] To ensure the highest operational privileges of local users and mitigate the risks of accidental or malicious operations, this method incorporates an emergency termination mechanism. In one optional embodiment, when a video call is established between the first terminal and the second terminal, if an auxiliary control command carrying a control token is received from the first terminal, the corresponding operation is executed based on the auxiliary control command, including: S3053: In the event that a video call is established between the first terminal and the second terminal, if an emergency termination command is received from the second terminal, the validity of the control token shall be terminated.

[0056] In this embodiment, the emergency termination command is triggered by the second user of the second terminal based on the emergency termination control.

[0057] If a user of the second terminal perceives a security risk in remote operation during any time during a video call, they can proactively trigger an emergency termination command through the emergency termination control that is always present on the second terminal interface.

[0058] Upon receiving the instruction, the cloud server immediately invalidates the control token corresponding to the current session, directly stripping the first terminal of all remote control privileges.

[0059] S3054: Send an operation abort notification message to the first terminal.

[0060] In this embodiment, the cloud server synchronously pushes an operation termination notification to the first terminal, clearly informing the remote user that the remote assistance has been forcibly terminated by the local user.

[0061] S3055: Sends an emergency lock command to the execution module of the smart kitchen appliance.

[0062] In this embodiment, the cloud server sends an emergency lock command to the execution module of the kitchen appliance. The emergency lock command is used to control the kitchen appliance to enter a preset safety lock state, which specifically includes closing the gas valve, stopping high-temperature heating, and forcing the device into standby mode, thereby avoiding safety accidents from the device's underlying layer.

[0063] S306: Terminate the validity of the control token if the video call ends at the first terminal and the second terminal.

[0064] In one possible embodiment, when either user hangs up the video call and the session automatically terminates, the cloud server monitors the session status in real time and actively cancels the control token bound to this session, so the first terminal can no longer issue any auxiliary control commands.

[0065] Figure 4 This is a flowchart illustrating an auxiliary control method for a smart kitchen appliance provided in an embodiment of this application. Figure 3 In one optional embodiment, the auxiliary control command includes auxiliary operation commands, such as a command to close the stove valve. If an auxiliary control command carrying a control token is received from the first terminal, the corresponding operation is performed based on the auxiliary control command, including: S401: If an auxiliary operation instruction carrying a control token is received from the first terminal, the operation risk level is determined based on the auxiliary operation instruction.

[0066] In one possible embodiment, a risk level database is pre-configured within the cloud server to classify all kitchen appliance operation behaviors into levels.

[0067] After receiving the auxiliary operation instructions, the cloud server parses the corresponding operation content, matches it with database tags, and determines the corresponding risk level. The risk levels are divided into a first risk level (low risk) and a second risk level (high risk).

[0068] S402: Determine whether the operational risk level is the first risk level. If yes, execute S403; otherwise, execute S404.

[0069] In one possible embodiment, the first risk level is basic operations with no safety hazards, including adjusting the kitchen air conditioner temperature and switching the equipment standby mode; the second risk level is high-risk operations that are prone to causing gas leaks, fires, and burns, including opening and closing the main gas valve, starting the oven in high-temperature mode, adjusting the stove to high power, and forcibly modifying the timer mode.

[0070] In one possible embodiment, if the operational risk level is the first risk level, the corresponding operation can be directly executed based on the auxiliary operation instructions.

[0071] In another possible embodiment, if the operation risk level is the second risk level, direct execution would pose potential dangers and risks. Therefore, it is necessary to perform a verification operation on the auxiliary operation instructions to obtain the verification result, and then execute the corresponding operation based on the verification result and the auxiliary operation instructions.

[0072] S403: Execute the corresponding operation based on the auxiliary operation instruction.

[0073] In one optional embodiment, if the operational risk level is the first risk level, the corresponding operation is performed based on the auxiliary operation instruction, including: S4031: If the operation risk level is the first risk level, the auxiliary operation instruction will be sent to the execution module of the smart kitchen appliance.

[0074] The cloud server directly forwards compliant, low-risk auxiliary operation instructions to the corresponding kitchen appliance's execution module.

[0075] S4032: If a successful execution response is received from the execution module based on the auxiliary operation instruction, send status change information to the second terminal.

[0076] After executing the module's parsing instructions and completing the corresponding operations, a successful execution response is sent back to the cloud.

[0077] After receiving the response from the cloud, the latest status change information of the device is pushed to the second terminal. The status change information is used to display the status after the execution module performs the operation on the second terminal, and the data of the control panel of the second terminal is updated in real time, so that the local user can intuitively grasp the operating status of the device.

[0078] S404: Perform a confirmation and verification operation on the auxiliary operation command to obtain the verification result.

[0079] In one optional embodiment, if the operational risk level is determined to be the second risk level, direct issuance of instructions is prohibited. Instead, an confirmation and verification operation is performed on the auxiliary operation instructions to obtain the verification result, including: S4041: Send a confirmation prompt message to the second terminal based on the auxiliary operation command.

[0080] Based on high-risk auxiliary operation instructions, the cloud pushes visual confirmation prompts to the second terminal. The confirmation prompts are used to instruct the second user on the second terminal to perform a secondary confirmation operation. The operation content is displayed in a large-font pop-up window, such as "Long press to confirm for 3 seconds and enter my voice of agreement", prompting the local second user to perform a secondary confirmation operation.

[0081] S4042: If the second terminal returns the second confirmation information generated based on the second confirmation operation, extract the confirmation keyword information and confirmation duration information from the second confirmation information.

[0082] The second user on the second terminal needs to complete the dual verification operation simultaneously, by pressing and holding the confirmation control for more than 3 seconds and recording preset confirmation keywords such as "I agree" or "allow operation" through the microphone.

[0083] The second terminal collects confirmation duration information and confirmation keyword information, generates secondary confirmation information, and sends it back to the cloud server. The cloud server performs voice recognition and duration judgment.

[0084] S4043: Determine whether the secondary confirmation information meets the confirmation requirements. If yes, execute S4044; otherwise, execute S4045.

[0085] S4044: Verification passed is the verification result.

[0086] S4045: Verification failure is the result of the verification.

[0087] In one optional embodiment, if both the keyword information and the confirmation duration information meet the confirmation requirements, the verification is determined to be successful as the verification result.

[0088] In another optional embodiment, if the keyword information or confirmation duration information does not meet the confirmation requirements, the verification is determined to fail as the verification result.

[0089] S405: Execute the corresponding operation based on the verification result and auxiliary operation instructions.

[0090] In one optional embodiment, the corresponding operation is performed based on the verification result and the auxiliary operation instruction, including: S4051: If the verification result is successful, send a confirmation message to the first terminal.

[0091] If the verification result is successful, the cloud will immediately send a confirmation message to the first terminal, informing the remote first user that the local user has been authorized.

[0092] S4052: Sends auxiliary operation instructions to the execution module of the smart kitchen appliance.

[0093] The cloud server forwards high-risk auxiliary operation instructions to the kitchen appliance execution module to complete the corresponding high-risk operation.

[0094] S4053: If a successful execution response is received from the execution module based on the auxiliary operation instruction, send status change information to the second terminal.

[0095] After the module operation is completed, a success response is sent back, and the cloud synchronously pushes the device status change information to the second terminal.

[0096] S4054: Generate and store auxiliary operation logs based on secondary confirmation information and auxiliary operation instructions.

[0097] The cloud server extracts data such as the secondary confirmation screenshot, voice recording, auxiliary operation instructions, timestamp, and device information from both ends of the operation, generates an unalterable exclusive auxiliary operation log, and stores it permanently for subsequent safety audits and accident liability tracing, in order to meet the requirements of kitchen appliance safety management regulations.

[0098] Figure 5 This is a flowchart illustrating an auxiliary control method for a smart kitchen appliance provided in an embodiment of this application. Figure 4 If an auxiliary control command carrying a control token is received from the first terminal, the corresponding operation is performed based on the auxiliary control command, including: S501: Receives kitchen appliance image information collected and sent by the second terminal.

[0099] In one possible embodiment, during the process of remotely assisting in the control of kitchen appliances, the second terminal calls the built-in camera to collect image information of the operating areas of the kitchen appliances, such as knobs, switches, and power indicator lights, in real time, and uploads it to the cloud server through a video call channel.

[0100] S502: Send the kitchen appliance image information to the first terminal.

[0101] In this embodiment, the kitchen appliance image information is used for image recognition processing based on the first terminal to obtain the operating component information.

[0102] In one possible embodiment, after receiving the image information of the kitchen appliances, the cloud server forwards it to the first terminal in real time. The first terminal has a built-in artificial intelligence (AI) image recognition algorithm, which can automatically identify the position of the stove knobs, the gear setting, the oven temperature parameters, the flame status, and other operating information in the image, providing data support for subsequent virtual operation.

[0103] In this embodiment, the operation information is used to generate a virtual operation device on the first terminal and overlay it onto the video call screen of the first terminal. After the first terminal identifies the location information and image information of the operation device, a controllable virtual operation device is generated and overlaid onto the video call screen of the first terminal. The kitchen appliance that needs to be controlled can be seen intuitively through AR technology.

[0104] S503: If a virtual operation instruction carrying a control token is received from the first terminal, triggered by a virtual operation device, the control parameters are determined based on the virtual operation instruction.

[0105] In this embodiment, the first terminal overlays AR virtual control elements such as virtual knobs, power sliders, and switch buttons onto the video call screen based on the identified control element information.

[0106] The first remote user can issue virtual operation commands by directly touching the virtual control panel. The first terminal captures the touch trajectory and maps the touch position, gear adjustment range, and other data into corresponding precise control parameters, such as adjusting the stove firepower to 70% or the oven to 180℃.

[0107] S504: Encapsulates control parameters into auxiliary control instructions.

[0108] The first terminal encapsulates the converted control parameters and control token to generate standardized auxiliary control commands, ensuring the integrity and legitimacy of command transmission.

[0109] S505: Send auxiliary control commands to the second terminal through the signaling channel of the video call.

[0110] In this embodiment, the auxiliary control command is used to be sent by the second terminal to the execution module of the smart kitchen appliance via a local protocol.

[0111] In one possible embodiment, this solution abandons the mode of directly issuing instructions from the cloud server. Instead, it sends the encapsulated auxiliary control instructions to the second terminal via a dedicated signaling channel for video calls. The second terminal then forwards the instructions to the execution module of the kitchen appliance via local communication protocols such as Bluetooth and Zigbee. This architecture complies with the regulatory requirement that high-risk operations must be initiated and executed by the local terminal.

[0112] S506: If a successful execution response is received from the execution module based on the auxiliary control command, send execution feedback information to the first terminal.

[0113] In one alternative embodiment, the execution feedback signal is used to trigger the first terminal to update the virtual control state on the video call screen.

[0114] In one possible embodiment, after the kitchen appliance execution module completes parameter adjustment, it sends back a successful execution response.

[0115] After receiving the successful execution response, the cloud server pushes the corresponding execution feedback information to the first terminal. The feedback information can directly trigger the first terminal to update the status of the AR virtual operation component in the video screen. At the same time, combined with the real-time camera image, the remote user can intuitively see the actual changes of the flame and the knob, forming a closed-loop control system of image acquisition, virtual operation, command execution and visual feedback, which completely solves the industry pain point of vague text and voice command descriptions.

[0116] This embodiment addresses the pain points of existing cooking instruction modes, such as vague instructions and poor teaching effectiveness, by adding an AR virtual control process, which can achieve remote, step-by-step, visual, and precise guidance.

[0117] This application also provides an auxiliary control device for smart kitchen appliances. Figure 6 This is a schematic diagram of the structure of an auxiliary control device for a smart kitchen appliance provided in an embodiment of this application, as shown below. Figure 6 As shown, the device 600 includes: The request generation module 610 is used to generate a video call request based on the remote control request sent by the first terminal; The request sending module 620 is used to send a video call request to the second terminal. The token generation module 630 is used to generate a control token if a video answering response is received from the second terminal. The token sending module 640 is used to send a control token to the first terminal; The auxiliary control module 650 is used to perform corresponding operations based on the auxiliary control command sent by the first terminal carrying a control token when a video call is established between the first terminal and the second terminal.

[0118] In one optional embodiment, the auxiliary control instructions include status query instructions; and further include: The acquisition module is used to acquire the operating status information of the smart kitchen appliance if it receives a status query command carrying a control token sent by the first terminal. The first sending module is used to send the running status information to the first terminal.

[0119] In one optional embodiment, the auxiliary control instructions include auxiliary operation instructions; and further include: The first determining module is used to determine the operation risk level based on the auxiliary operation instruction if it receives an auxiliary operation instruction carrying a control token sent by the first terminal. The operation execution module is used to execute the corresponding operation based on the auxiliary operation instruction if the operation risk level is the first risk level; or, if the operation risk level is the second risk level, to perform a confirmation and verification operation on the auxiliary operation instruction, obtain the verification result, and execute the corresponding operation based on the verification result and the auxiliary operation instruction.

[0120] In an optional embodiment, it further includes: The second sending module is used to send auxiliary operation instructions to the execution module of the smart kitchen appliance if the operation risk level is the first risk level. The third sending module is used to send status change information to the second terminal if it receives a successful execution response from the execution module based on the auxiliary operation instruction; the status change information is used to display the status of the execution module after performing the operation on the second terminal.

[0121] In an optional embodiment, it further includes: The fourth sending module is used to send a confirmation prompt message to the second terminal based on the auxiliary operation instructions; the confirmation prompt message is used to instruct the second user of the second terminal to perform a secondary confirmation operation; The information extraction module is used to extract confirmation keyword information and confirmation duration information from the secondary confirmation information generated based on the secondary confirmation operation returned by the second terminal. The second determination module is used to determine that if both the confirmed keyword information and the confirmed duration information meet the confirmation requirements, the verification passes and is determined as the verification result; or, if the confirmed keyword information or the confirmed duration information does not meet the confirmation requirements, the verification fails and is determined as the verification result.

[0122] In an optional embodiment, it further includes: The fifth sending module is used to send a confirmation message to the first terminal if the verification result is successful. The sixth sending module is used to send auxiliary operation instructions to the execution module of the smart kitchen appliance; The seventh sending module is used to send status change information to the second terminal if it receives a successful execution response from the execution module based on the auxiliary operation instruction. The instruction generation module is used to generate and store auxiliary operation logs based on secondary confirmation information and auxiliary operation instructions.

[0123] In an optional embodiment, it further includes: The token termination module is used to terminate the validity of the control token if an emergency termination command is received from the second terminal when a video call is established between the first terminal and the second terminal. The emergency termination command is triggered by the second user of the second terminal based on the emergency termination control. The eighth sending module is used to send an operation abort prompt message to the first terminal; The ninth sending module is used to send an emergency lock command to the execution module of the smart kitchen appliance; the emergency lock command is used to instruct the smart kitchen appliance to enter a preset safety lock state.

[0124] In an optional embodiment, it further includes: The receiving module is used to receive kitchen appliance image information collected and sent by the second terminal; The tenth sending module is used to send kitchen appliance image information to the first terminal; the kitchen appliance image information is used for image recognition processing based on the first terminal to obtain operating component information; the operating component information is used to generate virtual operating components on the first terminal and overlay them on the video call screen of the first terminal; The third determining module is used to determine control parameters based on the virtual operation instruction if it receives a virtual operation instruction sent by the first terminal that is triggered by a virtual operation device and carries a control token. The instruction encapsulation module is used to encapsulate control parameters into auxiliary control instructions; The eleventh sending module is used to send auxiliary control commands to the second terminal through the signaling channel of the video call; the auxiliary control commands are used by the second terminal to send to the execution module of the smart kitchen appliance through the local protocol.

[0125] In an optional embodiment, it further includes: The twelfth sending module is used to send execution feedback information to the first terminal if it receives a successful execution response from the execution module based on the auxiliary control command; the execution feedback signal is used to trigger the first terminal to update the status of the virtual operation device on the video call screen.

[0126] The apparatus and method embodiments in this application are based on the same application concept.

[0127] The methods and embodiments provided in this application can be executed on a computer terminal, server, or similar computing device. Taking running on a server as an example, Figure 7 This is a hardware structure block diagram of a server for an auxiliary control method for intelligent kitchen appliances provided in an embodiment of this application. For example... Figure 7 As shown, the server 700 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 710 (CPUs 710 may include, but are not limited to, microprocessors (MCUs) or programmable logic devices (FPGAs), a memory 730 for storing data, and one or more storage media 720 (e.g., one or more mass storage devices) for storing application programs 723 or data 722. The memory 730 and storage media 720 may be temporary or persistent storage. The program stored in the storage media 720 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the CPU 710 may be configured to communicate with the storage media 720 and execute the series of instruction operations stored in the storage media 720 on the server 700. Server 700 may also include one or more power supplies 760, one or more wired or wireless network interfaces 750, one or more input / output interfaces 740, and / or one or more operating systems 721, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0128] The input / output interface 740 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of server 700. In one example, the input / output interface 740 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 740 may be a radio frequency (RF) module used for wireless communication with the Internet.

[0129] Those skilled in the art will understand that Figure 7 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, server 700 may also include... Figure 7 The more or fewer components shown, or having the same Figure 7 The different configurations shown.

[0130] This application provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The processor loads and executes the at least one instruction, at least one program, code set, or instruction set to implement the above-described data processing method.

[0131] Embodiments of this application also provide a computer-readable storage medium, which can be disposed in a server to store at least one instruction, at least one program, code set, or instruction set related to implementing an auxiliary control method for a smart kitchen appliance in the method embodiment. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the aforementioned auxiliary control method for the smart kitchen appliance.

[0132] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0133] As can be seen from the embodiments of the auxiliary control method, device, electronic device, or storage medium for smart kitchen appliances provided in this application, this application generates a video call request based on a remote control request sent by a first terminal; sends the video call request to a second terminal; if a video call response is received from the second terminal, a control token is generated; the control token is sent to the first terminal; and when a video call is established between the first and second terminals, if an auxiliary control instruction carrying the control token is received from the first terminal, the corresponding operation is executed based on the auxiliary control instruction. In this application, through the core concept of authorization via video call, and utilizing the user's most familiar video call answering action, temporary control permissions are issued after the call is established, enabling auxiliary control of kitchen appliances during the call. This eliminates the cumbersome account registration and permission allocation process, allowing users to independently complete remote authorization without learning, significantly reducing the cognitive threshold and time cost of interaction. Moreover, by establishing a temporary token mechanism for video calls, the dynamic and controllable nature of operation permissions is ensured, enabling remote kitchen appliance control to be safely and reliably applied in daily cooking scenarios.

[0134] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0135] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0136] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0137] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An auxiliary control method for intelligent kitchen appliances, characterized in that, include: A video call request is generated based on the remote control request sent by the first terminal; Send the video call request to the second terminal; If a video call response is received from the second terminal, a control token is generated. Send the control token to the first terminal; When a video call is established between the first terminal and the second terminal, if an auxiliary control command carrying the control token is received from the first terminal, the corresponding operation is performed based on the auxiliary control command.

2. The auxiliary control method for a smart kitchen appliance according to claim 1, characterized in that, The auxiliary control commands include status query commands; If an auxiliary control command carrying the control token is received from the first terminal, the corresponding operation is performed based on the auxiliary control command, including: If a status query command carrying the control token is received from the first terminal, the operating status information of the smart kitchen appliance can be obtained. The operating status information is sent to the first terminal.

3. The auxiliary control method for a smart kitchen appliance according to claim 1, characterized in that, The auxiliary control commands include auxiliary operation commands; If an auxiliary control command carrying the control token is received from the first terminal, the corresponding operation is performed based on the auxiliary control command, including: If an auxiliary operation instruction carrying the control token is received from the first terminal, the operation risk level is determined based on the auxiliary operation instruction; If the operational risk level is the first risk level, the corresponding operation is executed based on the auxiliary operation instruction; or, if the operational risk level is the second risk level, the auxiliary operation instruction is verified to obtain a verification result, and the corresponding operation is executed based on the verification result and the auxiliary operation instruction.

4. The auxiliary control method for a smart kitchen appliance according to claim 3, characterized in that, If the operational risk level is the first risk level, the corresponding operation is executed based on the auxiliary operation instruction, including: If the operational risk level is the first risk level, the auxiliary operation instruction will be sent to the execution module of the smart kitchen appliance; If a successful execution response is received from the execution module based on the auxiliary operation instruction, status change information is sent to the second terminal; the status change information is used to display the status of the execution module after performing the operation on the second terminal.

5. The auxiliary control method for a smart kitchen appliance according to claim 3, characterized in that, The step of verifying the auxiliary operation command and obtaining the verification result includes: Based on the auxiliary operation command, a confirmation prompt message is sent to the second terminal; the confirmation prompt message is used to instruct the second user of the second terminal to perform a secondary confirmation operation. If the second terminal returns secondary confirmation information generated based on the secondary confirmation operation, extract the confirmation keyword information and confirmation duration information from the secondary confirmation information; If both the confirmation keyword information and the confirmation duration information meet the confirmation requirements, the verification is determined to be passed as the verification result; or, if either the confirmation keyword information or the confirmation duration information does not meet the confirmation requirements, the verification is determined to be failed as the verification result.

6. The auxiliary control method for a smart kitchen appliance according to claim 5, characterized in that, The step of performing the corresponding operation based on the verification result and the auxiliary operation instruction includes: If the verification result is that the verification is successful, a confirmation message is sent to the first terminal. The auxiliary operation instructions are sent to the execution module of the smart kitchen appliance; If a successful execution response is received from the execution module based on the auxiliary operation instruction, status change information is sent to the second terminal; An auxiliary operation log is generated and stored based on the secondary confirmation information and the auxiliary operation instructions.

7. The auxiliary control method for a smart kitchen appliance according to claim 1, characterized in that, In the case where a video call is established between the first terminal and the second terminal, if an auxiliary control command carrying the control token is received from the first terminal, the corresponding operation is performed based on the auxiliary control command, including: When a video call is established between the first terminal and the second terminal, if an emergency termination command is received from the second terminal, the validity of the control token is terminated; the emergency termination command is triggered by the second user of the second terminal based on the emergency termination control. Send an operation abort notification message to the first terminal; The emergency lock command is sent to the execution module of the smart kitchen appliance; the emergency lock command is used to instruct the smart kitchen appliance to enter a preset safety lock state.

8. The auxiliary control method for a smart kitchen appliance according to claim 1, characterized in that, If an auxiliary control command carrying the control token is received from the first terminal, the corresponding operation is performed based on the auxiliary control command, including: Receive kitchen appliance image information collected and sent by the second terminal; The kitchen appliance image information is sent to the first terminal; the kitchen appliance image information is used to perform image recognition processing based on the first terminal to obtain operating component information; the operating component information is used to generate virtual operating components on the first terminal and overlay them on the video call screen of the first terminal. If a virtual operation instruction carrying the control token is received from the first terminal, triggered by the virtual operation component, the control parameters are determined based on the virtual operation instruction. The control parameters are encapsulated into the auxiliary control commands; The auxiliary control command is sent to the second terminal through the signaling channel of the video call; the auxiliary control command is used by the second terminal to send to the execution module of the smart kitchen appliance through a local protocol.

9. The auxiliary control method for a smart kitchen appliance according to claim 8, characterized in that, The method further includes: If the execution module receives a successful execution response based on the auxiliary control command, it sends execution feedback information to the first terminal; the execution feedback signal is used to trigger the first terminal to update the status of the virtual operating device on the video call screen.

10. An auxiliary control device for a smart kitchen appliance, characterized in that, The device includes: The request generation module is used to generate a video call request based on the remote control request sent by the first terminal. The request sending module is used to send the video call request to the second terminal; The token generation module is used to generate a control token if a video answering response is received from the second terminal. The token sending module is used to send the control token to the first terminal; The auxiliary control module is used to perform corresponding operations based on the auxiliary control command sent by the first terminal carrying the control token when a video call is established between the first terminal and the second terminal.

Images