Multi-terminal control method, system and device based on AI and storage medium
By inputting operation commands through sensors and determining the connection between the terminal device and the AI model, the problem of the single application scenario of AI terminal devices is solved, and flexible interaction and multi-scenario adaptation across devices are realized.
Patent Information
- Application Number
- CN202511900733.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-16
- Publication Date
- 2026-03-03
AI Technical Summary
Existing AI terminal devices have limited application scenarios, lack the ability to interact flexibly with other devices, and are difficult to adapt to diverse scenario needs.
By inputting operation commands to the AI terminal device through sensors, other connected terminal devices are identified, and the operation commands are analyzed. Based on the analysis results, the execution terminal device and/or AI model are determined to achieve cross-device command execution.
It enables cross-device instruction execution between AI terminal devices and other terminal devices, combining the advantages of independent operation and multi-device collaborative interaction. Its functions are flexible and adaptable to the needs of various scenarios.
Smart Images
Figure CN121603898A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent interactive technology, and in particular to an AI-based multi-terminal control method, system, device, and storage medium. Background Technology
[0002] Artificial intelligence (AI) is a new technical science that studies, develops, and applies theories, methods, technologies, and application systems to simulate, extend, and expand human intelligence. In recent years, with the rapid development of AI technology, people have been using AI devices to simulate the behavior of human companions, providing users with emotional support and social interaction.
[0003] However, existing AI terminal devices mostly adopt a single, fixed AI model deployment model. For example, AI models deployed only on the terminal are difficult to update, while the response speed of large-parameter AI models deployed in the cloud is affected by the terminal device's computing resources, memory, and power consumption, making it difficult to adapt to diverse scenario requirements. Furthermore, most AI terminal devices only implement TV-related functions or can only connect to specific terminal devices, lacking a unified interaction protocol and collaboration mechanism, making it difficult to connect to other terminal devices. Therefore, existing AI terminal devices have limited application scenarios and lack the ability to flexibly interact with other devices.
[0004] Existing technologies still suffer from limitations such as the limited application scenarios of AI terminal devices and the lack of flexible interaction capabilities with other devices. Therefore, existing technologies need further improvement. Summary of the Invention
[0005] The technical problem to be solved by the present invention is to provide an AI-based multi-terminal control method, system, device and storage medium to address the shortcomings of existing technologies, thereby solving the problem that existing AI terminal devices have limited application scenarios and lack the ability to interact flexibly with other devices.
[0006] The technical solution adopted by this invention to solve the technical problem is as follows: In a first aspect, the present invention provides an AI-based multi-terminal control method, comprising: Operation commands are input to a first terminal device through at least one type of sensor; wherein, the first terminal device is an AI terminal device, and the AI terminal device includes at least one type of AI model; Identify other terminal devices currently connected to the first terminal device; Analyze the input operation instructions, and determine the terminal device and / or AI model to execute the operation instructions based on the analysis results; The operation instructions are executed based on the determined terminal device and / or AI model.
[0007] In one implementation, the sensor includes one or more of a touch sensor, a sound sensor, and an image sensor; the operation command includes one or more of a touch command, a voice command, and an image command.
[0008] In one implementation, the AI terminal device includes at least an on-device AI model and a connected cloud AI model.
[0009] In one implementation, determining other terminal devices currently connected to the first terminal device includes: Determine whether the first terminal device is currently connected to other terminal devices; If the determination result is that the first terminal device is connected to other terminal devices, then the second terminal device connected to the first terminal device is determined.
[0010] In one implementation, when the determination result is that the first terminal device is not connected to other terminal devices, the analysis of the input operation command, and the determination of the terminal device and / or AI model to execute the operation command based on the analysis result, includes: Analyze the input operation instructions to obtain the application scenarios of the operation instructions; Determine whether an AI model exists to execute the operation instructions based on the application scenario; If an AI model exists that executes the operation instruction, output the first terminal device and the AI model in the first terminal device used to execute the operation instruction; If no AI model exists to execute the operation instructions, output the first terminal device.
[0011] In one implementation, when the determination result is that the first terminal device is connected to other terminal devices, the analysis of the input operation command, based on the analysis result, determines the terminal device and / or AI model that executes the operation command, including: Analyze the input operation instructions to obtain the application scenarios of the operation instructions; The terminal device that executes the operation command is determined based on the application scenario; If the terminal device executing the operation instruction is the first terminal device, output the first terminal device and the AI model in the first terminal device used to execute the operation instruction; If the terminal device executing the operation instruction is the second terminal device, output the second terminal device.
[0012] In one implementation, the AI-based multi-terminal control method further includes: The execution result of the operation command is fed back.
[0013] Secondly, the present invention provides an AI-based multi-terminal control system, comprising: A sensor module includes at least one type of sensor for inputting operation commands to a first terminal device; wherein the first terminal device is an AI terminal device, and the AI terminal device includes at least one type of AI model; The interface module is used to determine other terminal devices currently connected to the first terminal device; The signal analysis and processing module is used to analyze the input operation instructions, determine the terminal device and / or AI model to execute the operation instructions based on the analysis results, and execute the operation instructions based on the determined terminal device and / or AI model.
[0014] Thirdly, the present invention provides a terminal, including: a processor and a memory, wherein the memory stores an AI-based multi-terminal control program, and the AI-based multi-terminal control program, when executed by the processor, is used to implement the operation of the AI-based multi-terminal control method as described in the first aspect.
[0015] Fourthly, the present invention also provides a computer-readable storage medium storing an AI-based multi-terminal control program, which, when executed by a processor, is used to implement the operation of the AI-based multi-terminal control method as described in the first aspect.
[0016] The present invention, by employing the above technical solution, has the following effects: This invention inputs operation commands to a first terminal device through at least one type of sensor. Using an AI terminal device containing at least one type of AI model, it can accurately respond to operation commands from multiple types of sensors. By identifying other terminal devices currently connected to the first terminal device, the connection relationship between the AI terminal device and other terminal devices is obtained. The input operation commands are analyzed, and based on the analysis results, the terminal device and / or AI model to execute the operation commands are determined. The operation commands are executed based on the determined terminal device and / or AI model, achieving cross-device command execution. It combines the advantages of independent operation and multi-device collaborative interaction, offering flexible functional expansion and adaptability to various usage scenarios. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.
[0018] Figure 1This is a flowchart of the AI-based multi-terminal control method in this invention.
[0019] Figure 2 This is a schematic diagram of the "cloud-edge-device" AI model architecture in one implementation of the present invention.
[0020] Figure 3 This is a schematic diagram of the overall structure of the AI model terminal device in one implementation of the present invention.
[0021] Figure 4 This is a flowchart illustrating how a first terminal device executes an operation instruction independently in one implementation of the present invention.
[0022] Figure 5 This is a schematic diagram illustrating a scenario in which the first terminal device and the second terminal device system execute operation instructions in one implementation of the present invention.
[0023] Figure 6 This is a flowchart illustrating the collaborative execution of operation instructions by a first terminal device and a second terminal device in one implementation of the present invention.
[0024] Figure 7 This is a schematic diagram illustrating a scenario in which the first terminal device and the second terminal device system execute operation instructions in one implementation of the present invention.
[0025] Figure 8 This is a flowchart illustrating the collaborative execution of operation instructions by a first terminal device and a second terminal device in one implementation of the present invention.
[0026] Figure 9 This is an internal module architecture diagram of the first terminal device in one implementation of the present invention.
[0027] Figure 10 This is a functional schematic diagram of the terminal in one implementation of the present invention.
[0028] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0029] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0030] Exemplary methods Existing AI terminal devices mostly adopt a single, fixed AI model deployment model. For example, AI models deployed only on the terminal are cumbersome to update, while the response speed of large-parameter AI models deployed in the cloud is affected by the terminal device's computing resources, memory, and power consumption, making it difficult to adapt to diverse scenario requirements. Furthermore, most AI terminal devices only implement TV-related functions or can only connect to specific terminal devices, lacking a unified interaction protocol and collaboration mechanism, making it difficult to connect to other terminal devices. Therefore, existing AI terminal devices have limited application scenarios and lack the ability to flexibly interact with other devices.
[0031] To address the above-mentioned technical problems, this invention provides an AI-based multi-terminal control method, system, device, and storage medium, comprising: inputting operation commands to a first terminal device via at least one type of sensor; wherein the first terminal device is an AI terminal device, and the AI terminal device includes at least one type of AI model; determining other terminal devices currently connected to the first terminal device; analyzing the input operation commands, and determining the terminal device and / or AI model to execute the operation commands based on the analysis results; and executing the operation commands based on the determined terminal device and / or AI model. This invention uses an AI terminal device containing at least one type of AI model, which can accurately respond to operation commands from multiple types of sensors; the AI terminal device is connected to other terminal devices, realizing cross-device command execution, combining the advantages of independent operation and multi-device collaborative interaction, with flexible functional expansion to adapt to the usage needs of multiple scenarios.
[0032] like Figure 1 As shown, this embodiment of the invention provides an AI-based multi-terminal control method, including the following steps: Step S100: Input operation instructions to the first terminal device through at least one type of sensor; wherein, the first terminal device is an AI terminal device, and the AI terminal device includes at least one type of AI model.
[0033] In this embodiment, the sensor includes, but is not limited to, any one or a combination of touch sensors, sound sensors, and image sensors. For example, the sensor for inputting operation commands can be only a touch sensor, or it can be a combination of a touch sensor and a sound sensor. The touch sensor can be a touchscreen, the sound sensor can be a microphone, and the image sensor can be a camera.
[0034] In this embodiment, in addition to the sensor that inputs operation commands to the first terminal device, there is also an output sensor. Such output sensors include, but are not limited to, vision sensors, infrared sensors, and sound output sensors. Vision sensors include displays, infrared sensors include infrared emitting modules, and sound output sensors include speakers.
[0035] In this embodiment, the sensor can also be a combination of an input operation command sensor and an output sensor, such as a touch display integrated screen.
[0036] In this embodiment, the input operation commands include, but are not limited to, any one or a combination of touch commands, voice commands, and image commands, and there is a one-to-one correspondence between the operation commands and the sensors. For example, touch commands correspond to touch screens or integrated touch displays, voice commands correspond to microphones, and image commands correspond to cameras.
[0037] In this embodiment, the AI terminal device includes at least an on-device AI model and a connected cloud AI model. The on-device AI model can handle lightweight operation commands, such as user voice recognition, face recognition, and object recognition, while the connected cloud AI model is used in scenarios where the input commands need to process a large amount of data.
[0038] It should be noted that AI terminal devices can also use various AI models, including those based on a "cloud-edge-device" architecture. For example... Figure 2 The diagram shown illustrates a "cloud-edge-device" AI model architecture in one implementation of this embodiment. The cloud includes a CloueCore component, multiple control nodes, and multiple computing nodes. The edge includes edge nodes, each containing an EdgCore component. The edge nodes are controlled by the CloueCore component in the cloud. The edge nodes communicate with various modules of the terminal device, such as sensors and cameras, through the EdgeX Foundry component, or directly with the terminal device, such as smart home devices, new retail IoT devices, smart agriculture IoT devices, and smart aquaculture IoT devices.
[0039] In one embodiment, an AI model terminal device is provided, such as Figure 3 The diagram shows the overall structure of the AI model terminal device. The AI model terminal device consists of a base 1 and a body 2. A Pogo-Pin interface module is located below the base 1, and the Pogo-Pin interface module uses the USB 2.0 protocol (+5V, D+, D-, GND) for data transmission control. The body 2 includes a motherboard, battery, wireless module, infrared transmitter module 21, left microphone 22, right microphone 23, camera module 24, touch display screen 25, microphone, speaker, temperature and humidity sensor, etc.
[0040] like Figure 1 As shown, this embodiment of the invention provides an AI-based multi-terminal control method, including the following steps: Step S200: Determine other terminal devices currently connected to the first terminal device.
[0041] It should be noted that during actual use, the first terminal device may be connected to other terminal devices, or it may be used independently without being connected to other terminal devices. When the number and type of connected devices change, the method of executing the corresponding operation instructions will also change. For example, the operation instruction could be a first instruction executed directly by the first terminal device, a second instruction executed by the edge AI model on the first terminal device, a third instruction executed collaboratively by the first terminal device and the edge AI model, a fourth instruction executed by the first terminal device and the second terminal device, and so on. Therefore, after receiving the operation instruction from the sensor, it is necessary to determine whether the first terminal device is connected to other terminal devices, and if so, the type of the connected terminal devices also needs to be determined.
[0042] Specifically, in one implementation of this embodiment, step S200 includes the following steps: Step S201: Determine whether the first terminal device is currently connected to other terminal devices.
[0043] In this embodiment, the first terminal device further includes an interface module, which determines whether the first terminal device is connected to other terminal devices by whether a specific interface module is occupied.
[0044] In this embodiment, the interface module of the first terminal device is a Pogo-Pin interface module, that is, a spring pin interface module. This type of interface is widely used in the fields of charging, signal transmission and device connection. Using this type of interface can significantly improve the scalability of the first terminal device's functions.
[0045] Step S202: If the determination result is that the first terminal device is connected to other terminal devices, then determine the second terminal device connected to the first terminal device.
[0046] It should be noted that in most application scenarios, the first terminal device is used to connect to another separate terminal device. Therefore, other terminal devices connected to the first terminal device are regarded as second terminal devices. However, since the first terminal device uses the Pogo-Pin interface module, it is only necessary to extend the Pogo-Pin interface module to achieve the effect of connecting to multiple other terminal devices at the same time.
[0047] In this embodiment, when the Pogo-Pin interface module determines that the first terminal device has other connected terminal devices, it is necessary to determine the device type of the second terminal device connected to the first terminal device, so that the first terminal device can realize signal transmission and control of the second terminal device through the Pogo-Pin interface module.
[0048] like Figure 1 As shown, this embodiment of the invention provides an AI-based multi-terminal control method, including the following steps: Step S300: Analyze the input operation instructions, and determine the terminal device and / or AI model that executes the operation instructions based on the analysis results.
[0049] Specifically, when the determination result is that the first terminal device is not connected to other terminal devices, the analysis of the input operation command, based on the analysis result, determines the terminal device and / or AI model to execute the operation command, including the following steps: Step S301a: Analyze the input operation instructions to obtain the application scenario of the operation instructions; Step S301b: Determine whether there is an AI model that can execute the operation command based on the application scenario; Step S301c: If an AI model exists that executes the operation instruction, output the first terminal device and the AI model in the first terminal device used to execute the operation instruction. Step S301d: If there is no AI model that can execute the operation instruction, output the first terminal device.
[0050] In this embodiment, when the determination result is that the first terminal device is not connected to other terminal devices, only the first terminal device executes the corresponding operation command. However, the first terminal device can execute operation commands in multiple ways, such as a first command executed directly by the first terminal device, a second command executed by the edge AI model on the first terminal device, or a third command executed collaboratively by the first terminal device and the edge AI model. Therefore, it is necessary to analyze the input operation commands to obtain the application scenarios of the operation commands, such as voice communication, question reply, intelligent retrieval, etc., and determine the AI model that can execute the operation commands based on the obtained application scenarios.
[0051] It should be noted that when the first terminal device cannot execute the operation command, it may be because the operation command is undefined or cannot be implemented. In this case, the terminal device that outputs the operation command is still the first terminal device. This means that the first terminal device informs the user that the current operation command cannot be executed through the output type sensor and directly ends the processing of the operation command.
[0052] In one implementation of this embodiment, when the determination result indicates that the first terminal device is connected to other terminal devices, the analysis of the input operation command, and the determination of the terminal device and / or AI model to execute the operation command based on the analysis result, includes the following steps: Step S302a: Analyze the input operation instructions to obtain the application scenario of the operation instructions; Step S302b: Determine the terminal device that will execute the operation command based on the application scenario; Step S302c: If the terminal device executing the operation instruction is the first terminal device, output the first terminal device and the AI model in the first terminal device used to execute the operation instruction; Step S302d: If the terminal device executing the operation instruction is the second terminal device, output the second terminal device.
[0053] In this embodiment, when the determination result is that the first terminal device is connected to other terminal devices, the device executing the corresponding operation instruction may be the first terminal device or the second terminal device, such as a TV or mobile device. Generally, other terminal devices do not contain AI models. Therefore, when analyzing operation instructions, it is first determined whether the current operation instruction is executed by other terminal devices. If the operation instruction is directly executed by other terminal devices, there is no need to further determine the AI model executing the operation instruction.
[0054] It should be noted that, in special circumstances, the operation instructions can also be executed collaboratively by the AI model of the first terminal device and the second terminal device. In this case, the application scenario of the operation instructions corresponds to multiple terminal devices.
[0055] For special cases where multiple terminal devices collaboratively execute operation instructions, the analysis of the input operation instructions, and the determination of the terminal device and / or AI model executing the operation instructions based on the analysis results, further includes the following steps: Step S303a: Analyze the input operation instructions to obtain the application scenario of the operation instructions; Step S303b: Determine all terminal devices that will execute the operation command based on the application scenario; Step S303c: If the first terminal device exists among all terminal devices executing the operation instruction, output the first terminal device and the AI model in the first terminal device used to execute the operation instruction; Step S303d: If the second terminal device exists among the terminal devices executing the operation instruction, output the second terminal device; Step S303e: Integrate all outputs to obtain the final result representing the terminal device and / or AI model executing the operation instruction, and output the final result.
[0056] like Figure 1 As shown, this embodiment of the invention provides an AI-based multi-terminal control method, including the following steps: Step S400: Execute the operation instructions based on the determined terminal device and / or AI model.
[0057] It should be noted that when a specific terminal device and / or AI model executes the operation instructions, there are multiple possibilities, corresponding to the execution of different types of operation instructions. For example, the first terminal device may execute the operation instructions alone, or the edge AI of the first terminal device may execute the operation instructions alone, or the cloud AI connected to the first terminal device may execute the operation instructions alone.
[0058] like Figure 1 As shown, this embodiment of the invention provides an AI-based multi-terminal control method, which further includes the following steps: Step S500: Feedback on the execution result of the operation instruction.
[0059] In this embodiment, when the terminal device executing the operation command is the first terminal device, the execution result of the operation command is output through the output sensor of the first terminal device, for example, the execution result of the operation command is fed back through voice and / or touch display integrated screen; when the terminal device executing the operation command is a combination of the first terminal device and other terminal devices, the execution result of the operation command is fed back through the terminal device connected to the first terminal device.
[0060] Furthermore, depending on the terminal device and / or AI model executing the operation instructions, the present invention also includes the following embodiments: In one embodiment of the present invention, the output terminal device is a first terminal device, that is, a scenario in which the first terminal device executes the operation instructions alone.
[0061] like Figure 4 The diagram shows a flowchart of a first terminal device executing an operation instruction independently in one implementation of the present invention, including the following steps: Step a, input touch commands via the touchscreen and / or voice commands via the microphone; Step b: Analyze the input operation command through the signal analysis and processing module, and determine the terminal device and / or AI model that executes the operation command based on the analysis results; Step c: Execute the operation instructions based on the determined terminal device and / or AI model; Step d: The execution result of the operation command is fed back through the voice module and the integrated touch display screen.
[0062] This embodiment is applicable to scenarios where users carry the first terminal device out to chat and communicate with it, or use it as a smart speaker. When used alone, relevant information commands are usually input externally through two means, such as inputting relevant text commands through the touch screen or inputting relevant commands through voice.
[0063] In one embodiment of the present invention, the output terminal device is a first terminal device and a second terminal device, that is, a scenario in which the first terminal device and the second terminal device cooperate to execute operation instructions, such as... Figure 5 The diagram shows a scenario where a first terminal device and a second terminal device work together to execute operation commands. The second terminal device is a television. Typically, the first terminal device is placed above the television, and data is transmitted between the first terminal device and the television via the Pogo-Pin interface. The Pogo-Pin interface can also be used by the television to supply power to the first terminal device.
[0064] In one implementation of this embodiment, a 4-pin Pogo-Pin interface can be reserved at the top of the second terminal device, i.e., the TV, or a magnetic structure for connecting to the second terminal device can be provided at the bottom of the first terminal device.
[0065] like Figure 6 The diagram shows a flowchart of a first terminal device and a second terminal device collaboratively executing operation instructions in one implementation of the present invention, including the following steps: Step a: Input one or more of the corresponding touch commands, voice commands, and image commands through any one or more of the three types of sensors: touch screen, microphone, and camera; Step b: Analyze the input operation command through the signal analysis and processing module, and determine the terminal device and / or AI model that executes the operation command based on the analysis results; Step c: Execute the operation instructions based on the determined terminal device and / or AI model; Furthermore, when the output terminal device includes the first terminal device and the first terminal device needs to directly participate in the execution of operation instructions, it executes the corresponding operation instructions through the infrared emission module, such as control instructions for controlling other terminal devices.
[0066] Step d: The execution result of the operation command is fed back through the voice module and the integrated touch display screen. When the executed command is a control command to control other terminal devices, the execution result of the operation command is directly fed back through the controlled terminal device.
[0067] In this embodiment, the second terminal device can be any terminal device capable of executing control commands, in addition to a television. The scenarios corresponding to this embodiment generally include voice control, remote conferencing, or intelligent control, as detailed below: Voice control works as follows: When a user issues a voice command, the first terminal device receives it via its microphone module, and the signal analysis and processing module determines which terminal device the command is intended for. If the command is for the TV, such as "turn the volume down," the TV will automatically lower the volume. If the command is for other home appliances, the first terminal device will control them via its infrared transmitter module. For example, if the command is "lower the air conditioner temperature," the first terminal device will send an air conditioner remote control code via its infrared transmitter module to lower the air conditioner temperature. If the command cannot be matched by the local database, it will be distributed to the edge or cloud AI module based on the amount of information and computation required, and the corresponding result will be displayed on the screen or output to the user via the speaker.
[0068] In applications such as remote conferencing or home calls, the primary terminal device transmits video content to a television display via the D+ and D- pins on its Pogo-Pin. The large screen provides a clear and convenient view of all participants. Simultaneously, the user's image and audio are transmitted to other participants via the camera and microphone. Furthermore, based on the meeting topic and shooting environment, the primary terminal device utilizes its on-device AI processing module for relevant processing, such as audio noise reduction and AI-generated background images, to process and transmit the desired audio and video content. Finally, the meeting's audio and video content and minutes are automatically named and saved on the primary terminal device for easy retrieval by the user.
[0069] With intelligent control, when the first terminal device captures the user's situation through the camera, the edge AI processing module will perform corresponding analysis and processing. For example, if the analysis result indicates that the user is asleep, the first terminal device's display screen will show a sleep emoji, and the signal analysis and processing module will send a command to the TV to adjust the TV's volume and brightness to the lowest level, while simultaneously triggering the recording function. This avoids waking the user and allows the user to review the content without missing anything.
[0070] In one embodiment of the present invention, the output terminal device is a first terminal device and a second terminal device, that is, a scenario in which the first terminal device and the second terminal device cooperate to execute operation instructions. If the second terminal device is a mobile device, the corresponding schematic diagram of the scenario in which the first terminal device and the second terminal device cooperate to execute operation instructions is as follows. Figure 7 As shown, the first terminal device is typically placed on top of the mobile device, and data is transmitted between the first terminal device and the mobile device via the Pogo-Pin interface. The Pogo-Pin interface can also be used for the mobile device to supply power to the first terminal device.
[0071] In one implementation of this embodiment, a 4-pin Pogo-Pin interface can be reserved on the top of the second terminal device, i.e., the mobile device, or a magnetic structure for connecting to the second terminal device can be provided on the bottom of the first terminal device.
[0072] like Figure 8 The diagram shows a flowchart of a first terminal device and a second terminal device collaboratively executing operation instructions in one implementation of the present invention, including the following steps: Step a: Input one or more of the corresponding touch commands, voice commands, and image commands through any one or more of the three types of sensors: touch screen, microphone, and camera; Step b: Analyze the input operation command through the signal analysis and processing module, and determine the terminal device and / or AI model that executes the operation command based on the analysis results; Step c: Execute the operation instructions based on the determined terminal device and / or AI model; In addition, when the output terminal device includes the first terminal device and the first terminal device needs to directly participate in the execution of operation instructions, it executes the corresponding operation instructions through the infrared emission module, such as control instructions for controlling other terminal devices; When the output terminal device includes a second terminal device and the second terminal device is a mobile device, it executes mobile operation instructions, such as controlling the mobile device to move by using wheels; Step d: The execution result of the operation command is fed back through the voice module and the integrated touch display screen. When the executed command is a control command to control other terminal devices, the execution result of the operation command is directly fed back through the controlled terminal device.
[0073] The second terminal device in this embodiment is a mobile device. Therefore, the corresponding scenario in this embodiment generally occurs in the timed inspection of mobile devices, electronic pets, or controlling the movement of home appliances, as detailed below: The system performs scheduled inspections. First, the first terminal device uses a camera to identify the user's home environment. The signal analysis and processing module transmits the image information to the edge AI module for processing, obtaining the mobile device's path planning information, which is then sent back to the signal analysis and processing module. The module then sends corresponding instructions to the mobile device to control it to follow the planned path. Second, the first terminal device follows the planned path and uses the camera to identify any abnormalities in the user's home, such as whether the air conditioner, lights, and faucets are turned off, or whether elderly people or children have fallen. If any abnormalities are detected, the device promptly provides feedback to family members via voice or sends a message to the user via a wireless module.
[0074] The electronic pet function involves pairing a primary terminal device with a mobile device, transforming the mobile device into an adorable pet shape, allowing users to treat it like a pet, with emotional interaction, verbal communication, feedback, and reminders, making users feel emotionally cared for.
[0075] Controlling the movement of home appliances, for example, when a user is in the living room and wants to turn off the air conditioner in the bedroom, they can use the voice command "Go turn off the bedroom air conditioner." First, after the microphone collects relevant voice information, the signal analysis module matches it with the local key control library of the first terminal device. The first terminal device then processes and controls the movement itself via the infrared transmission module. Next, the edge AI module sends the mobile device's path information to the signal analysis and processing module, which then distributes it to the mobile device for execution, guiding it to its destination along the corresponding path. This achieves the autonomous movement control function.
[0076] This embodiment achieves the following technical effects through the above technical solution: This embodiment inputs operation commands to a first terminal device through at least one type of sensor. Using an AI terminal device containing at least one type of AI model, it can accurately respond to operation commands from multiple types of sensors. By determining other terminal devices currently connected to the first terminal device, the connection relationship between the AI terminal device and other terminal devices is obtained, and the input operation commands are analyzed. Based on the analysis results, the terminal device and / or AI model to execute the operation commands are determined. The operation commands are executed based on the determined terminal device and / or AI model, realizing cross-device command execution. It has the advantages of independent operation and multi-device collaborative interaction, flexible function expansion, and adaptability to the usage needs of multiple scenarios.
[0077] Exemplary device Based on the above embodiments, the present invention also provides an AI-based multi-terminal control system, comprising: A sensor module includes at least one type of sensor for inputting operation commands to a first terminal device; wherein the first terminal device is an AI terminal device, and the AI terminal device includes at least one type of AI model; The interface module is used to determine other terminal devices currently connected to the first terminal device; The signal analysis and processing module is used to analyze the input operation instructions, determine the terminal device and / or AI model to execute the operation instructions based on the analysis results, and execute the operation instructions based on the determined terminal device and / or AI model.
[0078] like Figure 9 The diagram shown is an internal module architecture diagram of the first terminal device in this embodiment. The first terminal device is the core part of the AI-based multi-terminal control system, and specifically includes: A battery used to power the first terminal device; The power supply processing unit module is used for power distribution, monitoring, and management of the first terminal device; The Pogo-Pin interface module is used to connect with other terminal devices such as second terminal devices and third terminal devices, or to connect with a power supply to provide additional power to the power supply processing unit of each module, and to connect with a signal difference analysis processing module to realize signal transmission between other terminal devices and the first terminal device; The signal analysis and processing module is the core part of the first terminal device. It can acquire operation commands, analyze and process operation commands, and is also used to control the AI model on the control side and output control commands to the AI model connected to the cloud. The edge AI module is the AI module installed on the first terminal device, which can perform lightweight data processing operations; And various sensor modules, including: a temperature and humidity sensor module for collecting temperature and humidity data, a speaker module for outputting sound signals, a wireless communication module for realizing wireless data transmission; an infrared emitting module for outputting infrared signals; a camera module for collecting image signals; left and right microphone modules for collecting sound signals; and an integrated touch screen module for transmitting touch signals and display signals, etc.
[0079] In this embodiment, the AI-based multi-terminal control system also includes a cloud-based AI model, which is used to process the corresponding data when the user-input instructions require a large amount of computation, retrieval, and analysis.
[0080] Based on the above embodiments, the present invention also provides a device, the principle block diagram of which can be as follows: Figure 10 As shown.
[0081] The device includes: a processor, memory, interface, display screen, and communication module connected via a system bus; wherein, the processor provides computing and control capabilities; the memory includes a computer-readable storage medium and internal memory; the computer-readable storage medium stores an operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the computer-readable storage medium; the interface is used to connect to external devices; the display screen is used to display relevant information; and the communication module is used to communicate with a cloud server or other devices.
[0082] When executed by the processor, this computer program is used to implement the operation of an AI-based multi-terminal control method.
[0083] It will be understood by those skilled in the art that Figure 10 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the device to which the present invention is applied. Specific devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0084] In one embodiment, a device is provided, comprising: a processor and a memory, the memory storing an AI-based multi-terminal control program, which, when executed by the processor, is used to implement the operation of the AI-based multi-terminal control method described above.
[0085] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores an AI-based multi-terminal control program, which, when executed by a processor, is used to implement the operation of the AI-based multi-terminal control method described above.
[0086] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, database, or other media used in the embodiments provided by this invention can include both non-volatile and volatile memory.
[0087] In summary, this invention provides an AI-based multi-terminal control method, system, device, and storage medium, comprising: inputting operation commands to a first terminal device via at least one type of sensor; wherein the first terminal device is an AI terminal device, and the AI terminal device includes at least one type of AI model; determining other terminal devices currently connected to the first terminal device; analyzing the input operation commands, and determining the terminal device and / or AI model to execute the operation commands based on the analysis results; and executing the operation commands based on the determined terminal device and / or AI model. This invention uses an AI terminal device containing at least one type of AI model, which can accurately respond to operation commands from multiple types of sensors; the AI terminal device is connected to other terminal devices, realizing cross-device command execution, combining the advantages of independent operation and multi-device collaborative interaction, with flexible functional expansion to adapt to the usage needs of multiple scenarios.
[0088] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.
Claims
1. A multi-terminal control method based on AI, characterized in that, include: Operation commands are input to a first terminal device through at least one type of sensor; wherein, the first terminal device is an AI terminal device, and the AI terminal device includes at least one type of AI model; Identify other terminal devices currently connected to the first terminal device; Analyze the input operation instructions, and determine the terminal device and / or AI model to execute the operation instructions based on the analysis results; The operation instructions are executed based on the determined terminal device and / or AI model.
2. The AI-based multi-terminal control method according to claim 1, characterized in that, The sensor includes any one or more of a touch sensor, a sound sensor, and an image sensor; the operation command includes any one or more of a touch command, a voice command, and an image command.
3. The AI-based multi-terminal control method according to claim 1, characterized in that, The AI terminal device includes at least an on-device AI model and a connected cloud AI model.
4. The AI-based multi-terminal control method according to claim 1, characterized in that, The step of determining other terminal devices currently connected to the first terminal device includes: Determine whether the first terminal device is currently connected to other terminal devices; If the determination result is that the first terminal device is connected to other terminal devices, then the second terminal device connected to the first terminal device is determined.
5. The AI-based multi-terminal control method according to claim 4, characterized in that, When the determination result is that the first terminal device is not connected to other terminal devices, the analysis of the input operation command, based on the analysis result, determines the terminal device and / or AI model that executes the operation command, including: Analyze the input operation instructions to obtain the application scenarios of the operation instructions; Determine whether an AI model exists to execute the operation instructions based on the application scenario; If an AI model exists that executes the operation instruction, output the first terminal device and the AI model in the first terminal device used to execute the operation instruction; If no AI model exists to execute the operation instructions, output the first terminal device.
6. The AI-based multi-terminal control method according to claim 4, characterized in that, When the determination result is that the first terminal device is connected to other terminal devices, the operation command input is analyzed, and the terminal device and / or AI model that executes the operation command is determined based on the analysis result, including: Analyze the input operation instructions to obtain the application scenarios of the operation instructions; The terminal device that executes the operation command is determined based on the application scenario; If the terminal device executing the operation instruction is the first terminal device, output the first terminal device and the AI model in the first terminal device used to execute the operation instruction; If the terminal device executing the operation instruction is the second terminal device, output the second terminal device.
7. The AI-based multi-terminal control method according to claim 1, characterized in that, The AI-based multi-terminal control method also includes: The execution result of the operation command is fed back.
8. A multi-terminal control system based on AI, characterized in that, include: A sensor module includes at least one type of sensor for inputting operation commands to a first terminal device; wherein the first terminal device is an AI terminal device, and the AI terminal device includes at least one type of AI model; The interface module is used to determine other terminal devices currently connected to the first terminal device; The signal analysis and processing module is used to analyze the input operation instructions, determine the terminal device and / or AI model to execute the operation instructions based on the analysis results, and execute the operation instructions based on the determined terminal device and / or AI model.
9. A device, characterized in that, include: The processor and memory, wherein the memory stores an AI-based multi-terminal control program, which, when executed by the processor, is used to implement the operation of the AI-based multi-terminal control method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an AI-based multi-terminal control program, which, when executed by a processor, is used to implement the operation of the AI-based multi-terminal control method as described in any one of claims 1-7.