Electronic system and control method thereof

Abstract

To provide an electronic system and a control method thereof controlling connected peripheral devices by using a projection device as a central control device.SOLUTION: An electronic system includes at least one peripheral device, and a projection device having a communication interface and connecting the at least one peripheral device via the communication interface. A control method of the electronic system includes transmitting an original command to a natural language model via the projection device, generating at least one standard command based on the original command via the natural language model according to a fact that the original command corresponds to the operation of the at least one peripheral device, and controlling the at least one peripheral device to perform an operation corresponding to the original command via the projection device based on the at least one standard command.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a control mechanism, and more particularly to an electronic system capable of controlling peripheral devices and a control method thereof. [Background technology]

[0002] The current control method of a projection device is for a user to manually operate a remote controller of the projection device or to operate the projection settings of the projection device through a human-machine interface on the projection device, which makes the control method of the conventional projection device quite inconvenient.

[0003] The "Background Art" section is merely intended to aid in the understanding of the present invention, and as such, the disclosure of the "Background Art" section may include some content that does not constitute known technology to those skilled in the art. Nothing in the "Background Art" section represents that the content or the problem that one or more embodiments of the present invention seek to solve was already known or recognized by those skilled in the art prior to the filing of the present invention. Summary of the Invention [Problem to be solved by the invention]

[0004] When a user wants to control a peripheral device connected to a projection device, the user must first check the type and connection status of the peripheral device connected to the projection device before manually controlling it. In addition, each function of the peripheral device must be operated independently and separately, which makes controlling the conventional peripheral devices quite inconvenient. [Means for solving the problem]

[0005] The present invention provides an electronic system and a control method thereof that can control peripheral devices connected to a projection device as a central control device.

[0006] Other objects and advantages of the present invention can be further understood from the disclosed technical features of the present invention.

[0007] In order to achieve some or all of the above objects or other objects, the present invention provides a method for controlling an electronic system, the electronic system including a projection device and at least one peripheral device, the method includes: sending an original command to a natural language model via the projection device, the projection device having a communication interface and connecting with the at least one peripheral device via the communication interface; generating at least one standard command based on the original command via the natural language model in response to the original command corresponding to an operation of the at least one peripheral device; and controlling, via the projection device, the at least one peripheral device to perform an operation corresponding to the original command based on the at least one standard command.

[0008] In order to achieve some or all of the above objects or other objects, the present invention provides an electronic system, including at least one peripheral device and a projection device for receiving an original command, the projection device including a communication interface connecting with the at least one peripheral device and a processor coupled to the communication interface, the processor being configured to receive at least one standard command corresponding to the original command via the communication interface in response to the original command corresponding to an operation of the at least one peripheral device, and to control the at least one peripheral device to perform the operation corresponding to the original command based on the at least one standard command. Effect of the Invention

[0009] Based on the above, the electronic system and the control method thereof of the present invention can recognize an original command of a user to know an operation of a peripheral device corresponding to the original command, generate a corresponding standard command, and control the peripheral device to perform the corresponding operation according to the standard command via a projection device. Based on this, the projection device can be configured to control the peripheral device connected to the projection device. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic diagram of an electronic system according to a first embodiment of the present invention; [Diagram 2] FIG. 4 is a schematic diagram of an electronic system according to a second embodiment of the present invention. [Diagram 3] FIG. 5 is a flow diagram of a method for controlling an electronic system according to a second embodiment of the present invention. [Figure 4] FIG. 4 is a schematic diagram of an electronic system according to a third embodiment of the present invention. [Diagram 5] FIG. 6 is a flow diagram of a method for controlling an electronic system according to a third embodiment of the present invention. [Figure 6] FIG. 2 is a schematic diagram of a peripheral device definition flow according to one embodiment of the present invention. [Figure 7] FIG. 2 is a schematic diagram of a voice control flow according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The above and other technical contents, features, and effects of the present invention will be clearly shown through the detailed description of the preferred embodiments accompanied by the attached drawings below. The directional terms, such as up, down, left, right, front, or rear, mentioned in the following embodiments are merely references to the directions in the attached drawings. Therefore, the directional terms used are intended to be explanatory, and do not limit the present invention.

[0012] Fig. 1 is a schematic diagram of an electronic system according to a first embodiment of the present invention. Referring to Fig. 1, the electronic system includes a peripheral device P and a projection device 100. The projection device 100 includes a processor 110, a communication interface 120, and a storage 130. The processor 110 is coupled to the communication interface 120 and the storage 130. The projection device 100 is, for example, a projector.

[0013] In this embodiment, the number of processors 110 may be one or more. The processor 110 may be, for example, a Central Processing Unit (CPU), a Physics Processing Unit (PPU), a programmable microprocessor, an embedded control chip, a Digital Signal Processor (DSP), an Application Specific Integrated Circuits (ASIC), or other similar devices.

[0014] The communication interface 120 includes at least one connector to connect to at least one peripheral device. Although FIG. 1 illustrates one peripheral device P, the number of peripheral devices P may be one or more. The communication interface 120 may include at least one of a Bluetooth (BT (registered trademark)) connector, a High Definition Multimedia Interface (HDMI (registered trademark)) port, a Local Area Network (LAN) connection port, a network card connector, an audio source input / output port, and a Universal Serial Bus (USB) port. The peripheral device P is, for example, an electronic device that connects via Bluetooth, an electronic device that connects via HDMI, an electronic device that connects via Wi-Fi, or an electronic device that connects via USB.

[0015] The electronic device (peripheral device P) connected via Bluetooth adopts a Bluetooth communication protocol, and the electronic device is, for example, a Bluetooth microphone, a Bluetooth speaker, a Bluetooth speaker box, a Bluetooth amplifier, a Bluetooth earphone, etc. The electronic device (peripheral device P) connected via HDMI is, for example, a set-top box, a multimedia player, a computer, a smartphone, a game console, audio, an amplifier, a television, etc., equipped with an HDMI. The electronic device (peripheral device P) connected via Wi-Fi is, for example, an electronic device that connects to the projection device 100 using a Wi-Fi communication protocol, or an electronic device equipped with a Wi-Fi function, for example, an electronic device equipped with a wireless network card, a router, or a switch. The electronic device (peripheral device P) connected via USB is, for example, a set-top box, a multimedia player, a computer, a smartphone, a game console, audio, an amplifier, etc., equipped with a USB port. The peripheral device P may be other home appliances (for example, air conditioners, lights, etc.) that the projection device 100 connects to via a local area network, Bluetooth, etc., and is controlled by making the projection device 100 a central control device of the smart home IoT.

[0016] The peripheral device P of this embodiment may further include an input device, which may be, for example, a sound pickup device (microphone), a keyboard, a touch panel, a mouse, or a remote controller, and is used to receive an original command from a user. The input device is coupled to the communication interface 120, which is configured to receive an original command from the input device. For example, the communication interface 120 is connected to a sound pickup device, and the original command (voice command) received by the sound pickup device is transmitted to the processor 110 via the communication interface 120. For example, the communication interface 120 is connected to a keyboard, and the original command (text command) received by the keyboard is transmitted to the processor 110 via the communication interface 120. For example, the communication interface is connected to an electronic device connected to Wi-Fi, and the original command (voice command or text command) received by the electronic device is transmitted to the processor 110 via the communication interface 120.

[0017] The storage 130 may be realized through adopting any fixed or removable Random Access Memory (RAM), Read-Only Memory (ROM), Flash memory, hard disk, or other similar devices, or a combination of these devices. The storage 130 may include one or more program code snippets, which may be installed and then executed by the processor 110 to realize the method of controlling the electronic system described below.

[0018] In this embodiment, in response to the original command corresponding to an operation of the peripheral device P, the processor 110 receives at least one standard command corresponding to the original command via the communication interface 120, and the processor 110 controls the peripheral device P to perform the operation corresponding to the original command based on the at least one standard command.

[0019] In one embodiment, the electronic system may further include an environmental sensing device, such as a light sensor or a decibel meter. The environmental sensing device is configured to sense changes in the surrounding environment, and the projection device 100 may receive information input via the environmental sensing device, which may serve as reference information for the electronic system to control the projection device 100 and / or the peripheral device P. The light sensor is configured to sense, for example, the brightness of the environment, and the decibel meter is configured to sense, for example, the sound volume of the environment. The electronic system may assist in adjusting the brightness, color gamut, sound volume, etc. of the projection device 100 and / or the peripheral device P via the environmental sensing device.

[0020] 2 is a schematic diagram of an electronic system according to a second embodiment of the present invention. Referring to FIG. 2, the electronic system of the second embodiment includes a projection device 100, at least one peripheral device P_1 to P_N, and a natural language model 200. The projection device 100 of the first embodiment can be used in the second embodiment. The projection device 100 may be communicatively connected to the natural language model 200 wirelessly or wired via a communication interface 120.

[0021] In this embodiment, the natural language model 200 is, for example, a chatbot equipped with a machine learning algorithm. The chatbot may be, for example, any pre-trained chatbot, such as Chat Generative Pre-trained Transformer (ChatGPT), Microsoft Bing, Google Bard, or ERNIE Bot, or a dedicated chatbot trained with data in a specific domain. The natural language model 200 may be a separately trained language model. The natural language model 200 may be configured to perform natural language processing (NLP), natural language processing and understanding, dialogue management, speech-to-text, text-to-speech, and the like. The natural language model 200 can identify multiple languages ​​and many accents. The natural language model 200 may be provided on a cloud server. In one embodiment, the natural language model 200 may be provided within the projection device 100 and coupled to the processor 110 of the projection device 100 .

[0022] 3 is a flow diagram of a method for controlling an electronic system according to a second embodiment of the present invention. Referring to FIG. 3, in step S305, an original command is sent to the natural language model 200 via the projection device 100. Then, in step S310, in response to the original command corresponding to at least one operation of the peripheral devices P_1 to P_N, at least one standard command is generated based on the original command via the natural language model 200. In step S315, at least one of the peripheral devices P_1 to P_N is controlled to perform an operation corresponding to the original command based on the at least one standard command via the projection device 100. For example, for the control of the peripheral device P_1, the projection device 100 transmits a corresponding control signal to the control circuit (processor) of the peripheral device P_1 via the processor 110, and the control circuit (processor) of the peripheral device P_1 performs the corresponding operation.

[0023] In one embodiment, the original command may control the projection device 100 and the peripheral devices P_1 to P_N (simultaneously). In response to the original command corresponding to the operation of at least one of the projection device 100 and the peripheral devices P_1 to P_N (simultaneously), at least one standard command is generated based on the original command via the natural language model 200. Through the projection device 100, the projection device 100 and at least one of the peripheral devices P_1 to P_N are controlled to perform an operation corresponding to the original command based on the at least one standard command. The original command issued by the user may control the operation of only at least one of the peripheral devices P_1 to P_N, or may control the operation of at least one of the projection device 100 and the peripheral devices P_1 to P_N (simultaneously).

[0024] In one embodiment, the natural language model 200 may generate at least one standard command based on the original command, the projector rule command, and at least one peripheral device rule command. The projector rule command is used to make the standard command generated by the natural language model 200 conform to an operation executable by the projection device 100, and the peripheral device rule command is used to make the standard command generated by the natural language model 200 conform to an operation executable by at least one of the peripheral devices P_1 to P_N. In this embodiment, the projector rule command and the peripheral device rule command may be stored in the storage 130 of the projection device 100. The peripheral device rule command includes at least one of the product type of the peripheral device to which it corresponds and the description (e.g., the connection method) of the communication interface 120 to which the corresponding peripheral device is connected.

[0025] The processor 110 of the projection device 100 may convert at least one (received) standard command into a control code that the projection device 100 can recognize and execute. Alternatively, the processor 110 may convert at least one (received) standard command into a control code that the projection device 100 can recognize and execute via the natural language model 200 or a cloud server.

[0026] FIG. 4 is a schematic diagram of an electronic system according to a third embodiment of the present invention. Referring to FIG. 4, the electronic system of the third embodiment includes a projection device 100, peripheral devices P_1 to P_N, a natural language model 200, and a cloud server 400. The projection device 100 of the first embodiment and the natural language model 200 of the second embodiment can be used in the third embodiment. The projection device 100, the natural language model 200, and the cloud server 400 may communicate with each other via a wired and / or wireless communication method. The wired communication method is, for example, a cable. The wireless communication method includes, for example, Wi-Fi, Bluetooth, and the like. For example, the projection device 100, the natural language model 200, and the cloud server 400 may be connected with each other via a network transmission. The natural language model 200 may be provided in the cloud server 400 and / or the third-party cloud server, and the cloud server 400 and / or the third-party cloud server may include a processor (or multiple processors), a communication device, and a storage device. The storage device of the cloud server 400 and / or the third-party cloud server is configured to store, for example, a chatbot (the natural language model 200) with a machine learning algorithm, and the processor of the cloud server 400 and / or the third-party cloud server is used to execute, for example, the above-mentioned algorithm. The communication device of the cloud server 400 and / or the third-party cloud server is used to communicate with the projection device 100 and the natural language model 200.

[0027] 5 is a flow diagram of a method for controlling an electronic system according to a third embodiment of the present invention. Referring to FIG. 5, in step S505, a device description of at least one of the peripheral devices P_1 to P_N connected to the communication interface 120 via the projection device 100 is obtained, and the device description is transmitted to the cloud server 400.

[0028] The device description of the peripherals P_1 to P_N may be a device name, an on-screen display (OSD) name, a generic framework name, etc. The device description may have a brand or a feature of the device. The device description of the electronic device (peripheral devices P_1 to P_N) connected via HDMI may be, for example, "XXXBD", "AndroidTV", "TVDongle", "AppleTV", or "GameStation". The device description of the electronic device connected via Bluetooth may be, for example, a character string including a device model or brand, such as "BEX1850MH" (Bluetooth earphone) or "MITVS26" (Bluetooth audio). In one embodiment, the device description may be displayed on a display interface (for example, On-screen display (OSD)) of the projection device 100 to inform the user of the peripheral devices P_1 to P_N currently connected to the projection device 100.

[0029] When the peripheral devices P_1 to P_N are connected to the projection device 100 via a wired or wireless connection, the peripheral devices P_1 to P_N may provide the device descriptions of the peripheral devices P_1 to P_N to the projection device 100 via a connection signal. The projection device 100 may read the device descriptions of the peripheral devices P_1 to P_N at a predetermined time (for example, when the projection device 100 is started up, when a new peripheral device is detected, at regular fixed intervals, etc.) and provide the device descriptions to the cloud server 400. One peripheral device has one corresponding device description.

[0030] After the projection device 100 transmits the device description to the cloud server 400, in step S510, the cloud server 400 transmits the device description and a projector rule command to the natural language model 200 in response to receiving the device description from the projection device 100. The projector rule command is, for example, stored in the cloud server 400. The projector rule command is configured to restrict the natural language model 200 to generate a peripheral device rule command for the device description after receiving the device description. After receiving the device description and the projector rule command, the natural language model 200 generates a peripheral device rule command based on the device description and the projector rule command. In step S515, the natural language model 200 transmits the peripheral device rule command to the cloud server 400.

[0031] In order to allow the electronic system to control the peripheral devices P_1 to P_N, it is necessary to make the natural language model 200 generate at least one standard command corresponding to an operation that at least one of the peripheral devices P_1 to P_N can execute based on a peripheral device rule command of at least one of the peripheral devices P_1 to P_N. The peripheral devices P_1 to P_N connected to the projection device 100 may change at any time, for example, when a user plugs in and unplugs a different HDMI-connected electronic device as needed, or connects to a different electronic device via Bluetooth or Wi-Fi. Therefore, it is necessary to transmit the device description of the peripheral devices P_1 to P_N currently connected to the projection device 100 to the natural language model 200 every time an original command is transmitted. Based on this, the natural language model 200 can provide a standard command used to execute an operation of the peripheral devices P_1 to P_N connected to the projection device 100, and it is possible to avoid providing an invalid command that the natural language model 200 cannot execute.

[0032] When the natural language model 200 recognizes the device description, the device description may be a product code, a product number, or a serial number of a peripheral device. The projector rule command may be configured to request the natural language model 200 to search for peripheral devices P_1-P_N corresponding to the device description through its internal database or a network, and obtain software configurations, hardware configurations, and executable operations of the corresponding peripheral devices P_1-P_N. The natural language model 200 may generate a peripheral device rule command based on the obtained software configurations, hardware configurations, and executable operations. For example, the device description is "XX_PS5", and the natural language model 200 finds through a search that the corresponding peripheral device P_1 is a "game console", and associates the peripheral device P_1 with the related operations of the "game console" and the connection port to which the peripheral device P_1 is connected. Therefore, even if the original command does not clearly request which peripheral device is connected to the projection device 100 via which connection method, if the original command is related to a "game console", when the natural language model 200 converts the original command into a standard command, the standard command becomes a pre-defined command that controls the peripheral device P_1 connected to the connection port associated with the device description "XX_PS5".

[0033] In one embodiment, a reservation format of a peripheral rule command is provided in a projector rule command, and when the natural language model 200 recognizes a device description, the natural language model 200 fills in the product features corresponding to the device description into the variable portion of the reservation format to form a peripheral rule command.

[0034] In one embodiment, after the natural language model 200 sends the peripheral rule command to the cloud server 400, the cloud server 400 may store the peripheral rule command in association with the corresponding projection device 100, or the cloud server 400 may store the peripheral rule command in association with the corresponding user account of the projection device 100. In another embodiment, the cloud server 400 may also associate information such as a product serial number, an Internet Protocol (IP) address, and a Media Access Control (MAC) address with the peripheral rule command. The information is used to indicate the origin of the device description. Based on this, when the cloud server 400 later receives an original command issued by the same projection device 100, the cloud server 400 sends the original command, the projector rule command, and the peripheral rule command associated with the projection device 100 to the natural language model 200.

[0035] In one embodiment, the device description may be recognized by the natural language model 200 as a fixed-format description. The fixed format refers to a description that is common in the industry and is used across multiple products. For example, the natural language model 200 determines the type of each of the peripheral devices P_1-P_N and the operations that each of the peripheral devices can perform based on the device description, and generates a peripheral device rule command for each of the peripheral devices P_1-P_N based on the determination result. One peripheral device corresponds to one peripheral device rule command.

[0036] In one embodiment, the instructions used to generate peripheral rule instructions by having the natural language model 200 recognize the device description may not be included in the projector rule instructions, but may be implemented via other rule instructions.

[0037] After the user inputs an original command into the projection device 100, in step S520, the projection device 100 sends the original command to the cloud server 400. In step S525, in response to receiving the original command from the projection device 100, the cloud server 400 sends the original command, the projector rule command, and the peripheral device rule command to the natural language model 200 to allow the natural language model 200 to analyze the original command, and generate a standard command that corresponds to the original command and is decipherable by the projection device 100 based on the original command, the projector rule command, and the peripheral device rule command.

[0038] In this embodiment, the natural language model 200 may generate different standard commands based on the type of the original command. The types of the original command include a single-item control command, a multi-item control command, a composite control command, etc. The single-item control command is a command corresponding to the original command that only requires adjustment of one parameter of the projection device 100 or the peripheral devices P_1 to P_N, or that only requires one operation command. For example, the original command is a single-item control command that only requires turning down the volume of the peripheral devices P_1 to P_N, or that only requires switching the connected peripheral devices P_1 to P_N. The natural language model 200 may generate a standard command corresponding to a single-item operation. The multi-item control command is a command corresponding to the original command that only requires multiple single-item operations. For example, the original command is a multi-item control command that only requires turning down the volume of the peripheral devices P_1 to P_N and switching the connected peripheral devices P_1 to P_N, etc. The natural language model 200 may generate a standard command corresponding to multiple single-item operations. The composite control command is a command whose corresponding requirement includes adjusting multiple parameters of the projection device 100 and / or the peripheral devices P_1-P_N. The original command may be a composite control command that requires adjusting the visual effect of the projection screen, for example, the original command is "Please improve the color of the screen." For example, the natural language model 200 generates multiple standard commands to adjust multiple parameters such as color gamut, brightness, and sharpness of the projection screen.

[0039] The projector rule command is further used to restrict the natural language model 200 to output only commands that the projection device 100 can eventually understand or convert into control codes of the projection device 100. For example, the projector rule command may restrict the natural language model 200 to output only standard commands that the projection device 100 can execute, such as "start up", "shut down", "increase volume", "decrease volume", "connect HDMI1", "connect HDMI2", "audio on", "switch HDMI source", etc., or codes thereof, and require the natural language model 200 to reduce the original command received to any one of the above standard commands or to other standard commands after analyzing the semantics of the original command.

[0040] The projector rule directives and peripheral rule directives may be, for example, a set of tens, hundreds, or thousands of characters. The projector rule directives added each time the cloud server 400 sends the original directive to the natural language model 200 may be the same. The cloud server 400 may optimize the projector rule directives from time to time, for example, by increasing or decreasing the number of conditions in the projector rule directives to achieve faster feedback speed or more accurate feedback.

[0041] In step S530, the natural language model 200 transmits the standard command to the cloud server 400. The cloud server 400 converts the standard command into a corresponding control code. Then, in step S535, the cloud server 400 transmits the control code to the projection device 100, and controls at least one of the peripheral devices P_1 to P_N by making the projection device 100 execute the control code, or the projection device 100 simultaneously controls the operation of at least one of the peripheral devices P_1 to P_N and the projection device 100 based on the control code. Also, as shown in step S535', the natural language model 200 may transmit the standard command to the projection device 100, and convert the standard command into a corresponding control code via the projection device 100 itself, thereby executing the corresponding operation.

[0042] In one embodiment, the processor 110 of the projection device 100 may display a control interface on the display interface of the projection device 100, and a user may input an original command into the control interface. The original command may be a text command or a voice command. For example, the control interface may include a selection item capable of activating a recording function of a sound pickup device (e.g., a microphone) of the projection device 100, and after the sound pickup device is activated, the control interface may receive a voice signal through the sound pickup device. After receiving the voice signal, the projection device 100 converts the voice signal into a text-based original command through the processor 110. Alternatively, the control interface may include a selection item capable of activating manual input (handwriting pad, touch pad, or keyboard input) to receive a text-based original command input by a user.

[0043] In another embodiment, the voice signal may be converted into the original command in text format via a terminal device (e.g., a smartphone) that is paired with the projection device 100. For example, after receiving the voice signal via the projection device 100, the voice signal is transmitted to the paired terminal device via the projection device 100, and then the voice signal is converted into the original command in text format via the terminal device, and the original command is transmitted to the projection device 100. Alternatively, the voice signal may be received using a sound pickup device (e.g., a microphone) of the terminal device, and the terminal device may convert the voice signal into the original command in text format, and then transmit the original command to the projection device 100, and upload the original command to the cloud server 400 via the projection device 100. Alternatively, the terminal device and the cloud server 400 may be coupled, and the terminal device may directly upload the original command to the cloud server 400.

[0044] In another embodiment, the projection device 100 or the terminal device may receive the voice signal and then transmit the voice signal to the cloud server 400, so as to allow the cloud server 400 to convert the voice signal into an original command in text format. After the cloud server 400 receives the voice signal, the voice signal in natural language format may be converted into an original command in text format through a voice recognition model of the cloud server 400.

[0045] In another embodiment, the original command may be a voice signal in a natural language form input by a user, which voice signal can be directly uploaded to the natural language model 200, through which the original command in voice form is recognized.

[0046] 6 is a schematic diagram of a peripheral device definition flow according to an embodiment of the present invention. Referring to FIG. 6, in this embodiment, the communication interface 120 includes four connectors, for example, a BT connector 610, an HDMI connector 620, a Wi-Fi connector 630, and a USB connector 640. After the projection device 100 is started, the processor 110 may execute a monitoring program in the projection device 100 to scan the BT connector 610, the HDMI connector 620, the Wi-Fi connector 630, and the USB connector 640, and obtain the device description of the peripheral devices P_1 to P_N through the connection signals of each connection method. The monitoring program is stored in the storage 130.

[0047] For the BT connector 610, the processor 110 of the projection device 100 may obtain the device description via the name when issuing a connection request by the Bluetooth communication protocol. For the HDMI connector 620, the processor 110 of the projection device 100 may obtain the device description via the Consumer Electronics Control (CEC) protocol or the Monitor Control Command Set (MCCS) protocol. For the Wi-Fi connector 630, the processor 110 may obtain the device description when issuing a connection request via the Wi-Fi protocol. For the USB connector 640, the processor 110 may obtain the device description when issuing a connection request via the USB protocol, and each of the peripheral devices P_1 to P_N has an associated description text in a handshake packet that establishes a link with the projection device 100, and the processor 110 may obtain the device description by incorporating these description texts.

[0048] After acquiring the device description, the projection device 100 uploads the device description to the cloud server 400. At the same time, the monitoring program of the projection device 100 may detect in real time whether there is a change in the connection of the BT connector 610, the HDMI connector 620, the Wi-Fi connector 630, and the USB connector 640. If there is a change, the projection device 100 acquires an updated device description again and uploads the device description to the cloud server 400. If the monitoring program detects that one of the peripheral devices has been disconnected, the projection device 100 may delete the device description in the cloud server 400 corresponding to the disconnected peripheral device by sending an update command to the cloud server 400.

[0049] After the cloud server 400 receives the device description, the device description and the projector rule command are sent to the natural language model 200. The device description allows the natural language model 200 to identify the product types of the peripheral devices P_1 to P_N and generate corresponding peripheral device rule commands, which are sent to the cloud server 400 and stored.

[0050] The peripheral device rule command stored in the cloud server 400 may be overwritten when a new peripheral device rule command is received. For example, if there is a change in the peripheral devices P_1 to P_N, the projection device 100 may again transmit the device description to the cloud server 400, and the cloud server 400 again transmits the device description and the projector rule command to the natural language model 200, and the natural language model 200 generates a new peripheral device rule command and transmits it to the cloud server 400, thereby overwriting the old peripheral device rule command.

[0051] The natural language model 200 may generate a single peripheral device rule command or a combination of multiple peripheral device rule commands. The combination of multiple peripheral device rule commands generates, for example, a peripheral device rule command for each peripheral device (i.e., each device description). Therefore, when multiple peripheral devices P_1 to P_N are connected to the projection device 100, multiple peripheral device rule commands may be generated. The natural language model 200 may simultaneously generate standard commands for the projection device 100 and the peripheral devices P_1 to P_N based on the projector rule command and the peripheral device rule command based on the intention of the original command. Therefore, it is possible to achieve a control result across devices via one original command.

[0052] For example, the projection device 100 connects to a game console of brand A via the HDMI connector 620, and connects to a sound device of brand B via the BT connector 610. The processor 110 of the projection device 100 may obtain a device description “A_GameStation” of the game console via the CEC protocol, and obtain a device description “B_SoundBar” of the sound device via the Bluetooth communication protocol, and the processor 110 may send the device descriptions “A_GameStation” and “B_SoundBar” to the cloud server 400 via the communication interface 120, and the device description and projector rule command are sent to the natural language model 200 via the cloud server 400.

[0053] The natural language model 200 recognizes the device description "A_GameStation", acquires information such as the product type, type of connection port, and executable operations (e.g., standby, start, mode switching) of the "A_GameStation" from the network, and generates a peripheral device rule command (hereinafter simply referred to as "peripheral rule A") corresponding to the "A_GameStation". The peripheral rule A is used to restrict the natural language model 200 to generate standard commands only for operations that can be performed by a game console of brand A. The natural language model 200 recognizes the device description "B_SoundBar", and its internal database contains information such as the product type and executable operations corresponding to the "B_SoundBar", and generates a peripheral device rule command (hereinafter simply referred to as "peripheral rule B") corresponding to the "B_SoundBar". The peripheral rule B is used to restrict the natural language model 200 to generate standard commands only for operations that can be performed by a sound device of brand B.

[0054] Then, the natural language model 200 sends the peripheral rule A and the peripheral rule B to the cloud server 400. The cloud server 400 is configured to store a user account, an IP address, or a product serial number corresponding to the projection device 100. When the cloud server 400 receives the original command, the cloud server 400 sends the original command, the already stored projector rule command, and the peripheral rule A and the peripheral rule B to the natural language model 200. The natural language model 200 generates at least one standard command based on the requirements of the projection device 100, the game console of brand A, and the audio device of brand B in the original command.

[0055] 7 is a schematic diagram of a voice control flow according to an embodiment of the present invention. Before executing the voice control flow, at least one peripheral definition flow may be executed to make the natural language model 200 generate peripheral rule instructions, so that the subsequent flow for controlling the peripheral can be realized.

[0056] Referring to FIG. 7, in step S701, the projection device 100 receives a voice signal. In step S703, the projection device 100 converts the voice signal into an original command in text form, or the voice signal is directly used as the original command. In step S705, the projection device 100 sends the original command to the cloud server 400. In step S707, the cloud server 400 sends the original command, the projector rule command, and the peripheral device rule command to the natural language model 200. In step S709, the natural language model 200 generates a standard command for an operation corresponding to the original command based on the original command, the projector rule command, and the peripheral device rule command.

[0057] Suppose the original command is "I want to play a game", the natural language model 200 can analyze several points as follows: (1) The input signal source of the projection device 100 needs to be switched to one of the peripheral devices P_1 to P_N currently connected to the HDMI connector 620 (e.g., a game console); (2) One of the peripheral devices P_1 to P_N currently connected to the BT connector 610 (e.g., the audio device 800) needs to be activated; (3) The projection mode of the projection device 100 should be changed to "game mode". Therefore, based on the projector rule command and the peripheral device rule command, the natural language model 200 generates standard commands such as a first standard command to switch the signal input source to the HDMI source, a second standard command to activate the audio device 800 and adjust the volume or channel to a value corresponding to the game mode, and a third standard command to switch the projection mode of the projection device 100 to the game mode.

[0058] In step S711, the natural language model 200 transmits the standard command to the cloud server 400. In step S713, the standard command is converted into a control code executable by the projection device 100 via the cloud server 400, and in step S715, the control code is transmitted to the projection device 100.

[0059] In step S717, the projection device 100 switches the HDMI source to the game console based on the first standard command. For example, assuming that the HDMI switch includes multiple HDMI input ports (respectively connected to multiple peripheral devices P_1 to P_N) and one HDMI output port (the output media signal may be provided to the display interface of the projection device 100), the projection device 100 transmits a switch signal to the HDMI switch based on the first standard command via the processor 110, and causes the control circuit of the HDMI switch to select the HDMI input port to be connected by the switch of the switch, thereby receiving the multimedia signal provided by the corresponding one of the peripheral devices P_1 to P_N from the connected HDMI input port. The first standard command corresponds to an operation for controlling the projection device 100, and the first standard command is, for example, converted based on the original command and the projector rule command. In another embodiment, if the original command includes one specific game among the multiple games in the game console, the natural language model 200 may also generate a standard command corresponding to the adjustment to the specific game.

[0060] In step S719, the projection device 100 activates the audio device 800 based on the second standard command, and drives the audio device 800 to adjust the volume. For example, the projection device 100 transmits an activation signal to the audio device 800 based on the second standard command via the processor 110, thereby waking up or activating the audio device 800. Then, the processor 110 transmits a corresponding control signal to the control circuit of the audio device 800 based on the second standard command, and adjusts the volume or channel to a value corresponding to the game mode via the control circuit. The second standard command corresponds to an operation for controlling one of the peripheral devices P_1 to P_N (for example, the audio device 800), and the second standard command is, for example, converted based on at least an original command and a peripheral device rule command.

[0061] In step S721, the projection device 100 switches its projection mode to a game mode based on the third standard command, thereby applying parameter settings (e.g., brightness, color, etc.) of the game mode. For example, the projection device 100 selects a game mode from a plurality of projection modes based on the third standard command via the processor 110, reads out parameter settings of the game mode, and then adjusts settings of corresponding components of the projection device 100, such as a light source module and a display module, based on the parameter settings. The third standard command corresponds to an operation for controlling the projection device 100, and the third standard command is, for example, converted based on an original command and a projector rule command.

[0062] To sum up, the electronic system and the control method thereof of the present invention recognize peripheral devices connected to a projection device, provide information about the peripheral devices to a natural language model, generate a standard command corresponding to the natural language model, and control the peripheral devices based on the standard command via the projection device, so that the projection device can be used as a central control console for the peripheral devices. Based on this, even if the original command issued by the user does not clearly specify the object to be controlled, the electronic system can achieve the result desired by the user by simultaneously controlling multiple peripheral devices related to the request of the original command through identification by the natural language model. In addition, the user does not need to memorize information such as complicated function operations, physical connections of multiple external peripheral devices, or specified IP addresses, system names, device codes, etc., and can control the projection device in the most convenient way. As a result, even a user who is operating a projection device for the first time or a user who is unfamiliar with operating peripheral devices can use the peripheral devices without any trouble and adjust parameter settings for the best viewing, sound, and other effects.

[0063] The above is merely a preferred embodiment of the present invention, and does not limit the scope of the present invention. All simple equivalent changes and modifications made according to the claims and the contents of the invention are still within the patent scope of the present invention. Any embodiment or claim of the present invention does not necessarily achieve all the objectives, advantages, or features disclosed in the present invention. In addition, the abstract and the title (name of the invention) are only used to assist in searching the patent document, and are not used to limit the scope of the invention. In addition, the terms "first", "second", etc. mentioned in the specification or claims are only used to name the names of elements or to distinguish different embodiments or ranges, and are not used to limit the upper or lower limits on the quantity of elements. [Industrial Applicability]

[0064] The electronic system and control method of the present invention can be applied to a projection device to control peripheral devices connected thereto. [Explanation of symbols]

[0065] 100: Projection device 110: Processor 120: Communication interface 130: Storage P, P_1 to P_N: Peripheral devices 200: Natural Language Modeling 400: Cloud server 610: BT connector 620:HDMI connector 630: Wi-Fi connector 640: USB connector 800: Sound equipment S305~S315: Step S505~S335: Step S701~S721: Step

Claims

1. 1. A method for controlling an electronic system including a projection device and at least one peripheral device, comprising: transmitting an original command to a natural language model via the projection device, the projection device having a communication interface and communicatively connected to the at least one peripheral device via the communication interface; generating, via the natural language model, at least one standard command based on the original command in response to the original command corresponding to an operation of the at least one peripheral device; restricting the at least one peripheral device to perform the operation corresponding to the original command based on the at least one standard command via the projection device; Includes Characterized in that A method for controlling an electronic system.

2. generating, via the natural language model, the at least one standard command based on the original command in response to the original command corresponding to an operation of the projection device and the at least one peripheral device; controlling, via the projection device, the projection device and the at least one peripheral device to perform the operation corresponding to the original command based on the at least one standard command; Further includes Characterized in that A method for controlling an electronic system according to claim 1.

3. generating the at least one standard command based on the original command via the natural language model, generating the at least one standard command based on the original command, a projector rule command, and at least one peripheral device rule command via the natural language model; Including, the projector rule command is used to make the at least one standard command generated by the natural language model conform to an operation executable by the projection device; The at least one peripheral rule command is used to conform the at least one standard command generated by the natural language model to an operation that the at least one peripheral can perform. Characterized in that A method for controlling an electronic system according to claim 2.

4. acquiring at least one device description of the at least one peripheral device connected to the communication interface via the projection device, and transmitting the at least one device description to a cloud server, wherein the projector rules command is stored in the cloud server; in response to receiving the at least one device description from the projection device via the cloud server, transmitting the at least one device description and the projector rules directive to the natural language model; generating, via the natural language model, at least one peripheral rule command based on the at least one device description and the projector rule command, and sending the at least one peripheral rule command to the cloud server; Further comprising: generating the at least one standard command based on the original command via the natural language model, In response to receiving the original command from the projection device via the cloud server, transmitting the original command, the projector rule command, and the at least one peripheral rule command to the natural language model, and causing the natural language model to generate the at least one standard command. Includes Characterized in that A method for controlling an electronic system according to claim 3.

5. In response to the projection device detecting a change in the at least one peripheral device connected to the communication interface, acquiring at least one updated device description via the projection device and transmitting the updated at least one device description to the cloud server, thereby causing the natural language model to generate the at least one corresponding peripheral device rule command based on the updated at least one device description. Further includes Characterized in that A method for controlling an electronic system according to claim 4.

6. after the step of transmitting the at least one peripheral rule command to the cloud server via the natural language model; storing the at least one peripheral rule command associated with the projection device via the cloud server; Or, storing the at least one peripheral rule command and an account of a user of the projection device via the cloud server after associating the at least one peripheral rule command with the account of a user of the projection device; Further includes Characterized in that A method for controlling an electronic system according to claim 4.

7. The at least one peripheral rule directive includes at least one of a product type of the at least one peripheral and a description of the communication interface to which the at least one peripheral connects. Characterized in that A method for controlling an electronic system according to claim 3.

8. responsive to receiving an audio signal, converting the audio signal via the projection device into the original command in text form; Or, In response to receiving the audio signal, converting the audio signal into the original command in the text format via a terminal device paired with the projection device; Or, In response to receiving the audio signal, transmitting the audio signal to the cloud server via the projection device or the terminal device, and causing the cloud server to convert the audio signal into the original command in the text format. Further includes Characterized in that A method for controlling an electronic system according to claim 1.

9. In response to receiving a voice signal, transmitting the voice signal via the projection device to the natural language model, thereby causing the voice signal to be the original command. Further includes Characterized in that A method for controlling an electronic system according to claim 1.

10. converting, via the projection device, the at least one standard command into a corresponding at least one control code, and controlling the at least one peripheral device by executing the at least one control code, the at least one control code corresponding to the operation required by the original command; Or, converting the at least one standard command into the at least one corresponding control code via the cloud server, transmitting the at least one control code to the projection device, and controlling the at least one peripheral device via causing the projection device to execute the at least one control code. Further includes Characterized in that A method for controlling an electronic system according to claim 1.

11. 1. An electronic system comprising: At least one peripheral device; Projection equipment and Including, the projection device is used to receive an original command, and the projection device includes a communication interface and a processor; the communication interface is communicatively coupled to the at least one peripheral device; The processor is coupled to the communication interface, and the processor further comprises: receiving, via the communication interface, at least one standard command corresponding to the at least one original command in response to the original command corresponding to the operation of the at least one peripheral device; Controlling the at least one peripheral device to perform the operation corresponding to the original command based on the at least one standard command. It is configured as Characterized in that Electronic system.

12. The processor, receiving, via the communication interface, the at least one standard command corresponding to the original command, in response to the original command corresponding to the operation of the projection device and the at least one peripheral device; Controlling the projection device and the at least one peripheral device to perform the operation corresponding to the original command based on the at least one standard command. It is configured as Characterized in that 12. The electronic system of claim 11.

13. further comprising a natural language model; the projection device connects to the natural language model via the communication interface, the natural language model being used to generate at least one standard command based on the original command, a projector rule command, and at least one peripheral device rule command; the projector rule command is used to make the at least one standard command generated by the natural language model conform to an operation executable by the projection device; The at least one peripheral rule command is used to conform the at least one standard command generated by the natural language model to an operation that the at least one peripheral can perform. Characterized in that 13. The electronic system of claim 12.

14. Further comprising a cloud server; the projection device connects to the cloud server via the communication interface; the processor is configured to obtain, via the communication interface, at least one device description of the at least one peripheral device connected to the communication interface; the cloud server stores the projector rules command, and the cloud server is configured to, in response to receiving the at least one device description from the projection device, send the at least one device description and the projector rules command to the natural language model; The natural language model is configured to, in response to receiving the at least one device description and the projector rule command, generate the at least one peripheral rule command based on the at least one device description and the projector rule command, and transmit the at least one peripheral rule command to the cloud server. Characterized in that 14. The electronic system of claim 13.

15. the cloud server is configured to, in response to receiving the original command from the projection device, send the original command, the projector rule command, and the at least one peripheral rule command to the natural language model; The natural language model is configured to, in response to receiving the original command, the projector rule command, and the at least one peripheral rule command, generate the at least one standard command based on the original command, the projector rule command, and the at least one peripheral rule command. Characterized in that 15. The electronic system of claim 14.

16. The processor is configured to, in response to detecting a change in the at least one peripheral device connected to the communication interface, obtain at least one updated device description via the communication interface and transmit the updated at least one device description to the cloud server, thereby causing the natural language model to generate the at least one peripheral device rule command corresponding to the at least one updated device description. Characterized in that 15. The electronic system of claim 14.

17. The cloud server includes: storing said at least one peripheral rule command in association with said projection device; Or, storing said at least one peripheral rule command in association with a user account of said projection device; It is configured as Characterized in that 15. The electronic system of claim 14.

18. The at least one peripheral rule directive includes at least one of a product type of the at least one peripheral and a description of the communication interface to which the at least one peripheral is connected. Characterized in that 14. The electronic system of claim 13.

19. The processor is configured, in response to receiving a voice signal, to transmit the voice signal via the communication interface to the natural language model, thereby rendering the voice signal the original command. Characterized in that 14. The electronic system of claim 13.

20. The processor, responsive to receiving a voice signal, converting the voice signal into text form of the original command; Or, In response to receiving the voice signal, transmitting the voice signal to the cloud server, and causing the cloud server to convert the voice signal into the original command in text form. It is configured as follows: Or, Further comprising a terminal device, The terminal device is coupled to the projection device, and the terminal device responsive to receiving the voice signal, converting the voice signal into the original command in text form; Or, In response to receiving the voice signal, transmitting the voice signal to the cloud server, and causing the cloud server to convert the voice signal into the original command in text form. It is configured as Characterized in that 12. The electronic system of claim 11.

21. the processor is configured to convert the at least one standard command into a corresponding at least one control code, and then control the at least one peripheral device by executing the at least one control code, the at least one control code corresponding to the operation required by the original command; Or, Further comprising a cloud server; The cloud server converts the at least one standard command into the at least one corresponding control code, transmits the at least one control code to the projection device, and causes the projection device to execute the at least one control code to control the at least one peripheral device. Characterized in that 12. The electronic system of claim 11.

Images