Device visualization control method, device and computer readable storage medium

Abstract

The present invention discloses a method, device and computer-readable storage medium for visual control of equipment. The method is applied to a user terminal, and the method comprises: obtaining a scene screen, and obtaining a device page corresponding to the scene screen, wherein the scene screen is a screen captured by a camera of a scene in which the equipment is located, and the device page is a control page corresponding to the equipment; displaying the scene screen and the device page in a split screen on the user terminal, so that the user can watch the scene screen and trigger a control instruction through the device page; when the control instruction is detected, determining a device to be controlled according to the control instruction; sending the control instruction to the device to be controlled, so that the device to be controlled executes the control instruction. The present invention can realize visual remote control of equipment and improve the accuracy of remote control of equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to a method and device for visualizing and controlling equipment, and a computer-readable storage medium. Background Art

[0002] With the rapid development of science and technology, the application of smart devices related to the Internet of Things technology is becoming more and more widespread. For example, smart homes are becoming more and more popular with the development of the Internet of Things technology. Compared with ordinary homes, smart homes have a full range of information interaction functions, so that users can remotely control smart homes through APP (Application).

[0003] However, when users remotely control smart devices, they cannot view the current status of the smart devices and the surrounding environment of the smart devices. Blind operation may cause damage to the smart devices, and users cannot accurately control the smart devices. For example, when a user remotely controls a TV to turn off, the TV may not turn off successfully due to network delays and other reasons, and the user does not think that the TV has been successfully turned off, resulting in the user being unable to accurately control the smart device, greatly reducing the user experience. Therefore, how to improve the accuracy of remote control of devices is an urgent problem to be solved. Summary of the invention

[0004] The main purpose of the present invention is to provide a device visualization control method, device and computer-readable storage medium, aiming to realize device visualization remote control to improve the accuracy of device remote control.

[0005] To achieve the above object, the present invention provides a device visualization control method, which is applied to a user terminal. The device visualization control method comprises the following steps:

[0006] Acquire a scene screen, and acquire a device page corresponding to the scene screen, wherein the scene screen is a screen captured by a camera of a scene in which the device is located, and the device page is a control page corresponding to the device;

[0007] The scene image and the device page are displayed in split screen on the user terminal, so that the user can watch the scene image and trigger the control instruction through the device page at the same time;

[0008] When the control instruction is detected, determining a device to be controlled according to the control instruction;

[0009] The control instruction is sent to the device to be controlled, so that the device to be controlled executes the control instruction.

[0010] Optionally, the step of acquiring a scene picture includes:

[0011] When the user terminal displays a scene selection page, if a triggering scene selection instruction is detected, determining a scene to be viewed based on the scene selection instruction, wherein the scene selection page includes a plurality of scenes so that the user triggers the scene selection instruction based on the scene selection page;

[0012] Determine a camera for capturing the scene to be viewed, and send a camera instruction to the camera, so that the camera captures a scene image corresponding to the scene to be viewed, and sends the scene image to the user terminal;

[0013] Receive the scene image sent by the camera.

[0014] Optionally, the step of obtaining a device page corresponding to the scene screen includes:

[0015] The step of obtaining the device page corresponding to the scene screen includes:

[0016] Determining controllable devices belonging to the scene picture;

[0017] Acquire device information of the controllable device, wherein the device information includes device name, device status and device icon options;

[0018] According to the device information, a device page corresponding to the scene screen is constructed.

[0019] Optionally, the device page includes a device menu option, and when the control instruction is detected, before the step of determining the device to be controlled according to the control instruction, the step further includes:

[0020] When a menu access instruction triggered by the device menu option is detected, determining a to-be-controlled device corresponding to the device menu option;

[0021] The device page is replaced with a control menu page corresponding to the device to be controlled, wherein the control menu page can trigger multiple control instructions, so that the user can trigger one or more of the multiple control instructions based on the scene screen and the control menu page.

[0022] Optionally, the device page includes a device icon option, and the device icon option is used for a user to trigger a device switch instruction. When the control instruction is detected, before the step of determining the device to be controlled according to the control instruction, the step further includes:

[0023] When the device switch instruction triggered by the device icon option is detected, determining the device to be controlled corresponding to the device icon option, and detecting whether the device to be controlled is turned on;

[0024] If the device to be controlled is turned on, a closing control instruction is triggered to close the device to be controlled;

[0025] If the device to be controlled is turned off, a start control instruction is triggered to turn on the device to be controlled.

[0026] Optionally, the step of sending the control instruction to the device to be controlled includes:

[0027] Obtaining a secret key for encrypting the control instruction;

[0028] Encrypting the control instruction by using the secret key to generate an instruction ciphertext;

[0029] The instruction ciphertext is sent to a control server, and the control server forwards the instruction ciphertext to the device to be controlled, so that the device to be controlled decrypts the instruction ciphertext to obtain the control instruction.

[0030] Optionally, the device visualization control method further includes:

[0031] When the scene picture is displayed, if a scene change instruction is detected, determining the scene to be viewed after the change according to the scene change instruction;

[0032] Determine a camera for capturing the scene to be viewed, and send a camera instruction to the camera, so that the camera captures a current scene image corresponding to the scene to be viewed, and sends the current scene image to the user terminal;

[0033] Receive the current scene picture sent by the camera, and replace the scene picture with the current scene picture.

[0034] Optionally, the device visualization control method further includes:

[0035] Detecting whether the scene picture completely includes the picture of the device to be controlled;

[0036] If it is detected that the scene image does not completely include the image of the device to be controlled, determine a camera rotation instruction, and determine the camera corresponding to the device to be controlled;

[0037] The camera rotation instruction is sent to the camera, so that the camera executes the camera rotation instruction.

[0038] In addition, to achieve the above-mentioned purpose, the present invention also provides a device visualization control device, which is deployed in a user terminal, and the device visualization control device includes:

[0039] A picture acquisition module is used to acquire a scene picture and acquire a device page corresponding to the scene picture, wherein the scene picture is a picture captured by a camera of a scene in which the device is located, and the device page is a control page corresponding to the device;

[0040] A screen display module, used for displaying the scene screen and the device page in split screen on the user terminal, so that the user can watch the scene screen and trigger the control instruction through the device page at the same time;

[0041] A device determination module, configured to determine a device to be controlled according to the control instruction when the control instruction is detected;

[0042] The instruction sending module is used to send the control instruction to the device to be controlled so that the device to be controlled executes the control instruction.

[0043] In addition, to achieve the above-mentioned purpose, the present invention also provides an electronic device, which includes: a memory, a processor, and a device visualization control program stored in the memory and executable on the processor, and when the device visualization control program is executed by the processor, the steps of the device visualization control method described above are implemented.

[0044] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a device visualization control program is stored, and when the device visualization control program is executed by a processor, the steps of the device visualization control method described above are implemented.

[0045] The present invention provides a device visualization control method, device, electronic device and computer-readable storage medium. The device visualization control method is applied to a user terminal. The device visualization control method obtains a scene picture and a device page corresponding to the scene picture, wherein the scene picture is a picture captured by a camera of a scene in which the device is located, and the device page is a control page corresponding to the device; the scene picture and the device page are displayed in split screen on the user terminal so that the user can watch the scene picture and trigger a control instruction through the device page at the same time; when a control instruction is detected, the device to be controlled is determined according to the control instruction; the control instruction is sent to the device to be controlled so that the device to be controlled executes the control instruction. The present invention displays the scene picture and the device page in split screen so that the user can remotely watch the device status and surrounding environment in the scene while remotely controlling the device, so as to prevent blind control of the device from causing damage to the device and improve the accuracy of remote control of the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present invention;

[0047] Figure 2 It is a flow chart of the first embodiment of the device visualization control method of the present invention;

[0048] Figure 3 This is a schematic diagram of a first page involved in an embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram of a second page involved in an embodiment of the present invention;

[0050] Figure 5 A schematic diagram of a third page involved in an embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of a fourth page involved in an embodiment of the present invention;

[0052] Figure 7 This is a schematic diagram of a fifth page involved in an embodiment of the present invention;

[0053] Figure 8 This is a schematic diagram of a sixth page involved in an embodiment of the present invention;

[0054] Fig. 9 This is a schematic diagram of the seventh page involved in the embodiment of the present invention;

[0055] Fig.10 It is a schematic diagram of functional modules of the first embodiment of the visualization control device of the present invention.

[0056] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0057] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0058] The main idea of ​​the embodiment of the present invention is that when a user remotely controls an intelligent device, the intelligent device may not successfully execute the control command triggered by the user due to network delay or other reasons, or the intelligent device can only be remotely controlled under the premise of observing the surrounding environment, for example, it is necessary to observe the operation of the machine tool in the factory to perform the corresponding control operation, it is necessary to observe the status of the patient to perform the corresponding control operation, and when there are vulnerable groups such as the elderly and children at home, it is necessary to observe the status of the personnel to perform the corresponding control operation. Therefore, the embodiment of the present invention obtains the scene picture and obtains the device page corresponding to the scene picture, wherein the scene picture is the picture collected by the camera of the scene where the device is located, and the device page is the control page corresponding to the device; the scene picture and the device page are displayed in a split screen on the user terminal so that the user can watch the scene picture and trigger the control command through the device page; when the control command is detected, the device to be controlled is determined according to the control command; the control command is sent to the device to be controlled so that the device to be controlled executes the control command. Visual remote control of the device is realized, and the accuracy of remote control of the device is improved.

[0059] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present invention.

[0060] The terminal in the embodiment of the present invention is a device visualization control device, which can be a user terminal with interactive functions such as a smart phone, a PC (personal computer), a microcomputer, a notebook computer, a server, etc.

[0061] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU (Central Processing Unit), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0062] Those skilled in the art will understand that Figure 1The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0063] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device visualization control program.

[0064] exist Figure 1 In the terminal shown, the processor 1001 can be used to call the device visualization control program stored in the memory 1005 and perform the following operations:

[0065] Acquire a scene screen, and acquire a device page corresponding to the scene screen, wherein the scene screen is a screen captured by a camera of a scene in which the device is located, and the device page is a control page corresponding to the device;

[0066] The scene image and the device page are displayed in split screen on the user terminal, so that the user can watch the scene image and trigger the control instruction through the device page at the same time;

[0067] When the control instruction is detected, determining a device to be controlled according to the control instruction;

[0068] The control instruction is sent to the device to be controlled, so that the device to be controlled executes the control instruction.

[0069] Further, the processor 1001 may be used to call the device visualization control program stored in the memory 1005, and further perform the following operations:

[0070] When the user terminal displays a scene selection page, if a triggering scene selection instruction is detected, determining a scene to be viewed based on the scene selection instruction, wherein the scene selection page includes a plurality of scenes so that the user triggers the scene selection instruction based on the scene selection page;

[0071] Determine a camera for capturing the scene to be viewed, and send a camera instruction to the camera, so that the camera captures a scene image corresponding to the scene to be viewed, and sends the scene image to the user terminal;

[0072] Receive the scene image sent by the camera.

[0073] Further, the processor 1001 may be used to call the device visualization control program stored in the memory 1005, and further perform the following operations:

[0074] Determining controllable devices belonging to the scene picture;

[0075] Acquire device information of the controllable device, wherein the device information includes device name, device status and device icon options;

[0076] According to the device information, a device page corresponding to the scene screen is constructed.

[0077] Further, the processor 1001 may be used to call the device visualization control program stored in the memory 1005, and further perform the following operations:

[0078] When a menu access instruction triggered by the device menu option is detected, determining a to-be-controlled device corresponding to the device menu option;

[0079] The device page is replaced with a control menu page corresponding to the device to be controlled, wherein the control menu page can trigger multiple control instructions, so that the user can trigger one or more of the multiple control instructions based on the scene screen and the control menu page.

[0080] Further, the processor 1001 may be used to call the device visualization control program stored in the memory 1005, and further perform the following operations:

[0081] When the device switch instruction triggered by the device icon option is detected, determining the device to be controlled corresponding to the device icon option, and detecting whether the device to be controlled is turned on;

[0082] If the device to be controlled is turned on, a closing control instruction is triggered to close the device to be controlled;

[0083] If the device to be controlled is turned off, a start control instruction is triggered to turn on the device to be controlled.

[0084] Further, the processor 1001 may be used to call the device visualization control program stored in the memory 1005, and further perform the following operations:

[0085] Obtaining a secret key for encrypting the control instruction;

[0086] Encrypting the control instruction by using the secret key to generate an instruction ciphertext;

[0087] The instruction ciphertext is sent to a control server, and the control server forwards the instruction ciphertext to the device to be controlled, so that the device to be controlled decrypts the instruction ciphertext to obtain the control instruction.

[0088] Further, the processor 1001 may be used to call the device visualization control program stored in the memory 1005, and further perform the following operations:

[0089] When the scene picture is displayed, if a scene change instruction is detected, determining the scene to be viewed after the change according to the scene change instruction;

[0090] Determine a camera for capturing the scene to be viewed, and send a camera instruction to the camera, so that the camera captures a current scene image corresponding to the scene to be viewed, and sends the current scene image to the user terminal;

[0091] Receive the current scene picture sent by the camera, and replace the scene picture with the current scene picture.

[0092] Further, the processor 1001 may be used to call the device visualization control program stored in the memory 1005, and further perform the following operations:

[0093] Detecting whether the scene picture completely includes the picture of the device to be controlled;

[0094] If it is detected that the scene image does not completely include the image of the device to be controlled, determine a camera rotation instruction, and determine the camera corresponding to the device to be controlled;

[0095] The camera rotation instruction is sent to the camera, so that the camera executes the camera rotation instruction.

[0096] Based on the above hardware structure, various embodiments of the device visualization control method of the present invention are proposed.

[0097] The invention provides a device visualization control method.

[0098] Reference Figure 2 , Figure 2 It is a flow chart of the first embodiment of the device visualization control method of the present invention.

[0099] In this embodiment, the device visualization control method is applied to a user terminal, and the device visualization control method includes:

[0100] Step S10, obtaining a scene screen, and obtaining a device page corresponding to the scene screen, wherein the scene screen is a screen captured by a camera of a scene in which the device is located, and the device page is a control page corresponding to the device;

[0101] In this embodiment, the execution subject of the device visualization control method is a user terminal, which can be a terminal device with interactive functions such as a smart phone, a PC, a microcomputer, a laptop, a server, etc. The user terminal is described by taking a smart phone as an example, and the smart phone includes software and hardware for running a server. The smart phone communicates with a smart device through the Internet, and the smart device can be a smart home, a camera, a factory smart device, a device used in a shopping mall or a hospital, etc., which has an Internet communication function, and is not limited here.

[0102] First of all, in order to realize the visual remote control of the device, it is necessary to obtain the scene screen of the smart device and the corresponding device page. Therefore, when the user enters the main page of the visual remote control through the scene selection page or other methods, the scene screen is obtained, and the device page corresponding to the scene screen is obtained. Among them, the scene screen is the picture captured by the camera of the scene where the device is located, and the device page is the control page corresponding to the device.

[0103] It should be noted that the scene screen includes the screen of the device to be controlled and the environmental screen of the scene in which the device to be controlled is located, which is used for users to view the current status of the device to be controlled and its surrounding environment and other information, such as whether the device to be controlled is turned on, the direction of the device to be controlled, the status of the device to be controlled, etc. The user can rotate the camera by sliding the scene screen. Specifically, when a camera rotation instruction triggered by the scene screen is detected, the camera corresponding to the current scene is determined, and then the camera rotation instruction is sent to the camera so that the camera executes the camera rotation instruction. Among them, the camera rotation instruction includes an upward rotation instruction, a downward rotation instruction, a left rotation instruction, and a right rotation instruction. Of course, other camera control instructions can also be triggered by the scene screen. For example, the user slides two fingers from the inside to the outside to enlarge the scene screen so that the user can see the device to be controlled more clearly. Correspondingly, the scene screen can also be reduced, which is not limited here.

[0104] In addition, it should be noted that the device page is the page corresponding to the scene screen, so that the user can control the device to be controlled based on the device page by clicking on the options or icons on the device page. Different scene screens correspond to different device pages, that is, different scenes include different numbers and types of smart devices. Specifically, taking the smart home as an example, in the study scene, there are electric lights, sockets, table lamps, televisions and other devices, so the device page displays icons or options corresponding to electric lights, sockets, table lamps, televisions and other devices. Among them, if there are multiple devices of the same type, the device can be numbered, such as socket No. 1, socket No. 2, etc. For ease of understanding, please refer to Figure 3 , Figure 3 This is a schematic diagram of a first page involved in an embodiment of the present invention.

[0105] In addition, it should be noted that the scene images are obtained through cameras, and each scene has one or more cameras. The camera is usually located above or in front of the smart device to ensure that the camera can clearly and completely capture the smart device.

[0106] In one embodiment, the scene screen includes a scene name display area. For details, refer to Figure 4 , Figure 4 : This is a schematic diagram of the second page involved in an embodiment of the present invention. In the figure, the main page 1 is the main page of the visual remote control. The main page 1 includes a scene screen 2 and a device page 3. The scene screen 2 includes a scene name display area 4. The middle position of the scene name display area 4 usually displays the current scene name, that is, name 1, and is highlighted. On both sides of name 1 are name 2 and name 3, which display the names of other scenes. When the user slides the scene screen or clicks the name in the scene name display area, the scene change instruction can be triggered to dynamically change the scene screen to the screen of other scenes. It can be understood that the corresponding device page will also be changed. Specifically, when displaying the scene screen, if a scene change instruction is detected, the scene to be viewed after the change is determined according to the scene change instruction; the camera that captures the scene to be viewed is determined, and a camera instruction is sent to the camera so that the camera captures the current scene screen corresponding to the scene to be viewed, and sends the current scene screen to the user terminal; the current scene screen sent by the camera is received, and the scene screen is replaced with the current scene screen.

[0107] In one embodiment, the scene image can be obtained through the scene selection page in the user terminal APP. Specifically, when the user terminal displays the scene selection page, if a triggering scene selection instruction is detected, the scene to be viewed is determined based on the scene selection instruction, wherein the scene selection page includes multiple scenes, so that the user can trigger the scene selection instruction based on the scene selection page; determine the camera that captures the scene to be viewed, and send a camera instruction to the camera, so that the camera captures the scene image corresponding to the scene to be viewed, and send the scene image to the user terminal; finally, receive the scene image sent by the camera. For ease of understanding, please refer to Figure 5 , Figure 5 Schematic diagram of the third page involved in the embodiment of the present invention, wherein the main page 1 is the application page of the user terminal, and the main page 1 includes a scene selection page 5, and the scene selection page 5 includes scene 1, scene 2, and scene 3. It can be understood that when connecting different smart devices, all smart devices may include multiple scenes. Therefore, the main page of a specific visual remote control can be accurately entered through the scene selection page, that is, entering Figure 4In other implementations, it can also be obtained through the device selection page in the user terminal APP, that is, by selecting a device, you can enter the scene screen corresponding to the device. The specific execution process is basically the same as the above-mentioned scene selection page, which will not be repeated here.

[0108] In one embodiment, the device page can be obtained by determining the controllable device belonging to the scene screen, and then obtaining the device information of the controllable device, wherein the device information includes the device name, device status, and device icon options; finally, the device page corresponding to the scene screen is constructed according to the device information. For ease of understanding, please refer to Figure 6 , Figure 6 This is a schematic diagram of the fourth page involved in an embodiment of the present invention, wherein the main page 1 is the main page of the visual remote control, the main page 1 includes a scene screen 2 and a device page 3, icons 1, 2, and 3 represent different devices, respectively, and the icons can be used as device icon options to enter the control menu page or perform control operations such as turning the device on and off based on the device icon options. The device name and device status are displayed below the icon for users to view relevant information about the device. In other embodiments, the device page may also include other device information or corresponding control operations, which are not limited here.

[0109] Step S20, displaying the scene image and the device page in split screen on the user terminal, so that the user can watch the scene image and trigger a control instruction through the device page at the same time;

[0110] After the scene image and its device page are obtained, the scene image and the device page are displayed in split screen on the user terminal, so that the user can watch the scene image and trigger the control command through the device page at the same time. The split screen display area is the main page of the application page of the user terminal, and the main page is the user interface on the APP, which is used by the user to view the behavior status of the smart terminal and its surrounding environment based on the application interface while controlling the smart terminal.

[0111] It should be noted that the scene screen and device page can be split up and down, or left and right. The layout size of the scene screen and device page on the entire main page can be set according to actual needs, such as 50:50, 70:30, etc. Of course, the user can also set the layout by himself. Specifically, the layout can be rearranged by sliding the dividing line of the scene screen and device page. For easier understanding of split screen, please refer to Figure 4 , where the upper part is scene screen 2 and the lower part is device page 3.

[0112] In addition, it should be noted that different smart devices have different control instruction sets, which include multiple control instructions for instructing the smart terminal to perform various operations. For example, control instructions such as turn on control instructions, turn off control instructions, rotate control instructions, and adjust device properties. In addition, the control instruction set is bound to the device page so that the user can trigger the corresponding control instruction based on the device page.

[0113] Step S30: When the control instruction is detected, a device to be controlled is determined according to the control instruction.

[0114] In this embodiment, when a control instruction is detected, the device to be controlled is determined according to the control instruction. When the control instruction is triggered, it carries the device identification of the device to be controlled. The device identification can be used to determine the specific object to which the control instruction is sent. It can be understood that the control instruction is triggered by the device icon or option on the device page, so when the control instruction is triggered, the corresponding device identification can be embedded.

[0115] In one embodiment, the device page includes a device menu option, which can be a device icon option in the device page. The user can trigger the corresponding menu access instruction by clicking it, or by long pressing it. Each menu access instruction corresponds to a device, so the control menu page entered is the control page corresponding to a specific device. The control menu page is usually overlaid on the device page, which can include multiple control operations. For ease of understanding, please refer to Figure 7 , Figure 7 This is a schematic diagram of the fifth page involved in an embodiment of the present invention, wherein the main page 1 is the main page of the visual remote control, the main page 1 includes a scene screen 2 and a control menu page 6, the control menu page 6 includes a device image area and multiple control operations, the device image area is used to display the image of the device, control operation 1, control operation 2, control operation 3, and control operation 4 are different control operations, corresponding to different control instructions, which can be an opening operation, a closing operation, an increasing property operation, a decreasing property operation, etc. In other implementations, the control menu page can also include more or less content, which is not limited here.

[0116] In one embodiment, the device page includes a device icon option, which is used by the user to click or long press to trigger the device switch instruction. Specifically, when the device switch instruction triggered by the device icon option is detected, the device to be controlled corresponding to the device icon option is determined, and it is detected whether the device to be controlled is turned on; if the device to be controlled is turned on, the shutdown control instruction is triggered to turn off the device to be controlled; if the device to be controlled is turned off, the startup control instruction is triggered to turn on the device to be controlled. In other implementations, the device switch instruction can also be triggered by other operations, which are not limited here.

[0117] Furthermore, before the above step S30, the method further includes:

[0118] Step A, when a menu access instruction triggered by the device menu option is detected, determining the device to be controlled corresponding to the device menu option;

[0119] Step B, replacing the device page with a control menu page corresponding to the device to be controlled, wherein the control menu page can trigger multiple control instructions, so that the user can trigger one or more of the multiple control instructions based on the scene screen and the control menu page.

[0120] In this embodiment, when a menu access instruction triggered by a device menu option is detected, the device to be controlled corresponding to the device menu option is determined, and then the device page is replaced with a control menu page corresponding to the device to be controlled, wherein the control menu page can trigger multiple control instructions, so that the user can trigger one or more of the multiple control instructions based on the scene screen and the control menu page. Among them, the menu access instruction is used to enter the control menu page, which is a page for all control operations of the device to be controlled. Specifically, the user can trigger the menu access instruction by long pressing the device icon in the device page. Of course, the menu access instruction can also be triggered by clicking the device icon in the device page.

[0121] It should be noted that each device to be controlled includes a control instruction set, which includes multiple control instructions for instructing the smart terminal to perform various operations. Different smart terminals correspond to different control instruction sets. In addition, the control instruction set is bound to the control menu page so that the user can trigger the corresponding control instruction based on the control menu page.

[0122] Furthermore, before the above step S30, the method further includes:

[0123] Step C, when the device switch instruction triggered by the device icon option is detected, determining the device to be controlled corresponding to the device icon option, and detecting whether the device to be controlled is turned on;

[0124] Step D, if the device to be controlled is turned on, triggering a shutdown control instruction to shut down the device to be controlled;

[0125] Step E: If the device to be controlled is turned off, triggering a start control instruction to turn on the device to be controlled.

[0126] In this embodiment, when a device switch command triggered by a device icon is detected, the device to be controlled corresponding to the device icon is determined, and it is detected whether the device to be controlled is turned on. If the device to be controlled is turned on, a shutdown control command is triggered, and if the device to be controlled is turned off, a startup control command is triggered. The device page includes a device icon option, and the device icon option is used by the user to trigger the device switch command.

[0127] The device switch command is used to switch the startup state of the device, that is, to switch between on and off. Specifically, the user can trigger the device switch command by clicking the device icon in the device page.

[0128] It should be noted that when the device to be controlled is turned on, if the device icon of the device to be controlled is clicked, the shutdown control instruction is triggered, and then the shutdown control instruction is sent to the device to be controlled to turn off the device to be controlled; when the device to be controlled is turned off, if the device icon of the device to be controlled is clicked, the startup control instruction is triggered, and then the startup control instruction is sent to the device to be controlled to turn on the current device. Of course, the device switch instruction can also be triggered by long pressing the device icon, or by other icons or options, which are not limited here.

[0129] Step S40: sending the control instruction to the device to be controlled, so that the device to be controlled executes the control instruction.

[0130] In this embodiment, the control instruction is sent to the device to be controlled so that the device to be controlled executes the control instruction. The sending method can be direct sending or forwarding through a control server. Of course, the control instruction can also be encrypted before sending to verify the source of the control instruction and ensure the remote control security of the smart device.

[0131] For easier understanding, please refer to Figure 3, where the upper part is the scene screen and the lower part is the device page. When the user selects the study scene, he can view the status of the study light, socket No. 1, desk lamp, and infrared. For example, the current status of the study light is on, but when the user views the study scene screen, no one is found, so the user can turn off the light by clicking the study light icon. Of course, the user can also long press the study light icon to enter the control menu page corresponding to the study light to achieve more control operations, such as adjusting the brightness. In addition, the user can also remotely view the activity status of people in the scene through the scene screen, such as children or the elderly who need to be cared for. By viewing the scene screen while controlling the smart device, remote care of vulnerable people at home can be achieved. Of course, in some dangerous work scenarios, users can remotely view the workflow status and the operating status of the equipment to achieve remote visual control, thereby replacing engineers to enter the dangerous work scene.

[0132] The embodiment of the present invention provides a device visualization control method, which is applied to a user terminal. The device visualization control method obtains a scene screen and a device page corresponding to the scene screen, wherein the scene screen is a screen captured by a camera of the scene in which the device is located, and the device page is a control page corresponding to the device; the scene screen and the device page are displayed in a split screen on the user terminal so that the user can watch the scene screen and trigger a control instruction through the device page at the same time; when a control instruction is detected, the device to be controlled is determined according to the control instruction; the control instruction is sent to the device to be controlled so that the device to be controlled executes the control instruction. The embodiment of the present invention displays the scene screen and the device page in a split screen so that the user can remotely watch the device status and surrounding environment in the scene while remotely controlling the device, thereby preventing damage to the device caused by blindly controlling the device and improving the accuracy of remote control of the device.

[0133] Furthermore, based on the above-mentioned first embodiment, a second embodiment of the device visualization control method of the present invention is proposed.

[0134] In this embodiment, in the above step S10, obtaining the scene picture includes:

[0135] Step a11, when the user terminal displays a scene selection page, if a triggering scene selection instruction is detected, determining a scene to be viewed based on the scene selection instruction, wherein the scene selection page includes a plurality of scenes, so that the user triggers the scene selection instruction based on the scene selection page;

[0136] Step a12, determining a camera for capturing the scene to be viewed, and sending a camera instruction to the camera, so that the camera captures a scene image corresponding to the scene to be viewed, and sends the scene image to the user terminal;

[0137] Step a13, receiving the scene image sent by the camera.

[0138] In this embodiment, when the scene selection page is displayed, it is detected whether the scene selection instruction is triggered. If the scene selection instruction is detected, the scene to be viewed is determined based on the scene selection instruction, and then the camera to shoot the scene to be viewed is determined, and a camera instruction is sent to the camera so that the camera captures the scene picture corresponding to the scene to be viewed, and the scene picture is sent to the user terminal, and finally, the scene picture sent by the camera is received. Among them, the scene selection page includes multiple scenes, so that the user can trigger the scene selection instruction based on the scene selection page.

[0139] It should be noted that the scene selection page includes multiple scenes, such as staircase scene, study, garage, balcony, master bedroom, entertainment room and other scenes. Figure 8 , Figure 8 This is a schematic diagram of the sixth page involved in an embodiment of the present invention, wherein the scene option at the bottom is used to enter the scene selection page. Specifically, after the user clicks the scene option, the page at the top will be replaced with the scene option page. The scene selection page can be referred to Figure 5 , where the main page 1 is the application page of the user terminal, and the main page 1 includes a scene selection page 5, and the scene selection page 5 includes scene 1, scene 2, and scene 3. It can be understood that when connecting different smart devices, all smart devices may include multiple scenes. Therefore, the main page of a specific visual remote control can be accurately entered through the scene selection page, that is, entering Figure 4 page.

[0140] In addition, it should be noted that scene selection can also be done through Figure 8 The device option in the middle and lower part enters the device selection page, which can be displayed in the form of a device list. Among them, the device list includes the identification information of each smart device, and the scene corresponding to the smart device can be determined based on the identification information. The device list is used to manage all smart devices so that the smart device can accurately access the APP (Application) of the smartphone. In addition, the device identification is the unique identification of the smart device and can be used to determine the only smart device. It can be understood that smart devices can be managed through the device list so that multiple smart devices can be controlled by only one APP.

[0141] Each smart device is equipped with a camera to shoot the smart device, that is, there is at least one camera in each scene. The camera is usually located above or in front of the smart device to ensure that the camera can clearly and completely shoot the smart device. It can be understood that different smart devices may have different cameras. In a specific embodiment, the specific step of determining the camera that shoots the device to be controlled is to determine the camera identification of the device to be controlled, so as to accurately send the camera instruction to the corresponding camera according to the identification.

[0142] In addition, it should be noted that the camera instruction is used to instruct the camera to start shooting the device to be controlled, so that the camera returns the captured device image to the APP.

[0143] In this embodiment, the camera corresponding to the device to be controlled is instructed to take a picture through a camera instruction, so that the device screen can be displayed on the application page of the APP later, allowing the user to perform control operations based on the screen of the device to be controlled, thereby further ensuring the accuracy and safety of the device's visual control.

[0144] Furthermore, based on the above-mentioned first embodiment, a third embodiment of the device visualization control method of the present invention is proposed.

[0145] In this embodiment, in the above step S10, obtaining the device page corresponding to the scene screen includes:

[0146] Step a14, determining the controllable devices belonging to the scene picture;

[0147] Step a15, obtaining device information of the controllable device, wherein the device information includes device name, device status and device icon options;

[0148] Step a16, constructing a device page corresponding to the scene screen according to the device information.

[0149] In this embodiment, the controllable device is determined according to the scene screen, then the device information is obtained according to the controllable device, and finally the device page is determined according to the device information, wherein the device information includes the device name, device status and device icon options.

[0150] It should be noted that the scene currently being viewed can be determined based on the scene screen, and the corresponding controllable device can be determined based on the currently being viewed scene. The controllable device includes one or more. According to the determined controllable device, its device options and device status are obtained respectively. The device options can be displayed in the form of icons, and the device status includes online and disconnected. Specifically, the device page includes device options and device status, refer to Fig. 9 , Fig. 9This is a seventh page schematic diagram of an embodiment of the present invention, wherein the device option is an icon of a light, a name "garage" is displayed below the icon, and a device status "online" is displayed below the name. In addition, if the device status is displayed as "online", it means that the smart device is successfully connected to the network, otherwise it means that the smart device fails to connect to the network.

[0151] In this embodiment, a device page including device options and device status is obtained so that the device page can be displayed on the application page of the APP later, so that the user can control the device to be controlled based on the device page, thereby realizing the remote control function of the device, and combined with the scene screen, the convenience of visual control of the device can be improved.

[0152] Furthermore, based on the above-mentioned first embodiment, a fourth embodiment of the device visualization control method of the present invention is proposed.

[0153] In this embodiment, the above step S40 includes:

[0154] Step a41, obtaining a secret key for encrypting the control instruction;

[0155] Step a42, encrypting the control instruction by using the secret key to generate an instruction ciphertext;

[0156] Step a43, sending the instruction ciphertext to the control server, and the control server forwarding the instruction ciphertext to the device to be controlled, so that the device to be controlled decrypts the instruction ciphertext to obtain the control instruction.

[0157] In this embodiment, a secret key for encrypting the control instruction is obtained, and then the control instruction is encrypted by the secret key to generate an instruction ciphertext. Finally, the instruction ciphertext is sent to the control server, and the control server forwards the instruction ciphertext to the device to be controlled, so that the device to be controlled decrypts the instruction ciphertext to obtain the control instruction.

[0158] It should be noted that the secret key is a symmetric secret key for symmetric encryption of the control instruction. It should be noted that symmetric encryption requires the use of a symmetric encryption algorithm to achieve encrypted transmission of the control instruction. The symmetric encryption algorithm can be set according to actual needs, such as the DES (Data Encryption Standard) algorithm and the AES (Advanced Encryption Standard) algorithm, etc., which are not specifically limited here.

[0159] In specific implementation, the control instructions may also be encrypted asymmetrically. However, the encryption process of asymmetrical encryption is complicated, so a symmetric encryption algorithm is used for encryption to increase the encryption speed.

[0160] It is understood that the decryption requires the use of an inverse algorithm of an asymmetric encryption algorithm or an inverse algorithm of a symmetric encryption algorithm. The symmetric encryption algorithm or the asymmetric encryption algorithm is the same as the algorithm for symmetric encryption and asymmetric encryption of the control instruction. In addition, the decryption process requires the use of a public key for decryption.

[0161] In this embodiment, a secret key is used to encrypt the control instruction to prevent the control instruction from being cracked, thereby improving the transmission security of the control instruction. At the same time, the smart device decrypts the ciphertext of the control instruction to verify the source of the control instruction and prevent illegal users from controlling the smart device.

[0162] Furthermore, based on the above-mentioned first embodiment, a fifth embodiment of the device visualization control method of the present invention is proposed.

[0163] In this embodiment, the device visualization control method of the present invention further includes:

[0164] Step F, when displaying the scene screen, if a scene change instruction is detected, determining the scene to be viewed after the change according to the scene change instruction;

[0165] Step G, determining a camera for capturing the scene to be viewed, and sending a camera instruction to the camera, so that the camera captures a current scene image corresponding to the scene to be viewed, and sends the current scene image to the user terminal;

[0166] Step H, receiving the current scene picture sent by the camera, and replacing the scene picture with the current scene picture.

[0167] When the scene picture is displayed, the user can change the scene independently. Specifically, when the scene picture is displayed, if a scene change instruction is detected, the replaced scene to be viewed is determined according to the scene change instruction; the camera for capturing the scene to be viewed is determined, and a camera instruction is sent to the camera to enable the camera to capture the current scene picture corresponding to the scene to be viewed, and send the current scene picture to the user terminal; the current scene picture sent by the camera is received, and the scene picture is replaced with the current scene picture.

[0168] In one embodiment, the scene screen includes a scene name display area. For details, refer to Figure 4, the main page 1 in the figure is the main page of the visual remote control, which includes a scene screen 2 and a device page 3. The scene screen 2 includes a scene name display area 4. The middle position of the scene name display area 4 usually displays the current scene name, that is, name 1, and is highlighted, and on both sides of name 1 are name 2 and name 3, which display the names of other scenes. When the user slides the scene screen or clicks the name in the scene name display area, the scene change instruction can be triggered to dynamically change the scene screen to the screen of other scenes. It can be understood that the corresponding device page will also be changed. Specifically, when displaying the scene screen, if a scene change instruction is detected, the scene to be viewed after the change is determined according to the scene change instruction; the camera that captures the scene to be viewed is determined, and a camera instruction is sent to the camera, so that the camera captures the current scene screen corresponding to the scene to be viewed, and sends the current scene screen to the user terminal; the current scene screen sent by the camera is received, and the scene screen is replaced with the current scene screen.

[0169] In this embodiment, after the scene screen has been displayed, the scene screen can still be changed by triggering a scene change instruction, so as to improve the flexibility of remote control of the device and enhance the user experience.

[0170] Furthermore, based on the above-mentioned first embodiment, a sixth embodiment of the device visualization control method of the present invention is proposed.

[0171] In this embodiment, the device visualization control method of the present invention further includes:

[0172] Step I, detecting whether the scene picture completely includes the picture of the device to be controlled;

[0173] Step J, if it is detected that the scene image does not completely include the image of the device to be controlled, determining a camera rotation instruction, and determining the camera corresponding to the device to be controlled;

[0174] Step J: sending the camera rotation instruction to the camera, so that the camera executes the camera rotation instruction.

[0175] In this embodiment, the camera rotation instruction is determined based on the device to be controlled and the scene image, and the camera corresponding to the device to be controlled is determined. Then, the camera rotation instruction is sent to the control server, and the control server forwards the camera rotation instruction to the camera so that the camera executes the camera rotation instruction.

[0176] It should be noted that the scene screen may only contain part of the screen of the device to be controlled, so the scene screen can be analyzed to detect whether the operation screen executed by the control instruction can be captured by the camera, so as to determine whether to turn the camera. Specifically, the control instruction can be analyzed by establishing a scene model of the device to be controlled to simulate the operation executed by the control instruction, and then comparing it with the scene screen to detect whether the operation screen executed by the control instruction can be captured by the camera.

[0177] In addition, it should be noted that the camera rotation instructions include an upward rotation instruction, a downward rotation instruction, a left rotation instruction, a right rotation instruction, and of course, other control instructions may also be included.

[0178] In this embodiment, detecting whether the operation screen executed by the control instruction can be captured by the camera can ensure that the scene screen displays the behavior status of the device to be controlled in real time, so that the user can obtain a more accurate reference standard, thereby further improving the accuracy of the device's visual control.

[0179] The invention also provides a device for visualizing control of equipment.

[0180] Reference Fig.10 , Fig.10 It is a schematic diagram of functional modules of the first embodiment of the visualization control device of the present invention.

[0181] In this embodiment, the device visualization control device includes:

[0182] The screen acquisition module 10 is used to acquire a scene screen and acquire a device page corresponding to the scene screen, wherein the scene screen is a screen captured by a camera of a scene in which the device is located, and the device page is a control page corresponding to the device;

[0183] The screen display module 20 is used to display the scene screen and the device page in a split screen on the user terminal, so that the user can watch the scene screen and trigger the control instruction through the device page at the same time;

[0184] A device determination module 30, configured to determine a device to be controlled according to the control instruction when the control instruction is detected;

[0185] The instruction sending module 40 is used to send the control instruction to the device to be controlled, so that the device to be controlled executes the control instruction.

[0186] Among them, each virtual function module of the above-mentioned equipment visualization control device is stored in Figure 1The memory 1005 of the device visualization control device shown is used to implement all functions of the device visualization control program; when each module is executed by the processor 1001, the device visualization control function can be implemented.

[0187] Furthermore, the picture acquisition module 10 includes:

[0188] an instruction detection unit, configured to determine a scene to be viewed based on the scene selection instruction if a triggering scene selection instruction is detected when the scene selection page is displayed on the user terminal, wherein the scene selection page includes a plurality of scenes so that the user triggers the scene selection instruction based on the scene selection page;

[0189] An instruction sending unit, used for determining a camera for capturing the scene to be viewed, and sending a camera instruction to the camera, so that the camera captures a scene picture corresponding to the scene to be viewed, and sends the scene picture to the user terminal;

[0190] A picture receiving unit is used to receive the scene picture sent by the camera.

[0191] Furthermore, the picture acquisition module 10 includes:

[0192] A device determining unit, used to determine a controllable device belonging to the scene picture;

[0193] An information acquisition unit, configured to acquire device information of the controllable device, wherein the device information includes a device name, a device status, and a device icon option;

[0194] The page construction unit constructs a device page corresponding to the scene screen according to the device information.

[0195] Furthermore, the device visualization control device also includes:

[0196] An instruction detection module, configured to determine a to-be-controlled device corresponding to the device menu option when a menu access instruction triggered by the device menu option is detected;

[0197] A page changing module is used to change the device page to a control menu page corresponding to the device to be controlled, wherein the control menu page can trigger multiple control instructions so that the user can trigger one or more of the multiple control instructions based on the scene screen and the control menu page.

[0198] Furthermore, the device visualization control device also includes:

[0199] A device detection module, configured to determine the device to be controlled corresponding to the device icon option and detect whether the device to be controlled is turned on when the device switch instruction triggered by the device icon option is detected;

[0200] An instruction triggering module, used for triggering a shutdown control instruction to shut down the device to be controlled if the device to be controlled is turned on;

[0201] The instruction triggering module is further used to trigger the start control instruction to turn on the device to be controlled if the device to be controlled is turned off.

[0202] Furthermore, the instruction sending module 40 includes:

[0203] A key acquisition unit, used to acquire a key for encrypting the control instruction;

[0204] An instruction encryption unit, used to encrypt the control instruction by using the secret key to generate an instruction ciphertext;

[0205] The ciphertext sending unit is used to send the instruction ciphertext to the control server, and the control server forwards the instruction ciphertext to the device to be controlled, so that the device to be controlled decrypts the instruction ciphertext to obtain the control instruction.

[0206] Furthermore, the device visualization control device also includes:

[0207] A scene determination module, configured to determine the scene to be viewed after the change according to the scene change instruction if a scene change instruction is detected when the scene picture is displayed;

[0208] A camera determination module, used to determine a camera for capturing the scene to be viewed, and send a camera instruction to the camera so that the camera captures a current scene image corresponding to the scene to be viewed, and sends the current scene image to the user terminal;

[0209] The picture receiving module is used to receive the current scene picture sent by the camera and replace the scene picture with the current scene picture.

[0210] Furthermore, the device visualization control device also includes:

[0211] A picture detection module, used to detect whether the scene picture completely includes the picture of the device to be controlled;

[0212] A camera determination module, used to determine a camera rotation instruction and determine the camera corresponding to the device to be controlled if it is detected that the scene image does not completely include the image of the device to be controlled;

[0213] The instruction sending module 40 is further used to send the camera rotation instruction to the camera, so that the camera executes the camera rotation instruction.

[0214] Among them, the functional implementation of each module in the above-mentioned equipment visualization control device corresponds to the various steps in the above-mentioned equipment visualization control method embodiment, and its functions and implementation processes are no longer repeated here one by one.

[0215] The present invention also provides an electronic device, which includes: a memory, a processor, and a device visualization control program stored in the memory and executable on the processor, wherein the device visualization control program, when executed by the processor, implements the steps of the device visualization control method described in any of the above embodiments.

[0216] The specific embodiments of the electronic device of the present invention are basically the same as the embodiments of the above-mentioned device visualization control method, and will not be described in detail here.

[0217] The present invention also provides a computer-readable storage medium, on which a device visualization control program is stored. When the device visualization control program is executed by a processor, the steps of the device visualization control method described in any of the above embodiments are implemented.

[0218] The specific embodiments of the computer-readable storage medium of the present invention are basically the same as the embodiments of the above-mentioned device visualization control method, and will not be described in detail here.

[0219] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0220] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0221] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0222] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A device visualization control method, characterized in that: Applied to a user terminal, the device visualization control method comprises the following steps: Acquire a scene screen, and acquire a device page corresponding to the scene screen, wherein the scene screen is a screen captured by a camera of a scene in which the device is located, and the device page is a control page corresponding to the device; The scene screen and the device page are displayed in a split screen on the user terminal, so that the user can watch the scene screen and trigger the control instruction through the device page at the same time, wherein the split screen display area is the main page of the application page of the user terminal, and the main page is the user interface on the APP, so that the user can view the behavior status of the device to be controlled and its surrounding environment based on the user interface while controlling the user terminal; When the control instruction is detected, determining a device to be controlled according to the control instruction; Sending the control instruction to the device to be controlled so that the device to be controlled executes the control instruction; The step of obtaining the device page corresponding to the scene screen includes: Determining controllable devices belonging to the scene picture; Acquire device information of the controllable device, wherein the device information includes device name, device status and device icon options; According to the device information, construct a device page corresponding to the scene screen; When displaying the scene selection page, detecting whether a scene selection instruction is triggered, if the scene selection instruction is detected to be triggered, determining the scene to be viewed based on the scene selection instruction, then determining a camera for shooting the scene to be viewed, and sending a camera instruction to the camera so that the camera captures a scene picture corresponding to the scene to be viewed, and sends the scene picture to the user terminal, and finally, receiving the scene picture sent by the camera, wherein the scene selection page includes multiple scenes, so that the user triggers the scene selection instruction based on the scene selection page; The device visualization control method also includes: Detecting whether the scene picture completely includes the picture of the device to be controlled; If it is detected that the scene image does not completely include the image of the device to be controlled, determine a camera rotation instruction, and determine the camera corresponding to the device to be controlled; The camera rotation instruction is sent to the camera, so that the camera executes the camera rotation instruction.

2. The device visualization control method according to claim 1, characterized in that: The step of obtaining the scene picture comprises: When the user terminal displays a scene selection page, if a triggering scene selection instruction is detected, determining a scene to be viewed based on the scene selection instruction, wherein the scene selection page includes a plurality of scenes so that the user triggers the scene selection instruction based on the scene selection page; Determine a camera for capturing the scene to be viewed, and send a camera instruction to the camera, so that the camera captures a scene image corresponding to the scene to be viewed, and sends the scene image to the user terminal; Receive the scene image sent by the camera.

3. The device visualization control method according to claim 1, characterized in that: The device page includes a device menu option. When the control instruction is detected, before the step of determining the device to be controlled according to the control instruction, the step further includes: When a menu access instruction triggered by the device menu option is detected, determining a to-be-controlled device corresponding to the device menu option; The device page is replaced with a control menu page corresponding to the device to be controlled, wherein the control menu page can trigger multiple control instructions, so that the user can trigger one or more of the multiple control instructions based on the scene screen and the control menu page.

4. The device visualization control method according to claim 1, characterized in that: The device page includes a device icon option, and the device icon option is used by a user to trigger a device switch instruction. When the control instruction is detected, before the step of determining the device to be controlled according to the control instruction, the step further includes: When the device switch instruction triggered by the device icon option is detected, determining the device to be controlled corresponding to the device icon option, and detecting whether the device to be controlled is turned on; If the device to be controlled is turned on, a closing control instruction is triggered to close the device to be controlled; If the device to be controlled is turned off, a start control instruction is triggered to turn on the device to be controlled.

5. The device visualization control method according to claim 1, characterized in that: The step of sending the control instruction to the device to be controlled comprises: Obtaining a secret key for encrypting the control instruction; Encrypting the control instruction by using the secret key to generate an instruction ciphertext; The instruction ciphertext is sent to a control server, and the control server forwards the instruction ciphertext to the device to be controlled, so that the device to be controlled decrypts the instruction ciphertext to obtain the control instruction.

6. The device visualization control method according to any one of claims 1 to 5, characterized in that: The device visualization control method also includes: When the scene picture is displayed, if a scene change instruction is detected, determining the scene to be viewed after the change according to the scene change instruction; Determine a camera for capturing the scene to be viewed, and send a camera instruction to the camera, so that the camera captures a current scene image corresponding to the scene to be viewed, and sends the current scene image to the user terminal; Receive the current scene picture sent by the camera, and replace the scene picture with the current scene picture.

7. A device visualization control device, deployed in a user terminal, characterized in that: The equipment visualization control device comprises: A picture acquisition module is used to acquire a scene picture and acquire a device page corresponding to the scene picture, wherein the scene picture is a picture captured by a camera of a scene in which the device is located, and the device page is a control page corresponding to the device; A screen display module, used for displaying the scene screen and the device page in a split screen on the user terminal, so that the user can watch the scene screen and trigger the control instruction through the device page at the same time, wherein the split screen display area is the main page of the application page of the user terminal, and the main page is the user interface on the APP, so that the user can view the behavior status of the device to be controlled and its surrounding environment based on the user interface while controlling the user terminal; A device determination module, configured to determine a device to be controlled according to the control instruction when the control instruction is detected; An instruction sending module, used for sending the control instruction to the device to be controlled, so that the device to be controlled executes the control instruction; The screen acquisition module is also used to determine the controllable device belonging to the scene screen; obtain device information of the controllable device, wherein the device information includes device name, device status and device icon options; and construct a device page corresponding to the scene screen according to the device information; When displaying the scene selection page, detecting whether a scene selection instruction is triggered, if the scene selection instruction is detected to be triggered, determining the scene to be viewed based on the scene selection instruction, then determining a camera for shooting the scene to be viewed, and sending a camera instruction to the camera so that the camera captures a scene picture corresponding to the scene to be viewed, and sends the scene picture to the user terminal, and finally, receiving the scene picture sent by the camera, wherein the scene selection page includes multiple scenes, so that the user triggers the scene selection instruction based on the scene selection page; The device visualization control device also includes: Detecting whether the scene picture completely includes the picture of the device to be controlled; If it is detected that the scene image does not completely include the image of the device to be controlled, determine a camera rotation instruction, and determine the camera corresponding to the device to be controlled; The camera rotation instruction is sent to the camera, so that the camera executes the camera rotation instruction.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a device visualization control program, and when the device visualization control program is executed by a processor, the steps of the device visualization control method according to any one of claims 1 to 6 are implemented.

Images