Device control method, device and storage medium

Abstract

The present disclosure relates to a device control method, apparatus and storage medium, which solves the problem in the related art that the device control is affected by the angle of the device image, and the device cannot be controlled quickly and accurately. The method includes: obtaining an AR anchor point in the image captured by the camera, and the AR anchor point is pre-created according to the spatial position of the target device; determining the target device bound to the spatial position anchored by the AR anchor point; displaying the control menu of the target device; generating a control instruction in response to the operation on the control menu, and controlling the target device according to the control instruction. The control menu of the target device can be obtained at any angle and position in the space where the target device is located, and each device corresponds to an anchor point. In the case of multiple devices, the control menu of the target device can be accurately obtained to control the target device.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent device control, and in particular to a device control method, device and storage medium. Background Art

[0002] With the increasing popularity of smart devices, traditional methods such as hardware switches, control panels, and remote controls cannot meet users' needs for efficient control of smart devices.

[0003] In the related art, there are two ways to control smart devices. The first is to control the corresponding device by searching for the control menu of the device through the mobile phone APP; the second is to control the device by identifying the device image, obtaining the device type and determining the device control menu. However, the first method requires users to search one by one when there are many devices, which is inefficient; the device image of the second method needs to contain complete device identification information, which is affected by the angle of the device image, and because some devices have similar identifications or less feature information, it cannot control the device efficiently. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a device control method, apparatus and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided a device control method, including:

[0006] Obtaining an AR anchor point in the image captured by the camera, where the AR anchor point is pre-created based on the spatial position of the target device;

[0007] Determine a target device bound to the spatial position anchored by the AR anchor point;

[0008] Displaying a control menu of the target device;

[0009] Determining a corresponding control menu according to the identification information, and displaying the control menu;

[0010] In response to the operation on the control menu, a control instruction is generated, and the target device is controlled according to the control instruction.

[0011] Optionally, obtaining an AR anchor point in an image captured by a camera includes:

[0012] Performing AR anchor point recognition in the image captured by the camera;

[0013] When an AR anchor point is identified from the image captured by the camera, acquiring anchor point identification information of the AR anchor point;

[0014] The determining of a target device bound to a spatial position anchored by the AR anchor point includes:

[0015] According to the anchor point identification information of the AR anchor point and the correspondence between the pre-stored anchor point identification information and the device identification information, a target device bound to the spatial position anchored by the AR anchor point is acquired.

[0016] Optionally, when an AR anchor point is identified from the image captured by the camera, the method further includes:

[0017] Determining whether there is a target device corresponding to the AR anchor point in the image;

[0018] In a case where the target device corresponding to the AR anchor point does not exist in the image, a reminder message is issued to remind the user to delete the AR anchor point or update the AR anchor point of the target device.

[0019] Optionally, after controlling the target device according to the control instruction, the method includes:

[0020] If no control response from the target device is received within a preset time period, determining whether there is a target device corresponding to the AR anchor point in the image;

[0021] In a case where the target device corresponding to the AR anchor point does not exist in the image, a reminder message is issued to remind the user to delete the AR anchor point or update the AR anchor point of the target device.

[0022] Optionally, the AR anchor point is created in the following manner:

[0023] Acquire the spatial position of the target device through the camera, and acquire the display position of the image of the target device in the preview interface of the terminal;

[0024] Creating an AR anchor point corresponding to the target device according to the spatial position and the display position;

[0025] Anchor point identification information is generated according to the AR anchor point, the anchor point identification information is bound to the device identification information, and the binding relationship is sent to a server for storage, wherein the server also stores a correspondence between the device identification information of the target device and a control menu of the target device.

[0026] Optionally, obtaining the spatial position of the target device through the camera includes:

[0027] Performing spatial feature recognition through the camera to obtain multiple spatial feature points;

[0028] The creating an AR anchor point corresponding to the target device according to the spatial position and the display position includes:

[0029] The AR anchor point is created according to a positional relationship between the display position and the plurality of spatial feature points.

[0030] Optionally, displaying a control menu of the target device includes:

[0031] According to the position of the AR anchor point in the image, displaying the control menu of the target device within a preset range of the position; or,

[0032] In the case that there are multiple AR anchor points, a control menu of a target device corresponding to an AR anchor point closest to the user is displayed according to the distance of each AR anchor point relative to the user.

[0033] According to a second aspect of an embodiment of the present disclosure, there is provided a device control apparatus, which is applied to a terminal provided with a camera, and includes:

[0034] An acquisition module is configured to acquire an AR anchor point in an image captured by a camera, wherein the AR anchor point is pre-created according to a spatial position of a target device;

[0035] A control module is configured to determine a target device bound to a spatial position anchored by the AR anchor point;

[0036] A display module, configured to display a control menu of the target device;

[0037] The execution module is configured to generate a control instruction in response to an operation on the control menu, and control the target device according to the control instruction.

[0038] Optionally, the acquisition module is configured to perform AR anchor point recognition in the image captured by the camera;

[0039] When an AR anchor point is identified from the image captured by the camera, acquiring anchor point identification information of the AR anchor point;

[0040] According to the anchor point identification information of the AR anchor point and the correspondence between the pre-stored anchor point identification information and the device identification information, a target device bound to the spatial position anchored by the AR anchor point is determined.

[0041] Optionally, the apparatus further includes a determination module, configured to determine whether a target device corresponding to the AR anchor point exists in the image;

[0042] In a case where the target device corresponding to the AR anchor point does not exist in the image, a reminder message is issued to remind the user to delete the AR anchor point or update the AR anchor point of the target device.

[0043] Optionally, the execution module is configured to determine whether there is a target device corresponding to the AR anchor point in the image if no control response of the target device is received within a preset time period;

[0044] In a case where the target device corresponding to the AR anchor point does not exist in the image, a reminder message is issued to remind the user to delete the AR anchor point or update the AR anchor point of the target device.

[0045] Optionally, the apparatus further comprises an anchor point creation module, wherein the anchor point creation module is configured to obtain the spatial position of the target device through the camera, and obtain the display position of the target device placed in the preview interface of the terminal;

[0046] Creating an AR anchor point corresponding to the target device according to the spatial position and the display position;

[0047] Anchor point identification information is generated according to the AR anchor point, the anchor point identification information is bound to the device identification information, and the binding relationship is sent to a server for storage, wherein the server also stores a correspondence between the device identification information of the target device and a control menu of the target device.

[0048] Optionally, the anchor point creation module is configured to perform spatial feature recognition through the camera to obtain a plurality of spatial feature points;

[0049] The AR anchor point is created according to a positional relationship between the display position and the plurality of spatial feature points.

[0050] The display module includes a first display submodule and a second display submodule;

[0051] The first submodule is configured to display the control menu of the target device within a preset range of the position according to the position of the AR anchor point in the image;

[0052] The second submodule is configured to display a control menu of a target device corresponding to an AR anchor point closest to the user according to a distance of each AR anchor point relative to the user when there are multiple AR anchor points.

[0053] According to a third aspect of this embodiment, there is provided a device control apparatus, including:

[0054] processor;

[0055] a memory for storing processor-executable instructions;

[0056] Wherein, the processor is configured to:

[0057] Obtaining an AR anchor point in the image captured by the camera, where the AR anchor point is pre-created based on the spatial position of the target device;

[0058] Determine a target device bound to the spatial position anchored by the AR anchor point;

[0059] Displaying a control menu of the target device;

[0060] In response to the operation on the control menu, a control instruction is generated, and the target device is controlled according to the control instruction.

[0061] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the device control method provided in the first aspect of the present disclosure are implemented.

[0062] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: the present disclosure obtains an AR anchor point in an image, determines the target device bound to the spatial position anchored by the AR anchor point according to the AR anchor point, displays the control menu of the target device, and controls the target device through the control menu. The control menu of the target device can be obtained at any angle and position in the space where the target device is located, and each device corresponds to an anchor point. In the case of multiple devices, the control menu of the target device can be accurately obtained to control the target device.

[0063] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0065] Figure 1 The figure is a flow chart of a device control method according to an exemplary embodiment.

[0066] Figure 2 is a flowchart of step S110 of a device control method according to an exemplary embodiment.

[0067] Figure 3 The figure is a schematic diagram showing a control menu of the target device according to an exemplary embodiment.

[0068] Figure 4The diagram is a schematic diagram showing another method of displaying a control menu of the target device according to an exemplary embodiment.

[0069] Figure 5 is another flow chart of step S110 of a device control method according to an exemplary embodiment.

[0070] Figure 6 is another flowchart of a device control method according to an exemplary embodiment.

[0071] Figure 7 The invention is a block diagram of a device control apparatus according to an exemplary embodiment.

[0072] Figure 8 The invention is a block diagram of a device control apparatus according to an exemplary embodiment. DETAILED DESCRIPTION

[0073] With the increasing popularity of smart devices, traditional methods such as hardware switches, control panels, and remote controls cannot meet users' needs for efficient control of smart devices.

[0074] In the related art, there are two ways to control smart devices. The first is to use a mobile phone APP to find the corresponding device to be controlled, click in to obtain the device's control menu to operate, and realize the control of the smart device. This method requires users to search one by one when there are many devices, which is inefficient; when there are many devices of the same type, it is impossible to determine the specific device to be controlled.

[0075] The second method is to identify the device to be controlled by identifying the device image, obtain the device type and determine the device control menu to control the device. This method is affected by the shooting angle of the device image. If the device identification information in the device image is incomplete, it is impossible to determine the device to be controlled. In addition, since the identification of some devices is similar or the feature information is less, the device cannot be controlled efficiently.

[0076] The present disclosure provides a device control method, apparatus and storage medium that can overcome the technical problems in the related art of low device control efficiency and the inability to efficiently control the device due to the influence of the shooting angle of the device image.

[0077] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0078] Figure 1 is a flow chart of a device control method according to an exemplary embodiment. Figure 1 As shown, the device control method is used in a terminal provided with a camera, which terminal may be a mobile phone, a tablet, etc., and includes the following steps.

[0079] In step S110, an AR anchor point in the image captured by the camera is obtained, where the AR anchor point is pre-created according to the spatial position of the target device.

[0080] In step S120, a target device bound to the spatial position anchored by the AR anchor point is determined.

[0081] In step S130, a control menu of the target device is displayed.

[0082] In step S140, in response to the operation on the control menu, a control instruction is generated, and the target device is controlled according to the control instruction.

[0083] The control menu of the target device may be displayed in a preview interface (ie, an image viewfinder) of the terminal, which is not specifically limited in the present disclosure.

[0084] Specifically, the target device in this embodiment may be a terminal hardware device with intelligent capabilities, such as a smart lamp, a door lock, an air conditioner, and an air purifier.

[0085] For example, an AR anchor point in an image captured by a camera is obtained, target device identification information bound to the spatial position anchored by the AR anchor point is determined, a control menu of the target device is displayed to the user, a control instruction is generated in response to the user's operation on the control menu, and the target device is controlled according to the control instruction.

[0086] Optionally, the AR anchor point can be created in the following way:

[0087] Acquire the spatial position of the target device through the camera, and acquire the display position of the image of the target device in the preview interface of the terminal;

[0088] Creating an AR anchor point corresponding to the target device according to the spatial position and the display position;

[0089] Anchor point identification information is generated according to the AR anchor point, the anchor point identification information is bound to the device identification information, and the binding relationship is sent to a server for storage, wherein the server also stores a correspondence between the device identification information of the target device and a control menu of the target device.

[0090] The spatial position of the target device may be the position of the target device in the physical space, and the display position of the target device may be the position of the target device in the virtual space where the anchor point is established, which is not specifically limited in the present disclosure.

[0091] Among them, the preview interface of the terminal can be preset according to the user's usage habits, and the present disclosure does not make any specific limitations on this.

[0092] For example, it is determined that the device in the preview interface of the terminal is the target device for which an AR anchor point needs to be created, the display position of the device and the spatial position of the device are obtained through the camera, and an AR anchor point corresponding to the device is created according to the display position and the spatial position; anchor point identification information is generated according to the AR anchor point, and the anchor point identification information of the AR anchor point and the device identification information of the device are bound to achieve binding of the AR anchor point and the device; the binding relationship is sent to the server for storage, and the server also stores the correspondence between the device identification information of the device and the control menu of the device, so as to facilitate the subsequent determination of the control menu of the device according to the AR anchor point and control of the device through the control menu.

[0093] Optionally, obtaining the spatial position of the target device through the camera may include:

[0094] Performing spatial feature recognition through the camera to obtain multiple spatial feature points;

[0095] The creating an AR anchor point corresponding to the target device according to the spatial position and the display position may include:

[0096] The AR anchor point is created according to a positional relationship between the display position and the plurality of spatial feature points.

[0097] For example, spatial feature recognition is performed through a camera to obtain multiple spatial feature points, the position of the device relative to the space is determined based on the positional relationship between the display position and the multiple spatial feature points, and an AR anchor point is created at the position of the device relative to the space. After the AR anchor point is created, the position of the device in the current space can also be obtained based on the AR anchor point without moving the position of the device. For example, the spatial feature recognition of the living room is performed through a camera to obtain multiple spatial feature points. The spatial feature points include but are not limited to points on the windows, doors, and TV walls of the living room that characterize the distribution characteristics of the living room; there is an air conditioner placed in the living room, and the air conditioner is placed on the floor of the living room. Based on the position relationship between the position of the air conditioner and the spatial feature points of the living room, it is determined that the air conditioner is placed on the ground next to the living room window, and an AR anchor point is created at the ground position next to the living room window where the air conditioner is located.

[0098] Optional, such as Figure 2 As shown, step S110 may include:

[0099] In step S1101, AR anchor point recognition is performed in the image captured by the camera;

[0100] In step S1102, when an AR anchor point is identified from the image captured by the camera, anchor point identification information of the AR anchor point is acquired;

[0101] The determining of the target device bound to the spatial position anchored by the AR anchor point may include:

[0102] According to the anchor point identification information of the AR anchor point and the correspondence between the pre-stored anchor point identification information and the device identification information, a target device bound to the spatial position anchored by the AR anchor point is determined.

[0103] For example, the terminal captures an image of the current space through the camera of the terminal, and the terminal identifies the AR anchor point in the image. When the AR anchor point is identified from the image, the anchor point identification information of the AR anchor point is obtained; based on the anchor point identification information of the AR anchor point, and the correspondence between the pre-stored anchor point identification information and the device identification information, the target device bound to the spatial position anchored by the AR anchor point is determined. For example, the camera captures an image of the living room, identifies the AR anchor point in the image, and when the AR anchor point is identified from the image, the anchor point identification information of the AR anchor point is obtained; based on the anchor point identification information of the AR anchor point and the correspondence between the pre-stored anchor point identification information and the device identification information, the air conditioner bound to the spatial position anchored by the AR anchor point is determined.

[0104] Optionally, displaying the control menu of the target device may include:

[0105] According to the position of the AR anchor point in the image, displaying the control menu of the target device within a preset range of the position; or,

[0106] In the case that there are multiple AR anchor points, a control menu of a target device corresponding to an AR anchor point closest to the user is displayed according to the distance of each AR anchor point relative to the user.

[0107] Specifically, Figure 3As shown, in the case where the captured image includes multiple AR anchor points of curtains, lights, bed temperature, and fans, the control menus of curtains, lights, bed temperature, and fans can be displayed within the preset range of the position according to the position of each AR anchor point in the image, and the user can select the control menu as needed to control one of the curtains, lights, bed temperature, and fans, wherein the captured image may include one curtain AR anchor point, or may include multiple AR anchor points of curtains, lights, bed temperature, and fans at the same time. The present disclosure does not specifically limit this.

[0108] like Figure 4 As shown, in the case where the captured image includes multiple AR anchor points such as Curtains, Lights, Bed Temp, and Fan, a control menu for the light corresponding to the AR anchor point closest to the user can be displayed according to the distance of each AR anchor point relative to the user, and the user can control the light according to the control menu for the light. This disclosure does not specifically limit this.

[0109] Among them, when the control menus of multiple target devices are displayed, the main switch (All Off) can also be used to realize the one-key shutdown function of all devices in the image. It can be used when multiple devices are turned on at the same time and the user is in a hurry to go out. The present disclosure does not make specific limitations on this.

[0110] Among them, one anchor point identification information corresponds to one device identification information.

[0111] Optional, such as Figure 5 As shown, step S1101 may also include:

[0112] In step S1101, AR recognition is performed in the image captured by the camera;

[0113] In step S1103, when an AR anchor point is identified in the image captured by the camera, it is determined whether there is a target device corresponding to the AR anchor point in the image;

[0114] In step S1104, when there is no target device corresponding to the AR anchor point in the image, a reminder message is issued to remind the user to delete the AR anchor point or update the AR anchor point of the target device.

[0115] Specifically, the reminder message may be a text message or a voice message, which is not specifically limited in the present disclosure.

[0116] For example, when an AR anchor point is identified in the image of the living room captured by the camera, it is determined whether there is an air conditioner corresponding to the AR anchor point in the image; when the air conditioner is moved out of the living room and placed in the bedroom, if the air conditioner does not exist in the image of the living room captured, a reminder message is issued to remind the user to delete the AR anchor point; when the air conditioner is moved from its original position to another position in the living room, if the air conditioner does not exist in the image of the living room captured, a reminder message is issued to remind the user to update the AR anchor point of the air conditioner.

[0117] Optional, such as Figure 6 As shown, the device control method disclosed in this embodiment may also include the following steps:

[0118] In step S110, an AR anchor point in the image captured by the camera is obtained, where the AR anchor point is pre-created according to the spatial position of the target device.

[0119] In step S120, a target device bound to the spatial position anchored by the AR anchor point is determined.

[0120] In step S130, a control menu of the target device is displayed.

[0121] In step S140, in response to the operation on the control menu, a control instruction is generated, and the target device is controlled according to the control instruction.

[0122] In step S150, when no control response of the target device is received within a preset time period, it is determined whether there is a target device corresponding to the AR anchor point in the image.

[0123] In step S160 , when there is no target device corresponding to the AR anchor point in the image, a reminder message is issued to remind the user to delete the AR anchor point or update the AR anchor point of the target device.

[0124] The preset duration may be preset according to the actual response of the target device, and the present disclosure does not make any specific limitation on this.

[0125] For example, after the air conditioner is damaged and discarded or given to others, the AR anchor point in the image captured by the camera is obtained, the target device bound to the spatial position anchored by the AR anchor point is determined, and a control menu of the target device is displayed to the user. In response to the user's operation on the control menu, a control instruction is generated, and the air conditioner (target device) is controlled according to the control instruction. If no control response of the air conditioner is received within 5 minutes, it is determined whether there is an air conditioner corresponding to the AR anchor point in the image of the living room. If the air conditioner corresponding to the AR anchor point is not stored in the image, a reminder message for reminding the user to delete the AR anchor point is issued;

[0126] After moving the air conditioner from the living room to the study, obtain the AR anchor point in the image captured by the camera, determine the target device bound to the spatial position anchored by the AR anchor point, display the control menu of the target device to the user, generate a control instruction in response to the user's operation on the control menu, and control the air conditioner (target device) according to the control instruction. If no control response of the air conditioner is received within 5 minutes, determine whether there is an air conditioner corresponding to the AR anchor point in the image of the living room. If the air conditioner corresponding to the AR anchor point is not stored in the image, issue a reminder message to remind the user to update the AR anchor point of the air conditioner.

[0127] Optionally, controlling the target device according to the control instruction may include:

[0128] The control instruction is sent to a server, so that the server controls the target device corresponding to the device identification information according to the control instruction.

[0129] The device control method disclosed in this embodiment can obtain an AR anchor point in the image captured by a camera at any angle and position in the space where the device is located, determine the target device anchored by the AR anchor point, display the control menu of the target device, and control the target device through the control menu. Each device corresponds to an anchor point. When there are multiple devices in the system, the control menu of each device can be quickly and accurately obtained through the AR anchor point, and the device can be efficiently controlled without being affected by the angle of the image captured by the camera.

[0130] Figure 7 is a block diagram of a device control apparatus according to an exemplary embodiment. Figure 7 The device control apparatus 500 includes an acquisition module 501 , a control module 502 , a display module 503 and an execution module 504 .

[0131] The device control device disclosed in this embodiment can obtain the AR anchor point in the image captured by the camera at any angle and position in the space where the device is located, determine the target device bound to the spatial position anchored by the AR anchor point, display the control menu of the target device, and control the target device through the control menu. Each device corresponds to an anchor point. When there are multiple devices in the system, the control menu of each device can be quickly and accurately obtained through the AR anchor point, and the device can be efficiently controlled without being affected by the angle of the image captured by the camera.

[0132] The acquisition module 501 is configured to acquire an AR anchor point in an image captured by a camera, wherein the AR anchor point is pre-set according to a spatial position of a target device.

[0133] The control module 502 is configured to determine a target device bound to the spatial position anchored by the AR anchor point.

[0134] The display module 503 is configured to display a control menu of the target device.

[0135] The execution module 504 is configured to generate a control instruction in response to an operation on the control menu, and control the target device according to the control instruction.

[0136] Optionally, the acquisition module 501 may specifically perform AR anchor point recognition in the image captured by the camera;

[0137] When an AR anchor point is identified from the image captured by the camera, acquiring anchor point identification information of the AR anchor point;

[0138] According to the anchor point identification information of the AR anchor point and the correspondence between the pre-stored anchor point identification information and the device identification information, a target device bound to the spatial position anchored by the AR anchor point is determined.

[0139] Optionally, the execution module 504 may specifically determine whether there is a target device corresponding to the AR anchor point in the image when no control response of the target device is received within a preset time period;

[0140] In a case where the target device corresponding to the AR anchor point does not exist in the image, a reminder message is issued to remind the user to delete the AR anchor point or update the AR anchor point of the target device.

[0141] Optionally, the execution module 504 may also send the control instruction to a server, so that the server controls the target device according to the control instruction.

[0142] Optionally, the device control apparatus 500 further includes a determination module, wherein the determination module is configured to determine whether there is a target device corresponding to the AR anchor point in the image;

[0143] In a case where the target device corresponding to the AR anchor point does not exist in the image, a reminder message is issued to remind the user to delete the AR anchor point or update the AR anchor point of the target device.

[0144] Optionally, the device control apparatus 500 further includes an anchor point creation module, wherein the anchor point creation module is configured to obtain the spatial position of the target device through the camera, and obtain the display position of the target device placed in the preview interface of the terminal;

[0145] Creating an AR anchor point corresponding to the target device according to the spatial position and the display position;

[0146] Anchor point identification information is generated according to the AR anchor point, the anchor point identification information is bound to the device identification information, and the binding relationship is sent to a server for storage, wherein the server also stores a correspondence between the device identification information of the target device and a control menu of the target device.

[0147] Optionally, the anchor point creation module may specifically perform spatial feature recognition through the camera to obtain a plurality of spatial feature points;

[0148] The AR anchor point is created according to a positional relationship between the display position and the plurality of spatial feature points.

[0149] Optionally, the display module may include a first display submodule and a second display submodule;

[0150] The first submodule may display the control menu of the target device within a preset range of the position according to the position of the AR anchor point in the image;

[0151] When there are multiple AR anchor points, the second submodule may display a control menu of a target device corresponding to an AR anchor point closest to the user according to the distance of each AR anchor point relative to the user.

[0152] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0153] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, and the program instructions, when executed by a processor, implement the steps of the device control method provided by the present disclosure.

[0154] Specifically, the computer-readable storage medium can be a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, an optical disk, a server, etc.

[0155] Regarding the computer-readable storage medium in the above embodiment, the method steps when the computer program stored thereon is executed have been described in detail in the embodiment of the method and will not be elaborated here.

[0156] The present disclosure also provides a device control apparatus, which may be a computer, a platform device, etc. The device control apparatus includes:

[0157] processor;

[0158] a memory for storing processor-executable instructions;

[0159] Wherein, the processor is configured to:

[0160] Obtaining an AR anchor point in the image captured by the camera, where the AR anchor point is pre-created based on the spatial position of the target device;

[0161] Determine a target device bound to the spatial position anchored by the AR anchor point;

[0162] Displaying a control menu of the target device;

[0163] In response to the operation on the control menu, a control instruction is generated, and the target device is controlled according to the control instruction.

[0164] The device control device can obtain the AR anchor point in the image captured by the camera at any angle and position in the space where the device is located, determine the target device bound to the spatial position anchored by the AR anchor point, display the control menu of the target device, and control the target device through the control menu. Each device corresponds to an anchor point. When there are multiple devices in the system, the control menu of each device can be quickly and accurately obtained through the AR anchor point, and the device can be efficiently controlled without being affected by the angle of the image captured by the camera.

[0165] Figure 8 is a block diagram of a device control apparatus 600 according to an exemplary embodiment. For example, the apparatus 600 may be a mobile phone, a computer, a tablet device, etc.

[0166] Reference Figure 8 The device control apparatus 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0167] The processing component 602 generally controls the overall operation of the device 600, such as operations associated with the display. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned device control method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0168] The memory 604 is configured to store various types of data to support operations on the device 600. Examples of such data include instructions for any application or method operating on the device 600, images, correspondence between anchor identification information and device identification information, correspondence between device identification information of a target device and a control menu of the target device, etc. The memory 604 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0169] The power component 606 provides power to the various components of the device 600. The power component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 600.

[0170] The multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may acquire external images. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0171] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a speaker, and when the apparatus 600 is in an operating mode, such as controlling a target device, the speaker is configured to emit a voice message for reminding the user to delete the AR anchor point or update the AR anchor point of the target device.

[0172] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0173] Sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for device 600. For example, sensor assembly 614 can detect the open / closed state of device 600, the relative positioning of components, such as the display and keypad of device 600, and sensor assembly 614 can also detect changes in the position of device 600 or a component of device 600, the presence or absence of user contact with device 600, the orientation or acceleration / deceleration of device 600, and changes in the temperature of device 600. Sensor assembly 614 can also include optical sensors, such as CMOS or CCD image sensors, for use in imaging applications.

[0174] The communication component 616 is configured to facilitate wired or wireless communication between the device 600 and other devices. The device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0175] In an exemplary embodiment, the device 600 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned device control method.

[0176] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of the apparatus 600 to complete the above-mentioned device control method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0177] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0178] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A device control method, applied to a terminal provided with a camera, characterized in that: The method comprises: Acquire an AR anchor point in the image captured by the camera, where the AR anchor point is pre-created according to the spatial position of the target device; Determine a target device bound to the spatial position anchored by the AR anchor point; Displaying a control menu of the target device; In response to an operation on the control menu, a control instruction is generated, and the target device is controlled according to the control instruction; The obtaining of the AR anchor point in the image captured by the camera includes: Performing AR anchor point recognition in the image captured by the camera; When an AR anchor point is identified from the image captured by the camera, acquiring anchor point identification information of the AR anchor point; The determining of a target device bound to a spatial position anchored by the AR anchor point includes: Determine a target device bound to the spatial position anchored by the AR anchor point according to the anchor point identification information of the AR anchor point and the correspondence between the pre-stored anchor point identification information and the target device identification information; The AR anchor is created as follows: Acquire the spatial position of the target device through the camera, and acquire the display position of the image of the target device in the preview interface of the terminal; Creating an AR anchor point corresponding to the target device according to the spatial position and the display position; generating anchor point identification information according to the AR anchor point, binding the anchor point identification information with the device identification information, and sending the binding relationship to a server for storage, wherein the server also stores a correspondence between the device identification information of the target device and a control menu of the target device; The acquiring the spatial position of the target device through the camera includes: Performing spatial feature recognition through the camera to obtain multiple spatial feature points; The creating an AR anchor point corresponding to the target device according to the spatial position and the display position includes: The AR anchor point is created according to a positional relationship between the display position and the plurality of spatial feature points.

2. The method according to claim 1, characterized in that In the case where an AR anchor point is identified from the image captured by the camera, the method further includes: Determining whether there is a target device corresponding to the AR anchor point in the image; In a case where the target device corresponding to the AR anchor point does not exist in the image, a reminder message is issued to remind the user to delete the AR anchor point or update the AR anchor point of the target device.

3. The method according to claim 1, characterized in that After controlling the target device according to the control instruction, the method includes: If no control response from the target device is received within a preset time period, determining whether there is a target device corresponding to the AR anchor point in the image; In a case where the target device corresponding to the AR anchor point does not exist in the image, a reminder message is issued to remind the user to delete the AR anchor point or update the AR anchor point of the target device.

4. The method according to any one of claims 1 to 3, characterized in that: The displaying of the control menu of the target device comprises: According to the position of the AR anchor point in the image, displaying the control menu of the target device within a preset range of the position; or, In the case that there are multiple AR anchor points, a control menu of a target device corresponding to an AR anchor point closest to the user is displayed according to the distance of each AR anchor point relative to the user.

5. A device control device, applied to a terminal provided with a camera, characterized in that: The device comprises: an acquisition module, configured to acquire an AR anchor point in the image captured by the camera, wherein the AR anchor point is pre-created according to a spatial position of a target device; A control module is configured to determine a target device bound to a spatial position anchored by the AR anchor point; A display module, configured to display a control menu of the target device; an execution module, configured to generate a control instruction in response to an operation on the control menu, and control the target device according to the control instruction; The acquisition module is configured to perform AR anchor point recognition in the image captured by the camera; When an AR anchor point is identified from the image captured by the camera, acquiring anchor point identification information of the AR anchor point; a control module configured to determine a target device bound to a spatial position anchored by the AR anchor point according to the anchor point identification information of the AR anchor point and a correspondence between the pre-stored anchor point identification information and the device identification information; The apparatus further includes an anchor point creation module, wherein the anchor point creation module is configured to obtain a spatial position of a target device through the camera, and obtain a display position of the target device placed in a preview interface of the terminal; Creating an AR anchor point corresponding to the target device according to the spatial position and the display position; generating anchor point identification information according to the AR anchor point, binding the anchor point identification information with the device identification information, and sending the binding relationship to a server for storage, wherein the server also stores a correspondence between the device identification information of the target device and a control menu of the target device; The anchor point creation module is configured to perform spatial feature recognition through the camera to obtain a plurality of spatial feature points; The AR anchor point is created according to a positional relationship between the display position and the plurality of spatial feature points.

6. The device according to claim 5, characterized in that The apparatus further includes a determination module configured to determine whether there is a target device corresponding to the AR anchor point in the image; In a case where the target device corresponding to the AR anchor point does not exist in the image, a reminder message is issued to remind the user to delete the AR anchor point or update the AR anchor point of the target device.

7. The device according to claim 5, characterized in that The execution module is configured to determine whether there is a target device corresponding to the AR anchor point in the image if no control response of the target device is received within a preset time period; In a case where the target device corresponding to the AR anchor point does not exist in the image, a reminder message is issued to remind the user to delete the AR anchor point or update the AR anchor point of the target device.

8. The device according to any one of claims 5 to 7, characterized in that: The display module includes a first display submodule and a second display submodule; The first display submodule is configured to display the control menu of the target device within a preset range of the position according to the position of the AR anchor point in the image; The second display submodule is configured to display a control menu of a target device corresponding to an AR anchor point closest to the user according to a distance of each AR anchor point relative to the user when there are multiple AR anchor points.

9. A device control device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Obtaining an AR anchor point in the image captured by the camera, where the AR anchor point is pre-created based on the spatial position of the target device; Determine a target device bound to the spatial position anchored by the AR anchor point; Displaying a control menu of the target device; In response to an operation on the control menu, a control instruction is generated, and the target device is controlled according to the control instruction; The obtaining of the AR anchor point in the image captured by the camera includes: Performing AR anchor point recognition in the image captured by the camera; When an AR anchor point is identified from the image captured by the camera, acquiring anchor point identification information of the AR anchor point; The determining of a target device bound to a spatial position anchored by the AR anchor point includes: Determine a target device bound to the spatial position anchored by the AR anchor point according to the anchor point identification information of the AR anchor point and the correspondence between the pre-stored anchor point identification information and the target device identification information; The AR anchor is created as follows: Acquire the spatial position of the target device through the camera, and acquire the display position of the image of the target device in the preview interface of the terminal; Creating an AR anchor point corresponding to the target device according to the spatial position and the display position; generating anchor point identification information according to the AR anchor point, binding the anchor point identification information with the device identification information, and sending the binding relationship to a server for storage, wherein the server also stores a correspondence between the device identification information of the target device and a control menu of the target device; The acquiring the spatial position of the target device through the camera includes: Performing spatial feature recognition through the camera to obtain multiple spatial feature points; The creating an AR anchor point corresponding to the target device according to the spatial position and the display position includes: The AR anchor point is created according to a positional relationship between the display position and the plurality of spatial feature points.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the device control method described in any one of claims 1 to 4 are implemented.

Images