Equipment control method and device

Abstract

The embodiments of the present application disclose a device management method and apparatus, wherein the method comprises: obtaining the operation position of the user's target operation on a holographic screen; determining a target device located in the same space as the holographic screen based on the operation position; and managing the target device based on the target operation.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of holographic teaching, and relate to but are not limited to a device control method and apparatus. Background Art

[0002] A holographic screen allows viewers to see the image projected onto the screen by a projector while also seeing the objects behind the holographic screen.

[0003] With the development of holographic screens, users hope to interact with other devices in the same space as the holographic projection image, thereby improving the interactivity of holographic projection devices. However, the existing visualization device management requires selecting the corresponding device on the screen or graphical interface for control, which affects the user experience and interaction effect. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a device control method and apparatus.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a device control method, the method comprising: obtaining an operation position of a user's target operation on a holographic screen; determining a target device located in the same space as the holographic screen based on the operation position; and controlling the target device based on the target operation.

[0007] In a second aspect, an embodiment of the present application provides a device control apparatus, comprising: a first acquisition module, used to obtain an operation position of a user's target operation on a holographic screen; a first determination module, used to determine a target device located in the same space as the holographic screen based on the operation position; and a control module, used to control the target device based on the target operation.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and the processor implements the above method when executing the program.

[0009] In a fourth aspect, an embodiment of the present application provides a storage medium storing executable instructions for causing a processor to execute and implement the above method.

[0010] In the embodiment of the present application, the operation position of the user's target operation on the holographic screen is first obtained; then the target device located in the same space as the holographic screen is determined based on the operation position; finally, the target device is controlled based on the target operation. In this way, the effect of using the holographic screen to control the target device located in the same space as the holographic screen can be achieved, which improves the user experience and interaction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of an implementation flow of a device control method provided in an embodiment of the present application;

[0012] Figure 2 A schematic diagram of an implementation flow of a method for obtaining a device location provided in an embodiment of the present application;

[0013] Figure 3 A spatial schematic diagram for determining the location of a device provided in an embodiment of the present application;

[0014] Figure 4A A schematic diagram of an implementation flow of a method for acquiring a target device provided in an embodiment of the present application;

[0015] Figure 4B A spatial schematic diagram of determining a target device provided in an embodiment of the present application;

[0016] Figure 4C A spatial schematic diagram of adjusting the volume of a speaker provided in an embodiment of the present application;

[0017] Figure 5 A schematic diagram of the structure of a device control device provided in an embodiment of the present application;

[0018] Figure 6 A hardware entity schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the embodiments of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0020] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0023] The present application embodiment provides a device control method, such as Figure 1 As shown, the method includes:

[0024] Step S110, obtaining the operation position of the user's target operation on the holographic screen;

[0025] Here, the holographic screen can be a transparent screen. Taking the holographic screen set in the classroom environment as an example, the teacher can display the teaching content that needs to be displayed on the holographic screen. At the same time, due to the transparent property of the holographic screen, the teacher can also see through the holographic screen and see the equipment in the classroom environment on the other side of the holographic screen and the students watching the holographic screen.

[0026] During the implementation process, the user can touch the holographic screen with fingers or other parts of the body to perform target operations on the holographic screen; or click the holographic screen with a teaching stick or other teaching aids to perform target operations on the holographic screen. Here, there is no limitation on the form of target operations performed by the user.

[0027] When the user operates the holographic screen, the user's operation position on the holographic screen is obtained. For example, when the user clicks the holographic screen with a finger, the position where the finger clicks the holographic screen is obtained; when the user clicks the holographic screen with a teaching rod, the position where the teaching rod clicks the holographic screen is obtained. Here, the operation position can be described by a spatial coordinate system or by a pixel position on the holographic screen.

[0028] Step S120: determining a target device located in the same space as the holographic screen based on the operation position;

[0029] Here, the target device in the same space as the holographic screen can be the device that the user sees through the holographic screen. For example, if the holographic screen is set in a classroom environment, the teacher can see the lights, air conditioners, speakers, seats, other large screens in the classroom, and the terminal devices that students are using through the holographic screen.

[0030] During the implementation, the user can see the device behind the screen through the holographic screen, and can determine the target device by clicking on the holographic screen to see the position of the device, that is, the target device can be determined based on the user's operation position on the holographic screen. For example, a teacher can see other large screens in the classroom through the holographic screen, so the teacher can click on the holographic screen to see the position of the other large screen, and can determine that the target device is the large screen corresponding to the click.

[0031] Step S130: Based on the target operation, control the target device.

[0032] In some embodiments, the target device can be controlled based on the operation of clicking the holographic screen. For example, the holographic screen can be clicked to turn the light on or off; the holographic screen can also be clicked to control the opening or closing of the terminal seen through the holographic screen; the holographic screen can also be clicked to control the students seen through the holographic screen, so that the content created by the students on educational tablets and other devices can be displayed on the holographic screen.

[0033] In some embodiments, when the target device has diverse operations, a single click cannot satisfy all device control scenarios, and the user needs to operate the target device again according to actual needs to achieve different control scenarios for the device. For example, when the brightness of a light needs to be adjusted, or the volume of a speaker needs to be adjusted, the user's adjustment operation needs to be obtained to adjust the brightness of the light or the volume of the speaker.

[0034] In the embodiment of the present application, the operation position of the user's target operation on the holographic screen is first obtained; then the target device located in the same space as the holographic screen is determined based on the operation position; finally, the target device is controlled based on the target operation. In this way, the effect of using the holographic screen to control the target device located in the same space as the holographic screen can be achieved, which improves the user experience and interaction effect.

[0035] In some embodiments, the above step S120 "determining the target device located in the same space as the holographic screen based on the operation position" can be implemented by the following steps:

[0036] Step 121, obtaining the user's sight line position;

[0037] During implementation, a tracking camera device installed behind the user can be used to track the user's position and the direction of the user's head, and collect images to determine the user's line of sight position; a camera device installed on the holographic screen can also be used to collect images of the user's position and the direction of the user's head to determine the user's line of sight position. The line of sight position can be described using coordinate points of a three-dimensional coordinate system.

[0038] Step 122: Match the device position of a device located in the same space as the holographic screen based on the operation position and the sight line position to determine the target device.

[0039] During the implementation process, since the connection between the operation position and the line of sight position can form a ray in space, the ray can point to the device behind the holographic screen, and the ray is matched with the device position of the device located in the same space as the holographic screen, that is, the first device that intersects with the ray is determined to be the target device.

[0040] In the embodiment of the present application, the user's sight line position is first obtained; then, based on the operation position and the sight line position, the device position of the device located in the same space as the holographic screen is matched to determine the target device. In this way, the user's operation position and sight line position can be matched with the device position of the device located in the same space as the holographic screen to effectively determine the target device pointed to by the user.

[0041] In some embodiments, before the above step S110 of "obtaining the operation position of the user's target operation on the holographic screen", the device position of the device located in the same space as the holographic screen is obtained, such as Figure 2 As shown, this can be achieved by following the steps below:

[0042] Step S210, obtaining the sight line positions of the user at at least two different positions;

[0043] Figure 3 A spatial schematic diagram for determining the location of a device provided in an embodiment of the present application, such as Figure 3 As shown, it includes user 31, user 32, holographic screen 33, ray 34, ray 35 and light 36, wherein,

[0044] User 31 and user 32 may be the same user or different users. User 31 and user 32 need to face holographic screen 33 , and user 31 and user 32 are located at different positions.

[0045] During the implementation process, it is necessary to obtain the sight line position 341 of the user 31 and the sight line position 351 of the user 32 .

[0046] Step S220, determining the operation position on the holographic screen corresponding to the user's device pointing to the same holographic screen in the same space at each position;

[0047] like Figure 3 As shown, when both user 31 and user 32 point at light 36 through holographic screen 33 , the operation point of user 31 corresponding to holographic screen 33 is 342 , and the operation point of user 32 corresponding to holographic screen 33 is 352 .

[0048] Step S230, determining a set of mapping relationships based on the sight line position corresponding to each position and the associated operation position;

[0049] like Figure 3 As shown, a ray 34 may be determined based on the sight line position 341 and the associated operation position 342 of the user 31 ; and a ray 35 may be determined based on the sight line position 351 and the associated operation position 352 of the user 32 .

[0050] Step S240: Determine the device location of the device based on at least two groups of the mapping relationships.

[0051] like Figure 3 As shown, based on ray 34 and ray 35, the position 36 of the intersection can be determined as the device position of device (light) 36. In some embodiments, two clicks will cause ray 34 and ray 35 to intersect in space (but not necessarily strictly intersect), and the position of the two rays closest to each other in space can be used as the spatial position 36 of the light.

[0052] During the implementation process, the positions of multiple devices that need to be controlled in the same space as the holographic screen can be determined one by one in advance through the above steps.

[0053] In some embodiments, the user can also click on the holographic screen at multiple different positions to determine the location of the device. For example, in the holographic future classroom, the teacher can model the location of the device by clicking on the same device at multiple positions and angles behind the holographic screen. The location of a device can be determined by averaging multiple clicks to improve the accuracy of the device's location.

[0054] In the embodiment of the present application, firstly, the sight line position of the user at at least two different positions is obtained; then, the operation position on the holographic screen corresponding to the user at each position pointing to the same device located in the same space of the holographic screen is determined; based on the sight line position corresponding to each position and the associated operation position, a set of mapping relationships is determined; finally, the device position of the device is determined based on at least two sets of mapping relationships. In this way, the user only needs to click on the device to be determined through the holographic screen at different positions in advance to effectively determine the position of the device, and the method for determining the position is simple and easy.

[0055] In some embodiments, the above step S210 "obtaining the sight line positions of the user at at least two different positions" can be implemented by the following steps:

[0056] Step 211: Acquire at least two depth images corresponding to the user located at at least two different positions;

[0057] In some embodiments, a time of flight (TOF) camera can be used to obtain a depth image, that is, by emitting continuous near-infrared pulses to the target scene, and then using a sensor to receive the light pulses reflected by the object. By comparing the phase difference between the emitted light pulse and the light pulse reflected by the object, the transmission delay between the light pulses can be calculated and the distance of the object relative to the transmitter can be obtained, and finally a depth image can be obtained.

[0058] You can also use Kinect, a somatosensory device that combines color images and depth images, to collect depth images.

[0059] Step 212: Determine the sight line positions of the user at at least two different positions based on the at least two depth images.

[0060] Here, since the depth image includes distance information of objects in the image, the user's line of sight position can be determined based on the depth image.

[0061] In some embodiments, a camera for collecting depth images may be set to collect depth images at two different positions, and multiple cameras for collecting depth images may be set to collect depth images to improve the accuracy of using depth images to determine the user's line of sight position.

[0062] In the embodiment of the present application, at least two depth images corresponding to the user at at least two different positions are first collected; then the sight line position of the user at at least two different positions is determined based on the at least two depth images. In this way, the sight line position of the user can be effectively determined based on the depth image.

[0063] In some embodiments, Figure 4A As shown, the above step S120 "determining the target device located in the same space as the holographic screen based on the operation position" can be implemented by the following steps:

[0064] Step S123, obtaining the user's sight line position;

[0065] Figure 4B A spatial schematic diagram of determining a target device provided in an embodiment of the present application, such as Figure 4B As shown, it includes a user 41, a user's sight position 42, a position 43 of the light pointed by the user on the holographic screen, a position 44 of the actual light, and a preset range 45 determined based on the position 43 of the light pointed by the user on the holographic screen.

[0066] In some embodiments, due to the different sight angles of the user, it can be determined that the image the user sees on the holographic screen is different, so it is necessary to first obtain the user's sight position.

[0067] Step S124: based on the sight line position of the user, collecting an operation image within a preset range determined based on the operation position;

[0068] During implementation, based on the user's line of sight, that is, the camera device can be used to track the direction of the user's head, and collect operation images within a preset range determined based on the operation position.

[0069] In some embodiments, a preset range may be set based on the size of the device, and operation images within the preset range determined based on the operation position may be collected. Figure 4B An operation image of the preset range 45 is shown.

[0070] Step S125: Identify the target device in the operation image.

[0071] Here, since the operation image is determined based on the operation position acquired based on the user's line of sight position, the target device exists in the operation image, and the target device can be identified by using an image recognition method.

[0072] In the embodiment of the present application, the user's sight line position is first obtained; then, based on the user's sight line position, an operation image within a preset range determined based on the operation position is collected; and finally, the target device is identified in the operation image. In this way, an operation image including the target device can be collected, and then the target device can be effectively identified based on the operation image.

[0073] In some embodiments, the target operation includes a drag operation, and the target device includes a terminal device with a screen; the above step S130 "controlling the target device based on the target operation" can be implemented by the following steps:

[0074] Step 131: Based on the drag operation, present the display content of the terminal device on the holographic screen; or,

[0075] During the implementation process, the display content of the terminal can be presented on the holographic screen by dragging or other recognizable operations. Taking the holographic screen set up in a classroom environment as an example, the teacher can drag any terminal device that a student is using to make the display content of the terminal device appear on the holographic screen.

[0076] Step 132: Based on the dragging operation, present the display content on the holographic screen on the screen of the terminal device.

[0077] During the implementation process, the display content on the holographic screen can be presented on the screen of the terminal device by dragging or other recognizable operations. Taking the holographic screen set in the classroom environment as an example, the teacher can drag the display content on the holographic screen to any terminal device being used by the student, so that the display content of the holographic screen is presented on the terminal device.

[0078] In some embodiments, by dragging a document, picture or video displayed on a holographic screen, the dragged document, picture or video can be sent to a terminal device.

[0079] In the embodiment of the present application, based on the drag operation, the display content of the terminal device is presented on the holographic screen; or, based on the drag operation, the display content on the holographic screen is presented on the screen of the terminal device. In this way, the interactive display of the holographic screen and the terminal device content can be realized, which improves the user experience and interactive effect.

[0080] In some embodiments, the above step S130 "controlling the target device based on the target operation" can be implemented by the following steps:

[0081] Step 133: displaying a function icon corresponding to the target device on the holographic screen based on the target device;

[0082] Figure 4C A spatial schematic diagram of adjusting the volume of a speaker provided in an embodiment of the present application, such as Figure 4C As shown, it includes a user 41, a position 46 where the user points to the speaker on the holographic screen, a position 47 of the actual speaker, and a function icon 48 on the holographic screen for adjusting the volume.

[0083] like Figure 4C As shown, based on the device that the user is pointing to being a speaker 46, a function icon 48 for adjusting the volume corresponding to the speaker can be displayed on the holographic screen.

[0084] During the implementation process, corresponding function icons can be displayed based on the functions of the target device. For example, when the target device is a lamp, an icon for adjusting the brightness of the lamp can be displayed; when the target device is an air conditioner, an icon for adjusting the temperature or air volume of the air conditioner can be displayed.

[0085] Step 134: obtaining a function operation performed by the user based on the function icon;

[0086] During the implementation process, users can perform corresponding function operations based on the displayed function icons. Figure 4C As shown, the user can adjust the speaker volume based on the function icon 48.

[0087] Step 135: Based on the functional operation, control the target device.

[0088] During the implementation process, this function can be used to control the target device. Figure 4C As shown, an operation of adjusting the volume of the speaker based on the function icon 48 is acquired, and the volume of the speaker can be adjusted based on this operation.

[0089] In the embodiment of the present application, the function icon corresponding to the target device is first displayed on the holographic screen based on the target device; then the function operation of the user based on the function icon is obtained; finally, the target device is controlled based on the function operation. In this way, the function icon corresponding to the target device can be displayed on the holographic screen to realize the function operation based on the function icon, control the target device, and meet the user's diversified operation needs for the target device.

[0090] In some embodiments, the above step S130 "controlling the target device based on the target operation" can be implemented by the following steps:

[0091] Step 136: Acquire the user's gesture operation on the target device;

[0092] In some embodiments, acquiring a gesture operation may include acquiring a gesture posture of a user in the air. For example, the user may wave his hand up or down in the air.

[0093] Step 137: Based on the gesture operation, control the target device.

[0094] In some embodiments, the target device can be controlled based on the user swinging his hand up or down in the air. For example, the user can turn down the volume of the speaker by swinging his hand down; the user can turn up the volume of the speaker by swinging his hand up.

[0095] In the embodiment of the present application, the user's gesture operation on the target device is first obtained, and then the target device is controlled based on the gesture operation. In this way, the target device can be controlled based on the user's gesture operation.

[0096] In some embodiments, the above step S130 "controlling the target device based on the target operation" can be implemented by the following steps:

[0097] Step 138: Connect the target device to an IoT control function corresponding to the target device;

[0098] Here, the target device can be connected to the Internet of Things (IOT) control function of its background to achieve control of the target device based on the Internet of Things.

[0099] Step 139: Based on the target operation, the target device is controlled by utilizing the IoT control function corresponding to the target device.

[0100] Here, the target operation can also interact with the IoT control function of the device to achieve control of the target device using the IoT control function based on the target operation.

[0101] In the embodiment of the present application, the target device is first connected to the IoT control function corresponding to the target device; then, based on the target operation, the target device is controlled by using the IoT control function corresponding to the target device. In this way, the target operation can interact with the IoT control function to effectively control the target device.

[0102] Based on the foregoing embodiments, an embodiment of the present application provides a device control device, which includes the modules included, each module includes sub-modules, and can be implemented by a processor in an electronic device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0103] Figure 5 A schematic diagram of the structure of the device control device provided in the embodiment of the present application is shown in FIG. Figure 5 As shown, the device 500 includes:

[0104] A first acquisition module 510 is used to acquire the operation position of the user's target operation on the holographic screen;

[0105] A first determining module 520, configured to determine a target device located in the same space as the holographic screen based on the operation position;

[0106] The control module 530 is used to control the target device based on the target operation.

[0107] In some embodiments, the first determination module includes a first acquisition submodule and a matching submodule, wherein the first acquisition submodule is used to acquire the user's line of sight position; the matching submodule is used to match the device position of a device located in the same space as the holographic screen based on the operation position and the line of sight position to determine the target device.

[0108] In some embodiments, the device also includes an acquisition module, a second determination module, a third determination module and a fourth determination module, wherein the acquisition module is used to acquire the line of sight position of the user at at least two different positions; the second determination module is used to determine the operation position of the user on the holographic screen corresponding to the device pointing to the same holographic screen in the same space at each position; the third determination module is used to determine a set of mapping relationships based on the line of sight position corresponding to each position and the associated operation position; the fourth determination module is used to determine the device position of the device based on at least two sets of mapping relationships.

[0109] In some embodiments, the acquisition module includes a collection submodule and a fourth determination submodule, wherein the collection submodule is used to collect at least two depth images corresponding to the user located at at least two different positions; and the fourth determination submodule is used to determine the line of sight position of the user located at at least two different positions based on the at least two depth images.

[0110] In some embodiments, the first determination module includes an acquisition unit, a determination unit and an identification unit, wherein the acquisition unit is used to acquire the user's line of sight position; the determination unit is used to collect an operation image within a preset range determined based on the operation position based on the user's line of sight position; and the identification unit is used to identify the target device in the operation image.

[0111] In some embodiments, the target operation includes a drag operation, and the target device includes a terminal device with a screen; the management and control module includes a first presentation submodule and a second presentation submodule, wherein the first presentation submodule is used to present the display content of the terminal device on the holographic screen based on the drag operation; and the second presentation submodule is used to present the display content on the holographic screen on the screen of the terminal device based on the drag operation.

[0112] In some embodiments, the management and control module includes a display submodule, a second acquisition submodule and a first management and control submodule, wherein the display submodule is used to display a function icon corresponding to the target device on the holographic screen based on the target device; the second acquisition submodule is used to obtain the user's function operation based on the function icon; the first management and control submodule is used to control the target device based on the function operation.

[0113] In some embodiments, the control module includes a third acquisition submodule and a second control submodule, wherein the third acquisition submodule is used to acquire the user's gesture operation on the target device; and the second control submodule is used to control the target device based on the gesture operation.

[0114] In some embodiments, the management and control module includes a connection submodule and a third management and control submodule, wherein the connection submodule is used to connect the target device with the Internet of Things control function corresponding to the target device; and the third management and control submodule is used to control the target device based on the target operation using the Internet of Things control function corresponding to the target device.

[0115] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0116] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable an electronic device (which can be a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0117] Correspondingly, an embodiment of the present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the device management and control method provided in the above embodiment are implemented.

[0118] Correspondingly, an embodiment of the present application provides an electronic device, Figure 6 A hardware entity diagram of an electronic device provided in an embodiment of the present application, such as Figure 6 As shown, the hardware entity of the device 600 includes: a memory 601 and a processor 602, wherein the memory 601 stores a computer program that can be run on the processor 602, and when the processor 602 executes the program, the steps in the device management and control method provided in the above embodiment are implemented.

[0119] The memory 601 is configured to store instructions and applications executable by the processor 602, and can also cache data to be processed or processed by the processor 602 and various modules in the electronic device 600 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0120] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0121] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0122] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0123] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0124] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0125] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0126] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0127] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable an electronic device (which can be a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0128] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0129] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0130] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0131] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A device management and control method, the method comprising: Acquire the operation position of the user's target operation on the holographic screen, wherein the target operation includes a drag operation; Determine a target device located in the same space as the holographic screen based on the operation position; the target device includes a terminal device with a screen; Based on the target operation, controlling the target device includes: Based on the dragging operation, presenting the display content of the terminal device on the holographic screen; or, Based on the dragging operation, the display content on the holographic screen is presented on the screen of the terminal device.

2. The method of claim 1, wherein determining a target device located in the same space as the holographic screen based on the operation position comprises: Get the user's sight position; The target device is determined based on the operation position, the sight line position and the device position of the device located in the same space as the holographic screen.

3. The method according to claim 2, before acquiring the operation position of the user's target operation on the holographic screen, the method further comprises: Acquiring sight line positions of the user at at least two different positions; Determine the operation position on the holographic screen corresponding to the user's operation position on the holographic screen when the user points to a device located in the same space with the holographic screen at each position; Determine a set of mapping relationships based on the sight line position corresponding to each position and the associated operation position; Based on at least two sets of the mapping relationships, a device position of the device is determined.

4. The method according to claim 3, wherein obtaining the sight line positions of the user at at least two different positions comprises: Acquire at least two depth images corresponding to the user located at at least two different positions; The sight line positions of the user at at least two different positions are determined based on the at least two depth images.

5. The method of claim 1, wherein determining a target device located in the same space as the holographic screen based on the operation position comprises: Get the user's sight position; Based on the sight line position of the user, collecting an operation image within a preset range determined based on the operation position; The target device is identified in the operation image.

6. The method according to any one of claims 1 to 5, wherein the step of controlling the target device based on the target operation comprises: Based on the target device, displaying a function icon corresponding to the target device on the holographic screen; Obtaining a function operation of the user based on the function icon; Based on the functional operation, the target device is managed and controlled.

7. The method according to any one of claims 1 to 5, wherein the step of controlling the target device based on the target operation comprises: Acquire a user's gesture operation on the target device; Based on the gesture operation, the target device is controlled.

8. The method according to any one of claims 1 to 5, wherein the controlling the target device based on the target operation comprises: Connecting the target device to an IoT control function corresponding to the target device; Based on the target operation, the target device is controlled by utilizing an Internet of Things control function corresponding to the target device.

9. A device control apparatus, comprising: A first acquisition module is used to acquire an operation position of a user's target operation on the holographic screen, wherein the target operation includes a drag operation; A first determination module is used to determine a target device located in the same space as the holographic screen based on the operation position; the target device includes a terminal device with a screen; A management and control module is used to manage the target device based on the target operation, including: based on the drag operation, presenting the display content of the terminal device on the holographic screen; or, based on the drag operation, presenting the display content on the holographic screen on the screen of the terminal device.

Images