Method, device and storage medium for sending control instruction and electronic device

Abstract

The application discloses a control instruction sending method and device, a storage medium and an electronic equipment, relates to the technical field of smart homes, and the control instruction sending method comprises the following steps: in the case that a first device receives a gesture instruction of a target object, a three-dimensional coordinate system is established according to the position relationship between the first device and the target object; the included angle information between the gesture direction corresponding to the gesture instruction and the vertical axis of the three-dimensional coordinate system is determined, and a first area is determined according to the included angle information; the device information of a second device in the first area is determined, and a control instruction is sent to the second device according to the device information. By adopting the technical scheme, the problems that in the related art, when a user does not accurately describe a target position area, a device in the target position area cannot be accurately controlled and the like are solved.

Description

Technical Field

[0001] This application relates to the field of smart home technology, and more specifically, to a method, apparatus, storage medium, and electronic device for transmitting control commands. Background Technology

[0002] With the advancement of science and technology and the development of artificial intelligence, intelligent algorithms are increasingly being applied to daily life. Their intelligent development is of paramount importance, and the most critical issue in intelligentization lies in intelligently addressing users' urgent needs.

[0003] As living standards improve and users' needs become more diverse and sophisticated, their homes are filled with a variety of household items. When users need to control devices in a target area, they cannot accurately describe the area, thus making it impossible to accurately control the devices in that area, resulting in a very poor smart device control experience.

[0004] There is currently no effective solution to the problem that the device cannot be accurately controlled in the target location area when the user has not accurately described the target location area. Summary of the Invention

[0005] This application provides a method, apparatus, storage medium, and electronic device for sending control commands, to at least solve the problem in the related art that the device cannot be accurately controlled when the user has not accurately described the target location area.

[0006] According to one embodiment of this application, a method for sending control commands is provided, comprising: when a first device receives a gesture command from a target object, establishing a three-dimensional coordinate system based on the positional relationship between the first device and the target object; determining the angle information between the gesture direction corresponding to the gesture command and the vertical axis of the three-dimensional coordinate system, and determining a first region based on the angle information; determining device information of a second device in the first region, and sending a control command to the second device based on the device information.

[0007] In an exemplary embodiment, determining the angle information between the gesture pointing corresponding to the gesture instruction and the vertical axis of the three-dimensional coordinate system includes: determining the first spatial coordinate and the second spatial coordinate of the upper limb of the target object in the three-dimensional coordinate system; determining the spatial vector corresponding to the gesture pointing based on the first spatial coordinate and the second spatial coordinate; and determining the angle information between the gesture pointing and the vertical axis based on the spatial vector.

[0008] In an exemplary embodiment, determining a first region based on the included angle information includes: determining the third spatial coordinates of the intersection point of the extension line of the gesture and the plane where the ground is located based on the included angle information; determining a region of a preset size based on the third spatial coordinates, and using the region as the first region, wherein the region includes the position corresponding to the third spatial coordinates.

[0009] In an exemplary embodiment, establishing a three-dimensional coordinate system based on the positional relationship between the first device and the target object includes: determining the center position of the first device as the origin of the three-dimensional coordinate system; determining the positional direction of the target object relative to the first device, and defining the positional direction as the horizontal axis direction of the three-dimensional coordinate system; determining the visual rightward extension direction of the first device based on the horizontal axis direction, and defining the visual rightward extension direction as the vertical axis direction of the three-dimensional coordinate system; and determining the visual upward extension direction of the first device based on the horizontal axis direction, and defining the visual upward extension direction as the vertical axis direction of the three-dimensional coordinate system; and establishing the three-dimensional coordinate system based on the origin, the horizontal axis direction, the vertical axis direction, and the vertical axis direction.

[0010] In an exemplary embodiment, sending a control command to the second device based on the device information includes: acquiring environmental information of the first area and acquiring the current state of the second device based on the device information; determining a control strategy corresponding to the second device based on the environmental information, wherein the control strategy is used to adjust the second device to a target state; and generating the control command based on the control strategy and sending the control command to the second device when the target state is inconsistent with the current state.

[0011] In an exemplary embodiment, after determining the first region based on the included angle information, the method further includes: determining whether the gesture instruction is used to instruct control of the first device; if the gesture instruction is determined to be used to instruct control of the first device, controlling the first device to move to the first region; and if the gesture instruction is not used to instruct control of the first device, determining device information of a second device in the first region.

[0012] In an exemplary embodiment, controlling the first device to move to the first area includes: obtaining map parameters corresponding to the second area, wherein the map parameters include: a correspondence between the second area and a third device, wherein the first area is a sub-area of ​​the second area, and the third device includes at least: the first device and the second device; determining the third spatial coordinates of the third device in the three-dimensional coordinate system; determining a movement route for the first device to move to the spatial position corresponding to the second device based on the correspondence and the third spatial coordinates, and controlling the first device to move to the first area according to the movement route.

[0013] In an exemplary embodiment, before sending a control command to the second device based on the device information, the method further includes: if the control command is determined based on the device information, broadcasting voice information carrying the device information and the control command; receiving response information corresponding to the voice information, and determining whether to send a control command to the second device based on the response information; if it is determined not to send a control command to the second device, adjusting the control command based on the response information.

[0014] According to another embodiment of the present application, a control command sending device is also provided, comprising: an establishment module, configured to establish a three-dimensional coordinate system based on the positional relationship between the first device and the target object when the first device receives a gesture command from the target object; a first determination module, configured to determine the angle information between the gesture direction corresponding to the gesture command and the vertical axis of the three-dimensional coordinate system, and determine a first region based on the angle information; and a sending module, configured to determine the device information of a second device in the first region, and send a control command to the second device based on the device information.

[0015] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the above-described method for sending control instructions when it is run.

[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-described method for sending control instructions through the computer program.

[0017] In this embodiment, when the first device receives a gesture instruction from the target object, a three-dimensional coordinate system is established based on the positional relationship between the first device and the target object; the angle information between the gesture pointing to the gesture instruction and the vertical axis of the three-dimensional coordinate system is determined, and a first region is determined based on the angle information; the device information of the second device in the first region is determined, and a control instruction is sent to the second device based on the device information; in this embodiment, there is no need for the user to describe the target location region; simply by pointing to the target location region with a gesture, the corresponding device can be controlled based on the device information of the target location region. That is, by adopting the above technical solution, the problem of not being able to accurately control the device in the target location region when the user has not accurately described the target location region is solved, and the effect of accurately controlling the device in the target location region is achieved. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the hardware environment for a method of sending control commands according to an embodiment of this application;

[0021] Figure 2 This is a flowchart of a method for sending control commands according to an embodiment of this application;

[0022] Figure 3 This is a timing diagram of a control command transmission method according to an embodiment of this application;

[0023] Figure 4 This is a structural block diagram (a) of a control command sending device according to an embodiment of this application;

[0024] Figure 5 This is a structural block diagram (II) of a control command sending device according to an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] According to one aspect of the embodiments of this application, a method for sending control commands is provided. This method for sending control commands is widely applicable to whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligence house ecosystems. Optionally, in this embodiment, the above-mentioned method for sending control commands can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0028] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0029] This embodiment provides a method for sending control commands, applied to a terminal device. Figure 2 This is a flowchart of a method for sending control commands according to an embodiment of this application. The process includes the following steps:

[0030] Step S202: When the first device receives a gesture command from the target object, a three-dimensional coordinate system is established based on the positional relationship between the first device and the target object;

[0031] It should be noted that the above-mentioned gesture commands are used to control the first device to move to the first area and to control the first device to perform a target operation in the first area. They can also be used to control the first device to send control commands to a second device in the first area. The specific control commands for the gesture commands are not limited in this embodiment of the invention.

[0032] Optionally, taking a robotic vacuum cleaner as an example, when a gesture command is used to control the first device to move to a first area and to control the first device to perform a target operation in the first area, the robotic vacuum cleaner determines the first area according to the finger command and performs a cleaning operation when it moves to the first area.

[0033] Optionally, taking a smart speaker as an example, when a gesture command is used to control the first device to send a control command to a second device in the first area, the smart speaker sends the control command to the second device according to the device information of the second device in the first area.

[0034] Step S204: Determine the angle information between the gesture direction corresponding to the gesture command and the vertical axis of the three-dimensional coordinate system, and determine the first region based on the angle information;

[0035] Step S206: Determine the device information of the second device in the first area, and send a control command to the second device according to the device information.

[0036] It should be noted that the above-mentioned device information includes, but is not limited to, device status information; the number of second devices can be one or more, and this embodiment of the invention does not limit this.

[0037] Optionally, the second device may not be limited to smart refrigerators, smart ovens, smart stoves, smart washing machines, smart water heaters, smart dishwashers, smart projectors, smart TVs, smart clothes racks, smart curtains, smart audio-visual equipment, etc.

[0038] Through the above steps, when the first device receives a gesture command from the target object, a three-dimensional coordinate system is established based on the positional relationship between the first device and the target object; the angle information between the gesture pointing to the gesture command and the vertical axis of the three-dimensional coordinate system is determined, and a first region is determined based on the angle information; the device information of the second device in the first region is determined, and a control command is sent to the second device based on the device information; in this embodiment, there is no need for the user to describe the target location region; simply by pointing a gesture to the target location region, the corresponding device can be controlled based on the device information of the target location region. That is, by adopting the above technical solution, the problem of not being able to accurately control the device in the target location region when the user has not accurately described the target location region is solved, and the effect of accurately controlling the device in the target location region is achieved.

[0039] There are many ways to implement the above step S204. This embodiment of the invention provides one implementation method: In an exemplary embodiment, the first spatial coordinate and the second spatial coordinate of the upper limb of the target object in the three-dimensional coordinate system are determined; the spatial vector corresponding to the gesture direction is determined according to the first spatial coordinate and the second spatial coordinate; and the angle information between the gesture direction and the vertical axis is determined according to the spatial vector.

[0040] In other words, the spatial vector of the gesture direction is determined based on the spatial coordinates of the upper limb of the target object, and then the angle information between the gesture direction and the vertical axis is determined based on the spatial vector and the vector corresponding to the vertical axis.

[0041] Optionally, determining the first and second spatial coordinates of the upper limb of the target object in the three-dimensional coordinate system includes one of the following: determining the first spatial coordinates of the target object's shoulder in the three-dimensional coordinate system and the second spatial coordinates of the target object's fingers in the three-dimensional coordinate system; determining the first spatial coordinates of the target object's elbow in the three-dimensional coordinate system and the second spatial coordinates of the target object's fingers in the three-dimensional coordinate system; determining the first spatial coordinates of the target object's shoulder in the three-dimensional coordinate system and the second spatial coordinates of the target object's wrist in the three-dimensional coordinate system; determining the first spatial coordinates of the target object's elbow in the three-dimensional coordinate system and the second spatial coordinates of the target object's wrist in the three-dimensional coordinate system.

[0042] Optionally, determining the angle information between the gesture direction and the vertical axis based on the spatial vector includes: Where (x1, y1, z1) is the spatial vector pointing to the gesture, and α is the included angle information.

[0043] There are many ways to implement the above step S206. This embodiment of the invention provides one implementation method: In an exemplary embodiment, the third spatial coordinates of the intersection point of the extension line of the gesture and the plane where the ground is located are determined according to the included angle information; a region of a preset area size is determined according to the third spatial coordinates, and the region is used as the first region, wherein the region includes the position corresponding to the third spatial coordinates.

[0044] It should be noted that by determining the extension of the straight line where the gesture is pointing, and by determining the intersection of the extension line with the ground, the area pointed to by the target object can be determined.

[0045] Optionally, determining the area of ​​a preset size based on the third spatial coordinates includes: determining the area of ​​a preset size with the third spatial coordinates as the midpoint, wherein the shape of the area can be: a circle, a square, a rectangle, etc., and this embodiment of the invention does not limit this.

[0046] In an exemplary embodiment, step S202 described above can be implemented through the following steps: determining the center position of the first device as the origin of the three-dimensional coordinate system; determining the positional direction of the target object relative to the first device, and defining the positional direction as the horizontal axis direction of the three-dimensional coordinate system; determining the visual rightward extension direction of the first device based on the horizontal axis direction, and defining the visual rightward extension direction as the vertical axis direction of the three-dimensional coordinate system; determining the visual upward extension direction of the first device based on the horizontal axis direction, and defining the visual upward extension direction as the vertical axis direction of the three-dimensional coordinate system; and establishing the three-dimensional coordinate system based on the origin, the horizontal axis direction, the vertical axis direction, and the vertical axis direction.

[0047] It should be noted that the visual extension direction to the right can be understood as: the horizontal axis direction rotated 90 degrees to the right based on the xy plane; the visual extension direction to the upward can be understood as: the horizontal axis direction rotated 90 degrees upward based on the xz plane.

[0048] Optionally, step S202 above can be implemented by the following steps: determining the center position of the target object on the ground as the origin of the three-dimensional coordinate system; determining the position direction of the first device relative to the target object, and defining the position direction as the horizontal axis direction of the three-dimensional coordinate system; determining the visual rightward extension direction of the first device based on the horizontal axis direction, and defining the visual rightward extension direction as the vertical axis direction of the three-dimensional coordinate system; and determining the visual upward extension direction of the first device based on the horizontal axis direction, and defining the visual upward extension direction as the vertical axis direction of the three-dimensional coordinate system; and establishing the three-dimensional coordinate system based on the origin, the horizontal axis direction, the vertical axis direction, and the vertical axis direction.

[0049] In an exemplary embodiment, sending a control command to the second device based on the device information includes: acquiring environmental information of the first area and acquiring the current state of the second device based on the device information; determining a control strategy corresponding to the second device based on the environmental information, wherein the control strategy is used to adjust the second device to a target state; and generating the control command based on the control strategy and sending the control command to the second device when the target state is inconsistent with the current state.

[0050] For example, information such as temperature, humidity, and air quality in the first area is obtained. If the temperature does not match the standard temperature, and / or the humidity does not match the standard humidity, and / or the air quality does not match the standard air quality, the device information of the second device existing in the first area is determined. If only an air conditioner exists in the first area and the air conditioner is in the off state, an instruction to turn on the air conditioner and an instruction to turn on the mode corresponding to the temperature are generated, and the generated instructions are sent to the air conditioner.

[0051] In an exemplary embodiment, after determining the first region based on the included angle information, the method further includes: determining whether the gesture instruction is used to instruct control of the first device; if the gesture instruction is determined to be used to instruct control of the first device, controlling the first device to move to the first region; and if the gesture instruction is not used to instruct control of the first device, determining device information of a second device in the first region.

[0052] It should be noted that when the first device is a mobile device (e.g., a robot vacuum cleaner or a mobile robot), the gesture commands of the target object can be used to instruct the first device to move to the target area and perform the target operation in the target area.

[0053] Optionally, determining whether the gesture instruction is used to instruct control of the first device includes at least one of the following: sending voice information to the target object to determine whether the gesture instruction is used to instruct control of the first device based on the response information of the target object; determining whether the gesture instruction is an instruction for turning on or off the functional mode of the first device, and if the gesture instruction is determined to be an instruction for turning on or off the functional mode of the first device, determining that the gesture instruction is used to instruct control of the first device.

[0054] For example, if the robot vacuum recognizes a gesture command used to instruct the robot vacuum to perform a cleaning operation, then the gesture command is used to control the robot vacuum.

[0055] Optionally, determining the device information of the second device in the first region includes: obtaining map parameters corresponding to the second region, wherein the map parameters include: the correspondence between the second region and the third device; determining the second device in the first region and the device information of the second device based on the correspondence.

[0056] In an exemplary embodiment, controlling the first device to move to the first area includes: obtaining map parameters corresponding to the second area, wherein the map parameters include: a correspondence between the second area and a third device, wherein the first area is a sub-area of ​​the second area, and the third device includes at least: the first device and the second device; determining the third spatial coordinates of the third device in the three-dimensional coordinate system; determining a movement route for the first device to move to the spatial position corresponding to the second device based on the correspondence and the third spatial coordinates, and controlling the first device to move to the first area according to the movement route.

[0057] Optionally, determining the movement route of the first device to the spatial position corresponding to the second device based on the correspondence and the third spatial coordinates includes: determining a straight line route for the first device to move to the spatial position corresponding to the second device; determining whether the third device exists on the straight line route; if the third device exists on the straight line route, adjusting the straight line route into a broken line route based on the third spatial coordinates, and using the adjusted broken line route as the movement route, wherein the broken line route is a route that does not include the third coordinate position.

[0058] In an exemplary embodiment, before sending a control command to the second device based on the device information, the method further includes: if the control command is determined based on the device information, broadcasting voice information carrying the device information and the control command; receiving response information corresponding to the voice information, and determining whether to send a control command to the second device based on the response information; if it is determined not to send a control command to the second device, adjusting the control command based on the response information.

[0059] Optionally, sending a control command to the second device based on the device information includes: if the control command is determined based on the device information, broadcasting voice information carrying the device information and the control command; and if a response information corresponding to the voice information is received, and the response information indicates that the control command should be sent to the second device, then sending the control command to the second device.

[0060] Optionally, adjusting the control command based on the response information includes: adjusting the receiving object of the control command to the third device when the response information indicates that the control command is sent to the third device; and adjusting the control command to another control command when the response information indicates that the control command is sent to the second device.

[0061] To better understand the process of sending the above control commands, the implementation flow of sending the above control commands will be described below in conjunction with optional embodiments, but this is not intended to limit the technical solutions of the embodiments of this application.

[0062] This embodiment provides a control command sending system, the main components of which include:

[0063] 1. Voice Module: Installed on the device for voice interaction. It has built-in voice middleware for communication and data transmission with the cloud server.

[0064] 2. Cloud server: Provides complete services including data collection, data analysis, data storage, data security, algorithm training, and model training.

[0065] 3. Camera: Installed on the device for image scanning. It has built-in image recognition middleware that can upload image recognition data to a cloud server.

[0066] 4. Main control board: The core control component of the device, which communicates with the voice module and camera via serial port, and integrates cloud service middleware, enabling communication and data transmission with the cloud server.

[0067] This embodiment provides a method for sending control commands, implemented through the aforementioned control command sending system. Figure 3 This is a timing diagram of a control command transmission method according to an embodiment of this application, such as... Figure 3 As shown, the specific steps are as follows:

[0068] Step S301: The user binds the device within the APP;

[0069] Step S302: The APP uploads the user and device binding relationship (user ID, device ID) to the cloud server;

[0070] Step S303: The cloud server sends the user and device binding relationship to the device's voice module;

[0071] Step S304: The voice module synchronizes the binding relationship between the user and the device to the device's main control board;

[0072] Step S305: The main control board of the device starts the image scanning operation;

[0073] Step S306: The device's main control board enables the camera on the device.

[0074] Step S307: The camera on the device performs an image scanning operation, obtains indoor map parameters through SLAM technology, and obtains indoor object parameters through image recognition technology;

[0075] Step S308: The camera uploads map parameters and item parameters to the cloud server;

[0076] Step S309: The cloud server uses SLAM technology to build an indoor floor plan map and uses image recognition technology to identify indoor item categories;

[0077] Step S310: The cloud server sends the floor plan map and item category identification information to the device's main control board;

[0078] Step S311: The device's camera recognizes the user's hand gesture;

[0079] Step S312: The camera on the device accesses the floor plan map and item category recognition information stored on the main control board;

[0080] Step S313: Identify the location of the device and the straight-line distance between the device and the user. Construct a three-dimensional coordinate system with the horizontal line of the camera as the Y-axis, the straight-line distance between the user and the camera as the X-axis, and the vertical distance of the user's finger as the Z-axis. Calculate the ground target area corresponding to the user's gesture by calculating the angle between the user's gesture and the Z-axis.

[0081] Step S314: Design a route to move to the target area;

[0082] Based on the object recognition results, the device determines the shape and height of the object between the device's current starting position and the target area. The device then chooses to either detour or pass directly to continue to the target area.

[0083] Step S315: Push the target region recognition result to the voice module;

[0084] Step S316: The voice module combines the location of the target area with the control functions of the devices in the target area to generate a device control intent judgment;

[0085] Step S317: Announce the device control intent to the user;

[0086] Step S318: After hearing the announcement of the target area location and device control intent, the user confirms whether the target area location is correct and whether the device control intent is valid.

[0087] Step S319: If the device control intent is valid, the voice module sends the target area location and the device control intent to the main control board, and the main control board controls the device to go to the target area location to complete the device control operation;

[0088] Specifically: If the target area location is incorrect but the device control intent is valid: the user can verbally specify the target area range, and the voice module will send the new target area location to the main control board. The main control board, combining the new target area location with the device's executable control functions, will control the device to move to the target area location to complete the device control operation. If the target area location is correct and the device control intent is valid: the voice module will send the target area location and the device control intent to the main control board, and the main control board will control the device to move to the target area location to complete the device control operation.

[0089] Step S320: The main control board of the equipment executes equipment control operations;

[0090] Step S321: Return the device control results to the voice module;

[0091] Step S322: The voice module plays the device control execution result to the user;

[0092] Step S323: The voice module synchronizes the device control execution results to the cloud server;

[0093] Step S324: The cloud server pushes the device control execution result to the APP.

[0094] Step S325: The APP generates and displays the device control execution record;

[0095] Step S326: If the device control intent is invalid, the voice module uploads the device control intent to the cloud server and incorporates it into the training set of the device control intent algorithm, providing negative data for subsequent optimization of the device control intent algorithm; if the device control intent is valid, the voice module uploads the target area location to the cloud server and incorporates it into the training set of the target area location algorithm, providing positive data for subsequent optimization of the target area location algorithm.

[0096] This invention provides image recognition algorithms for objects, image recognition algorithms for indoor layouts and complex environments, user gesture recognition algorithms, location calculation algorithms, and device control intent models. These solutions address the problem in related technologies where devices cannot be accurately controlled in the target location area when the user has not accurately described the target location area, thus achieving the effect of accurately controlling devices in the target location area.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0098] Figure 4 This is a structural block diagram (a) of a control command transmitting device according to an embodiment of this application; as shown... Figure 4 As shown, it includes:

[0099] Module 42 is used to establish a three-dimensional coordinate system based on the positional relationship between the first device and the target object when the first device receives a gesture command from the target object.

[0100] The first determining module 44 is used to determine the angle information between the gesture direction corresponding to the gesture instruction and the vertical axis of the three-dimensional coordinate system, and to determine the first region based on the angle information;

[0101] The sending module 46 is used to determine the device information of the second device in the first area and send control commands to the second device according to the device information.

[0102] With the above-described device, when the first device receives a gesture command from the target object, a three-dimensional coordinate system is established based on the positional relationship between the first device and the target object; the angle information between the gesture direction corresponding to the gesture command and the vertical axis of the three-dimensional coordinate system is determined, and a first region is determined based on the angle information; the device information of the second device in the first region is determined, and a control command is sent to the second device based on the device information. This solves the problem in related technologies where the device in the target location area cannot be accurately controlled when the user has not accurately described the target location area, thus achieving the effect of accurately controlling the device in the target location area.

[0103] In an exemplary embodiment, the first determining module 44 is configured to determine the first spatial coordinates and the second spatial coordinates of the upper limb of the target object in the three-dimensional coordinate system; determine the spatial vector corresponding to the gesture pointing according to the first spatial coordinates and the second spatial coordinates; and determine the angle information between the gesture pointing and the vertical axis according to the spatial vector.

[0104] In an exemplary embodiment, the first determining module 44 is configured to determine the third spatial coordinates of the intersection point of the extension line of the gesture and the plane where the ground is located based on the included angle information; determine a region of a preset area based on the third spatial coordinates, and use the region as the first region, wherein the region includes the position corresponding to the third spatial coordinates.

[0105] In an exemplary embodiment, a module 42 is configured to: determine the center position of the first device as the origin of the three-dimensional coordinate system; determine the positional direction of the target object relative to the first device, and define the positional direction as the horizontal axis direction of the three-dimensional coordinate system; determine the visual rightward extension direction of the first device based on the horizontal axis direction, and define the visual rightward extension direction as the vertical axis direction of the three-dimensional coordinate system; and determine the visual upward extension direction of the first device based on the horizontal axis direction, and define the visual upward extension direction as the vertical axis direction of the three-dimensional coordinate system; and establish the three-dimensional coordinate system based on the origin, the horizontal axis direction, the vertical axis direction, and the vertical axis direction.

[0106] In an exemplary embodiment, the sending module 46 is configured to acquire environmental information of the first area and acquire the current state of the second device based on the device information; determine a control strategy corresponding to the second device based on the environmental information, wherein the control strategy is used to adjust the second device to a target state; and, if the target state is inconsistent with the current state, generate a control command based on the control strategy and send the control command to the second device.

[0107] In one exemplary embodiment, Figure 5 This is a structural block diagram (II) of a control command transmitting device according to an embodiment of this application; as shown Figure 5 As shown, the above-mentioned device further includes a second determining module 52, configured to determine whether the gesture command is used to instruct control of the first device; if the gesture command is determined to be used to instruct control of the first device, control the first device to move to the first area; if the gesture command is not used to instruct control of the first device, determine the device information of the second device in the first area.

[0108] In one exemplary embodiment, Figure 5 This is a structural block diagram (II) of a control command transmitting device according to an embodiment of this application; as shown Figure 5As shown, the above-mentioned device further includes a control module 54, used to acquire map parameters corresponding to the second region, wherein the map parameters include: the correspondence between the second region and the third device, wherein the first region is a sub-region of the second region, and the third device includes at least: the first device and the second device; determine the third spatial coordinates of the third device in the three-dimensional coordinate system; determine the movement route of the first device to move to the spatial position corresponding to the second device according to the correspondence and the third spatial coordinates, and control the first device to move to the first region according to the movement route.

[0109] In an exemplary embodiment, the first determining module 44 is configured to, when determining the control command based on the device information, broadcast voice information carrying the device information and the control command; receive response information corresponding to the voice information, and determine whether to send the control command to the second device based on the response information; and, if it is determined that the control command should not be sent to the second device, adjust the control command based on the response information.

[0110] Embodiments of this application also provide a storage medium including a stored program, wherein the program executes any of the methods described above when it is run.

[0111] Optionally, in this embodiment, the storage medium may be configured to store program code for performing the following steps:

[0112] S1, when the first device receives a gesture command from the target object, a three-dimensional coordinate system is established based on the positional relationship between the first device and the target object;

[0113] S2, determine the angle information between the gesture direction corresponding to the gesture command and the vertical axis of the three-dimensional coordinate system, and determine the first region based on the angle information;

[0114] S3, determine the device information of the second device in the first area, and send a control command to the second device according to the device information.

[0115] Embodiments of this application also provide an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0116] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0117] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0118] S1, when the first device receives a gesture command from the target object, a three-dimensional coordinate system is established based on the positional relationship between the first device and the target object;

[0119] S2, determine the angle information between the gesture direction corresponding to the gesture command and the vertical axis of the three-dimensional coordinate system, and determine the first region based on the angle information;

[0120] S3, determine the device information of the second device in the first area, and send a control command to the second device according to the device information.

[0121] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0122] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0123] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0124] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for sending control commands, characterized in that, include: When the first device receives a gesture command from the target object, a three-dimensional coordinate system is established based on the positional relationship between the first device and the target object. Determine the angle information between the gesture direction corresponding to the gesture command and the vertical axis of the three-dimensional coordinate system, and determine the first region based on the angle information; Determine the device information of the second device in the first area, and send control commands to the second device based on the device information.

2. The method for sending control commands according to claim 1, characterized in that, Determining the angle information between the gesture direction corresponding to the gesture command and the vertical axis of the three-dimensional coordinate system includes: Determine the first and second spatial coordinates of the upper limb portion of the target object in the three-dimensional coordinate system; The spatial vector corresponding to the gesture is determined based on the first spatial coordinates and the second spatial coordinates; The angle information between the gesture direction and the vertical axis is determined based on the spatial vector.

3. The method for sending control commands according to claim 1, characterized in that, The first region is determined based on the included angle information, including: The third spatial coordinates of the intersection point of the extended line pointing to the hand gesture and the plane where the ground is located are determined based on the included angle information; A region of a preset size is determined based on the third spatial coordinates, and this region is taken as the first region, wherein the region includes the position corresponding to the third spatial coordinates.

4. The method for sending control commands according to claim 1, characterized in that, Establishing a three-dimensional coordinate system based on the positional relationship between the first device and the target object includes: The center position of the first device is determined as the origin of the three-dimensional coordinate system; Determine the position and direction of the target object relative to the first device, and define the position and direction as the horizontal axis direction of the three-dimensional coordinate system; The visual rightward extension direction of the first device is determined based on the horizontal axis direction, and the visual rightward extension direction is determined as the vertical axis direction of the three-dimensional coordinate system; and the visual upward extension direction of the first device is determined based on the horizontal axis direction, and the visual upward extension direction is determined as the vertical axis direction of the three-dimensional coordinate system. The three-dimensional coordinate system is established based on the origin, the horizontal axis, the vertical axis, and the center axis.

5. The method for transmitting control commands according to any one of claims 1-4, characterized in that, Sending control commands to the second device based on the device information, including: Obtain environmental information of the first area, and obtain the current status of the second device based on the device information; The control strategy corresponding to the second device is determined based on the environmental information, wherein the control strategy is used to adjust the second device to the target state; If the target state is inconsistent with the current state, the control command is generated according to the control strategy and sent to the second device.

6. The method for transmitting control commands according to any one of claims 1-4, characterized in that, After determining the first region based on the included angle information, the method further includes: Determine whether the gesture command is used to instruct control of the first device; If it is determined that the gesture command is used to instruct control of the first device, the first device is controlled to move to the first area; When the gesture command is not used to instruct control of the first device, the device information of the second device in the first area is determined.

7. The method for sending control commands according to claim 6, characterized in that, Controlling the first device to move to the first area includes: Obtain map parameters corresponding to the second region, wherein the map parameters include: the correspondence between the second region and the third device, wherein the first region is a sub-region of the second region, and the third device includes at least: the first device and the second device; Determine the third spatial coordinates of the third device in the three-dimensional coordinate system; Based on the correspondence and the third spatial coordinates, the movement route of the first device to the spatial position corresponding to the second device is determined, and the first device is controlled to move to the first area according to the movement route.

8. The method for sending control commands according to claim 1, characterized in that, Before sending control commands to the second device based on the device information, the method further includes: When the control command is determined based on the device information, voice information carrying the device information and the control command is broadcast. Receive the response information corresponding to the voice information, and determine whether to send a control command to the second device based on the response information; If it is determined that no control command will be sent to the second device, the control command is adjusted based on the response information.

9. A control command transmitting device, characterized in that, include: A module is established to establish a three-dimensional coordinate system based on the positional relationship between the first device and the target object when the first device receives a gesture command from the target object. The first determining module is used to determine the angle information between the gesture direction corresponding to the gesture instruction and the vertical axis of the three-dimensional coordinate system, and to determine the first region based on the angle information; The sending module is used to determine the device information of the second device in the first area and send control commands to the second device according to the device information.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method described in any one of claims 1 to 8.

11. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 8 through the computer program.

Images