Device control method and apparatus, electronic device, storage medium, and computer program product

Abstract

A device control method and apparatus, an electronic device (300), a computer-readable storage medium, and a computer program product. The method is applied to a first device (100). The first device (100) comprises a base plate (101) and a mechanical arm (102) provided on the base plate (101), and the mechanical arm (102) is provided with a signal transmitter (103). The method comprises: in response to receiving control information for a second device (200), controlling the base plate (101) to move and / or adjusting the position of the mechanical arm (102) (401); and by means of a signal transmitter (103), sending a control signal carrying the control information to the second device (200), so that the second device (200) performs an action on the basis of the control information (402), thereby enabling a user to conveniently and accurately control more devices located in a larger spatial range.

Description

Equipment control methods, devices, electronic equipment, storage media and computer program products

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to Chinese Patent Application No. 202411536018.3, filed on October 30, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to equipment control technology, and more particularly to an equipment control method, apparatus, electronic device, storage medium, and computer program product. Background Technology

[0004] With the development of smart devices, the types and functions of these devices are increasing, requiring more accurate and convenient methods for controlling them.

[0005] Currently, devices set up within a certain area often require users to move their own positions to control them. In real-world scenarios, due to the different locations and types of devices, the corresponding control methods also vary. When users control devices within the area, they need to adjust their own positions and control methods one-to-one for each different device, which is very inconvenient for users. Summary of the Invention

[0006] This application provides a device control method, apparatus, electronic device, computer-readable storage medium, and computer program product that enables users to conveniently and accurately control more devices located in a larger spatial area.

[0007] The technical solution of this application embodiment is implemented as follows:

[0008] This application provides a device control method executed by a first device, the first device including a chassis and a robotic arm mounted on the chassis, the robotic arm being equipped with a signal transmitter, the method including:

[0009] In response to receiving control information for the second device, control the chassis to move and / or adjust the pose of the robotic arm;

[0010] The signal transmitter sends a control signal carrying the control information to the second device, so that the second device can perform an action based on the control information.

[0011] In one embodiment, controlling the movement of the chassis and / or adjusting the pose of the robotic arm includes:

[0012] The first spatial location of the second device is determined based on a pre-built 3D map;

[0013] Based on the first spatial position, control the chassis to move and / or adjust the posture of the robotic arm so that the signal transmitter is within the signal receiving range of the second device.

[0014] In one embodiment, controlling the chassis to move and / or adjusting the pose of the robotic arm based on the first spatial position, so that the signal transmitter is within the signal receiving range of the second device, includes:

[0015] Based on the first spatial location, the transmission position and transmission direction of the signal transmitter are determined;

[0016] Based on the launch position and the launch direction, control the chassis to move and / or adjust the posture of the robotic arm so that the signal transmitter is in the launch position and launch direction.

[0017] In one embodiment, determining the transmission position and transmission direction of the signal transmitter based on the first spatial position includes:

[0018] Determine the second spatial location of the first device;

[0019] Based on the second spatial location and the first spatial location, path planning is performed on the first device to obtain the planned path;

[0020] Determine the transmission position and direction of the signal transmitter on the planned path.

[0021] In one embodiment, the method further includes:

[0022] Determine whether the second device has successfully executed the action;

[0023] After determining that the second device has failed to perform the action, the transmission position and / or transmission direction of the signal transmitter are re-determined;

[0024] Based on the redefined launch position and / or launch direction, control the chassis to move and / or adjust the pose of the robotic arm so that the signal transmitter is in the redefined launch position and / or launch direction;

[0025] The control signal carrying the control information is retransmitted to the second device via the signal transmitter.

[0026] In one embodiment, determining whether the second device has successfully performed the action includes:

[0027] Receive feedback signals from the second device;

[0028] The second device is determined to have successfully performed the action based on the feedback signal.

[0029] In one embodiment, determining whether the second device has successfully performed the action includes:

[0030] If, within a target time after the signal transmitter sends the control signal, the control information for the second device is received again, it is determined that the second device has failed to execute the action.

[0031] In one embodiment, determining whether the second device has successfully performed the action includes:

[0032] Before the signal transmitter sends the control signal, the first status information of the second device is collected;

[0033] After the signal transmitter sends the control signal, the second status information of the second device is collected;

[0034] Based on the first state information and the second state information, the actual change state of the second device is determined, and whether the action is successfully executed is determined based on whether the actual change state of the second device is consistent with the control change state corresponding to the control information.

[0035] In one embodiment, the robotic arm is equipped with a detection sensor, and the method further includes:

[0036] As the chassis moves within the target area, the position and posture of the robotic arm are adjusted.

[0037] During the robotic arm pose adjustment process, the spatial information of the target area is collected by the detection sensor, and at least one device in the target area is identified;

[0038] After the device is identified, the location information of the identified device in space is determined, and the second device is one of the at least one device;

[0039] Based on the spatial information and the location information, a three-dimensional map of the target area is constructed.

[0040] In one embodiment, after determining the location information of the identified device in space, the method further includes:

[0041] Based on the spatial location of the identified device, control the chassis to move and / or adjust the posture of the robotic arm so that the signal transmitter is within the signal receiving range of the identified device;

[0042] The signal transmitter sends a matching signal to the identified device so that the identified device can pair with the signal transmitter.

[0043] In one embodiment, the first device is a cleaning robot and the second device is a household appliance.

[0044] This application provides a device control apparatus for a first device, the first device including a chassis and a robotic arm mounted on the chassis, the robotic arm being equipped with a signal transmitter, the apparatus including:

[0045] A response module is used to control the chassis to move and / or adjust the posture of the robotic arm in response to receiving control information for the second device.

[0046] The signal transmitting module is used to send a control signal carrying the control information to the second device through the signal transmitter, so that the second device can perform an action based on the control information.

[0047] This application provides an electronic device, including: a chassis and a robotic arm mounted on the chassis, the robotic arm having a signal transmitter and the chassis having a control unit.

[0048] The control unit is configured to control the chassis to move and / or adjust the posture of the robotic arm in response to receiving control information for the second device;

[0049] The signal transmitter sends a control signal carrying the control information to the second device, so that the second device can perform an action based on the control information.

[0050] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the device control method provided in the above embodiments of this application.

[0051] This application provides a computer program product that stores a computer program for implementing the device control method provided in the above embodiments of this application when executed by a processor. Attached Figure Description

[0052] Figure 1 is a schematic diagram of an optional structure of the first device 100 provided in an embodiment of this application;

[0053] Figure 2 is a schematic diagram of an optional structure of the equipment control system provided in an embodiment of this application;

[0054] Figure 3 is a schematic diagram of an optional structure of the electronic device 300 provided in an embodiment of this application;

[0055] Figure 4 is a schematic flowchart of an optional device control method provided in an embodiment of this application;

[0056] Figure 5 is an optional schematic diagram of a first device sending a control signal to a second device according to an embodiment of this application;

[0057] Figure 6 is an optional detailed flowchart of step 401 of the device control method provided in the embodiment of this application;

[0058] Figure 7 is an optional detailed flowchart of step 602 of the device control method provided in the embodiment of this application. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0061] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0062] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this application is for the purpose of describing embodiments of this application only and is not intended to be limiting of this application.

[0063] This application provides a device control method, apparatus, electronic device, and computer-readable storage medium that enable users to conveniently and accurately control more devices located in a larger spatial area.

[0064] First, the first device provided in the embodiments of this application will be described. Referring to Figure 1, Figure 1 is an optional structural schematic diagram of the first device 100 provided in the embodiments of this application. The first device 100 includes a chassis 101 and a robotic arm 102. A signal transmitter 103 is provided on the robotic arm 102. Here, the robotic arm 102 is disposed on the chassis 101. The chassis 101 is movable, causing the first device 100 to move. The signal transmitter 103 is disposed at the end of the robotic arm 102. Specifically, in one embodiment, the chassis 101 is provided with a wheeled or tracked walking assembly (not shown in the figure) for realizing the overall movement of the first device 100. The robotic arm 102 is disposed on the chassis 101. The robotic arm 102 is a multi-degree-of-freedom robotic arm, such as a three-degree-of-freedom robotic arm, a four-degree-of-freedom robotic arm, etc., which is not specifically limited in this application. A gripper 104 is provided at the end of the robotic arm 102, and the gripper 104 is rotatable around the robotic arm 102. The gripper 104 includes at least two gripper fingers for fixing the signal transmitter 103. The position and orientation of the signal transmitter 103 are adjusted by controlling the movement of the chassis 101 and / or adjusting the posture (i.e., position and attitude) of the robotic arm 102, thereby achieving flexible adjustment of the spatial position and emission angle of the signal transmitter 103. Here, the signal emitted by the signal transmitter 103 can be an infrared signal, a Bluetooth signal, or a WiFi signal. The first device 100 also includes a control chip for executing the method provided in the embodiments of this application. In some embodiments, the first device 100 may also be equipped with other functions, such as an intelligent sweeping function, i.e., the chassis 101 is equipped with functional components for sweeping, thereby enabling multiple functions to be achieved through a single first device 100. Specifically, the first device 100 may be a cleaning robot with functions such as vacuuming and mopping, i.e., the chassis 101 is equipped with functional components such as a sweeping roller brush, a vacuum fan, and side brushes, and may also include a mopping module to achieve a mopping function.

[0065] The device control system provided in the embodiments of this application will be described below. Referring to Figure 2, Figure 2 is an optional structural schematic diagram of the device control system provided in the embodiments of this application. The first device 100 and the second device 200 can be communicatively connected. Here, the communication connection can be an infrared connection, a Bluetooth connection, or a wireless network connection. The second device 200 can be a home appliance, such as a television, air conditioner, washing machine, lighting equipment, monitoring equipment, or audio equipment, etc.

[0066] The electronic device for implementing the above-described device control method provided in the embodiments of this application will be described next. Referring to FIG3, FIG3 is an optional structural schematic diagram of the electronic device 300 provided in the embodiments of this application. In practical applications, the electronic device 300 can be implemented as the first device 100 in FIG2. The electronic device for implementing the device control method of the embodiments of this application will be described below.

[0067] The electronic device 300 shown in Figure 3 includes at least one processor 301 and a memory 302. The various components in the electronic device 300 are coupled together via a bus system 303. It is understood that the bus system 303 is used to implement communication between these components. In addition to a data bus, the bus system 303 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 303 in Figure 3.

[0068] Processor 301 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0069] The memory 302 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 302 may optionally include one or more storage devices physically located away from the processor 301.

[0070] Memory 302 may include volatile memory or non-volatile memory, or both. Non-volatile memory may be read-only memory (ROM), and volatile memory may be random access memory (RAM). The memory 302 described in this application embodiment is intended to include any suitable type of memory.

[0071] In some embodiments, the memory 302 can store data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof. In this embodiment, the memory 302 stores an operating system 3021 and a device control device 3022; specifically,

[0072] Operating system 3021 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, for implementing various basic business functions and handling hardware-based tasks.

[0073] In some embodiments, the device control apparatus provided in this application can be implemented in software. Figure 3 shows a device control apparatus 3022 stored in memory 302, which can be software in the form of programs and plug-ins, including the following software modules: a response module 30221 and a signal transmission module 30222. These modules are logically related and can therefore be arbitrarily combined or further split according to the functions they implement. The functions of each module will be described below.

[0074] In other embodiments, the device control device provided in this application can be implemented in hardware. As an example, the device control device provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the device control method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0075] The following describes the device control method provided in this application embodiment, with reference to the exemplary application and implementation of the first device provided in the embodiments of this application.

[0076] Referring to Figure 4, which is an optional flowchart of the device control method provided in an embodiment of this application, Figure 4 includes the following steps:

[0077] Step 401: In response to receiving control information for the second device, control the chassis to move and / or adjust the pose of the robotic arm;

[0078] Step 402: Send a control signal carrying the control information to the second device via a signal transmitter, so that the second device performs an action based on the control information.

[0079] In practical implementation, the second device can be a household appliance installed in the target area, which can be an area within the movement range of the first device. The first device can be a cleaning robot capable of cleaning the target area during movement. The first device can receive control information from the second device. This control information can be issued by the user. Specifically, the user can issue control information via voice; after the first device collects the user's voice, it performs voice recognition to obtain the control information carried in the user's voice. Control information can also be issued by the user through a user terminal. The user terminal can establish a communication connection with the first device and send control information to the first device through this connection. Here, the communication connection between the user terminal and the first device can be a local area network (LAN) or Bluetooth. Specifically, the user terminal can have a target application installed, and a communication connection can be established with the first device based on the target application. The user can then send control information to the first device by operating the target application.

[0080] In step 401, in response to receiving control information for the second device, the first device adjusts the pose of the signal transmitter. Specifically, the pose of the signal transmitter is adjusted by controlling the movement of the chassis and / or the robotic arm of the first device, so that the signal transmitter is adjusted to a suitable pose so that the second device can receive the signal transmitted by the signal transmitter. In practice, if the first device can adjust the signal transmitter to a suitable pose simply by adjusting the pose of the robotic arm when it receives the control information, then the chassis does not need to be moved, and only the pose of the robotic arm needs to be adjusted. If the first device cannot adjust the signal transmitter to a suitable pose simply by adjusting the pose of the robotic arm when it receives the control information, then the chassis of the first device needs to be moved. The suitable pose means that the signal transmitted by the signal transmitter in that pose can be received by the second device. Here, if the current pose of the robotic arm after moving the chassis can put the signal transmitter in a suitable pose, then the robotic arm does not need to be adjusted; otherwise, the pose of the robotic arm needs to be adjusted further.

[0081] In this embodiment, the robotic arm can adjust the spatial position of the signal transmitter in three-dimensional space. Furthermore, the robotic arm can rotate, and the rotatable angle can be a target angle rotated around any coordinate axis in three-dimensional space. The target angle can be 360 ​​degrees. Specifically, the three-dimensional space of the three-dimensional map has an x-axis, a y-axis, and a z-axis, and the robotic arm can rotate the target angle based on the x-axis, y-axis, and z-axis, respectively. Through the movement and rotation of the robotic arm, the pose of the signal transmitter can be controlled more precisely and over a wider range.

[0082] In step 402, after adjusting the position of the signal transmitter, the signal transmitter is controlled to send a control signal carrying control information. Here, the control signal can be an infrared signal, and the signal transmitter is an infrared transmitter. The second device is equipped with a corresponding infrared receiver. After receiving the control signal, the second device performs a corresponding action based on the control information. Here, the control information is used to control the second device to perform a corresponding action. For example, if the control information is "power on," the second device will perform the power-on action. If the control information is "increase volume," which controls the television (the second device), the television, after receiving the control signal to increase volume, will increase the volume.

[0083] For example, referring to Figure 5, Figure 5 is an optional schematic diagram of a first device sending a control signal to a second device according to an embodiment of this application. Here, the transmission position and transmission direction of the signal transmitter 103 of the first device 100 are within the signal receiving range of the second device 200, and the control signal transmitted by the signal transmitter 103 at this transmission position and transmission direction can be received by the second device 200.

[0084] In this embodiment, in response to receiving control information for the second device, the chassis is moved and / or the robotic arm's pose is adjusted. A control signal carrying the control information is sent to the second device via the signal transmitter, so that the second device performs an action based on the control information. The first device chassis is equipped with a robotic arm. By moving the chassis and adjusting the robotic arm's pose, the pose of the signal transmitter is adjusted, enabling a wider signal transmission range. This allows a single first device to control devices located in a larger spatial area, achieving convenient and accurate control of more devices located in a larger spatial area.

[0085] In some embodiments, referring to FIG6, FIG6 is an optional detailed flowchart of step 401 of the device control method provided in the embodiments of this application. In step 401, controlling the movement of the chassis and / or adjusting the pose of the robotic arm includes:

[0086] Step 601: Determine the first spatial location of the second device based on a pre-built 3D map;

[0087] Step 602: Based on the first spatial position, control the chassis to move and / or adjust the posture of the robotic arm so that the signal transmitter is within the signal receiving range of the second device.

[0088] In step 601, in response to receiving control information for the second device, the first device determines the first spatial location of the second device based on a pre-constructed 3D map. Here, the first spatial location of the second device is pre-marked on the 3D map. The first spatial location includes the planar position and height of the target area where the second device is located. After receiving control information for the second device, the first device can determine the first spatial location of the second device from the 3D map. In one embodiment, the first spatial location may be the location of the signal receiving center of the second device. The signal receiving center may be the signal receiver of the second device, such as an infrared receiver.

[0089] In step 602, based on the first spatial position, the signal transmitter is adjusted to be within the signal receiving range of the second device. Here, the signal receiving range can be preset based on the signal receiving center of the second device, which is a certain positional and angular range of the signal receiving center of the second device. The signal receiving range can also be determined based on the device type of the second device and the first spatial position. Different types of second devices may have different corresponding signal receiving ranges. In practice, the position of the signal transmitter can be adjusted by controlling the movement of the chassis of the first device and / or the robotic arm. In practice, if, upon receiving control information, the first device is within the target spatial range of the second device, and adjusting the posture of the robotic arm is sufficient to place the signal transmitter within the signal receiving range of the second device, then the chassis need not be moved, and only the posture of the robotic arm needs to be adjusted. If, upon receiving control information, the first device is not within the target spatial range of the second device, and adjusting the posture of the robotic arm is insufficient to place the signal transmitter within the signal receiving range of the second device, then the chassis of the first device needs to be moved. The target spatial range refers to the spatial range within which, when the first device is within this range, the control signal transmitted by the signal transmitter can be received by the second device simply by adjusting the posture of the robotic arm. Here, if the current posture of the robotic arm allows the signal transmitter to be within the signal receiving range of the second device after moving the chassis, then the robotic arm does not need to be adjusted; otherwise, the posture of the robotic arm needs to be adjusted further.

[0090] In some embodiments, referring to FIG7, FIG7 is an optional detailed flowchart of step 602 of the device control method provided in the embodiments of this application, wherein step 602 includes:

[0091] Step 701: Based on the first spatial position, determine the transmission position and transmission direction of the signal transmitter;

[0092] Step 702: Based on the launch position and the launch direction, control the chassis to move and / or adjust the posture of the robotic arm so that the signal transmitter is in the launch position and launch direction.

[0093] In practical implementation, after determining the first spatial position of the second device, the transmission position and direction that enable the signal emitted by the signal transmitter to be within the signal receiving range of the second device can be determined. Then, the chassis is moved and / or the posture of the robotic arm is adjusted by a motion control algorithm, thereby positioning the signal transmitter in the determined transmission position and direction.

[0094] In some embodiments, determining the transmission position and transmission direction of the signal transmitter based on the first spatial position includes: determining the second spatial position of the first device; performing path planning on the first device based on the second spatial position and the first spatial position to obtain a planned path; and determining the transmission position and transmission direction of the signal transmitter on the planned path.

[0095] In practical implementation, the process also includes determining the second spatial location of the first device. Based on the 3D map and the first and second spatial locations, path planning is performed on the first device to obtain a planned path. Here, path planning can be based on the shortest path principle. The planned path includes the destination of the first device's movement. In one embodiment, the planned path also includes the transmission pose of the signal transmitter, which includes the transmission position and transmission direction. After obtaining the planned path, the chassis can be moved and / or the robotic arm's pose can be adjusted to ensure the signal transmitter reaches the corresponding transmission position and is adjusted to the corresponding transmission direction.

[0096] In some embodiments, the method further includes: determining whether the second device has successfully performed an action; after determining that the second device has not successfully performed an action, re-determining the transmission position and / or transmission direction of the signal transmitter; based on the re-determined transmission position and / or transmission direction, controlling the chassis to move and / or adjusting the pose of the robotic arm so that the signal transmitter is in the re-determined transmission position and / or transmission direction; and re-sending a control signal carrying the control information to the second device through the signal transmitter.

[0097] In practice, the first device determines whether the second device has successfully executed its action. If the second device has successfully executed its action, the first device can remain in place or return to the target location to await the next control message. The target location is a reference point or reference state set for efficient and seamless multi-task scheduling; its specific meaning is entirely determined by the system's workflow and design requirements. If the first device has other functions and receives control information while performing those functions, it pauses the execution of those functions, responds to the received control information, and then, after confirming that the second device has successfully executed its action, returns to the target location to perform those other functions. Here, the other functions could be, for example, a sweeping function, and the target location could be the location where the first device was initially interrupted during the sweeping task.

[0098] If it is determined that the second device has failed to execute the action, the transmission position and / or transmission direction of the signal transmitter are redefined to determine a new transmission posture. The redefined transmission position differs from the transmission position determined in step 701, and / or the redefined transmission direction differs from the transmission direction determined in step 701. It should be understood that, based on the redefined transmission position and direction, the signal transmitter can be within the signal receiving range of the second device. In actual implementation, the first device, based on the redefined transmission position and / or transmission direction, again controls the chassis movement and / or adjusts the posture of the robotic arm to position the signal transmitter in the redefined transmission position and / or transmission direction. Then, the signal transmitter retransmits a control signal carrying control information to the second device.

[0099] Here, by determining whether the second device has successfully executed the action, and if it has not, the transmission posture of the signal transmitter is readjusted so that the control signal is transmitted based on the new transmission posture, thereby improving the control accuracy of the second device.

[0100] In some embodiments, determining whether the second device has successfully performed the action includes: receiving a feedback signal from the second device; and determining whether the second device has successfully performed the action based on the feedback signal.

[0101] In practical implementation, the feedback signal can be transmitted via WiFi, Bluetooth, or infrared. Here, if the feedback signal is an infrared signal, the second device also has an infrared transmitter, and correspondingly, the first device also has an infrared receiver. After successfully executing an action, the second device sends a feedback signal indicating successful action execution to the first device. If the first device does not receive a feedback signal from the second device within a time threshold after sending the control signal, it determines that the second device has not successfully executed the action. Here, the time threshold can be preset, for example, it can be set to 30 seconds. In some embodiments, the first device can also determine that the second device has not successfully executed the action after receiving a feedback signal from the second device indicating unsuccessful execution. After receiving the control signal, if the second device cannot recognize the control information due to a weak control signal or other reasons, or if the control signal fails due to a signal malfunction or other reasons, it sends a feedback signal indicating unsuccessful execution to the first device. Through the feedback signal from the second device, it is possible to accurately and efficiently determine whether the second device has successfully executed the action.

[0102] In some embodiments, determining whether the second device has successfully performed the action includes: if the control information for the second device is received again within a target time after the signal transmitter sends the control signal, then it is determined that the second device has not successfully performed the action.

[0103] Here, the target time can be preset, for example, 30 seconds. In actual implementation, if control information for the second device is received again within the target time after the signal transmitter sends the control signal, it is determined that the second device has failed to execute the action. In a real-world scenario, if the user determines that the second device has failed to execute the action, they will send control information for the second device again within a short period of time. After the first device receives this control information again within the target time, it determines that the second device has failed to execute the action and adjusts the transmission posture of the signal transmitter to send the control signal again.

[0104] In some embodiments, determining whether the second device has successfully executed an action includes: acquiring first state information of the second device before the signal transmitter sends the control signal; acquiring second state information of the second device after the signal transmitter sends the control signal; determining the actual change state of the second device based on the first state information and the second state information; and determining whether the action has been successfully executed based on whether the actual change state of the second device is consistent with the control change state corresponding to the control information.

[0105] In actual implementation, the first device is also equipped with a data acquisition device. Before the signal transmitter sends the control signal, the data acquisition device acquires the first state information of the second device. After the signal transmitter sends the control signal, it acquires the second state information of the second device. Based on the change information of the second state information compared to the first state information, the actual change state of the second device is determined. Whether the action was successfully executed is determined based on whether the actual change state is consistent with the control change state corresponding to the control information. If they are consistent, the second device is determined to have successfully executed the action; otherwise, the second device is determined to have failed to execute the action.

[0106] For example, for a television set as the second device, the first device can capture the display screen of the television set and use the display screen as corresponding status information. If the actual change in the display screen matches the control change corresponding to the control information, then the television set has successfully executed the action; otherwise, the action has not been successfully executed. Here, the capturing device can be a camera or an RGB sensor.

[0107] In some embodiments, the robotic arm is equipped with a detection sensor, and the method further includes: adjusting the pose of the robotic arm during the movement of the chassis within the target area; during the pose adjustment of the robotic arm, acquiring spatial information of the target area through the detection sensor and identifying at least one device within the target area; after identifying a device, determining the location information of the identified device in space, wherein the second device is one of the at least one device; and constructing the three-dimensional map of the target area based on the spatial information and the location information.

[0108] In practical implementation, the detection sensor includes an RGB sensor. In one embodiment, the detection sensor also includes a Time-of-Flight (ToF) sensor. As the chassis of the first device moves within the target area, the robot arm's pose is adjusted to maintain continuous pose changes. During these pose changes, spatial information of the target area is collected by the detection sensor. Here, the spatial information of the target area includes spatial information of objects within the target area. The objects within the target area include at least one device. The second device is one of the at least one devices. During the collection of spatial information, target recognition is also performed on the target area. The recognized target is at least one device. After the device is recognized, its spatial location is determined. Here, the detection sensor can also determine the device's dimensions. Then, based on the spatial information of the target area and the position and dimensions of at least one device, a 3D map of the target area is constructed. Here, after the device is recognized, its type is also identified and marked on the 3D map. 3D modeling can be performed based on the device's position and dimensions, and the device model can be added to the 3D map with its type labeled.

[0109] In some embodiments, after determining the location information of the identified device in space, the method further includes: controlling the chassis to move and / or adjusting the pose of the robotic arm according to the location of the identified device in space, so that the signal transmitter is within the signal receiving range of the identified device; and sending a matching signal to the identified device through the signal transmitter so that the identified device and the signal transmitter can be paired.

[0110] In practical implementation, after a device is identified, the signal transmitter can be positioned within the signal receiving range of the identified device based on its spatial location. Then, the signal transmitter sends a matching signal to the identified device to pair it with the signal transmitter. Here, signal pairing can be based on the type of identified device. In this embodiment, the first device can pair with multiple second devices to control them.

[0111] The following continues to describe an exemplary structure of the device control device 3022 provided in this application embodiment as a software module. In some embodiments, as shown in FIG3, the software module stored in the device control device 3022 in the memory 302 may include:

[0112] The response module 30221 is used to control the chassis to move and / or adjust the posture of the robotic arm in response to receiving control information for the second device.

[0113] The signal transmitting module 30222 is used to send a control signal carrying the control information to the second device through the signal transmitter, so that the second device performs an action based on the control information.

[0114] In some embodiments, the response module 30221 is further configured to determine a first spatial position of the second device based on a pre-built three-dimensional map; and based on the first spatial position, control the chassis to move and / or adjust the pose of the robotic arm so that the signal transmitter is within the signal receiving range of the second device.

[0115] In some embodiments, the response module 30221 is further configured to determine the transmission position and transmission direction of the signal transmitter based on the first spatial position; and control the chassis to move and / or adjust the posture of the robotic arm based on the transmission position and the transmission direction, so that the signal transmitter is in the transmission position and transmission direction.

[0116] In some embodiments, determining the transmission position and transmission direction of the signal transmitter based on the first spatial position includes: determining the second spatial position of the first device; performing path planning on the first device based on the second spatial position and the first spatial position to obtain a planned path; and determining the transmission position and transmission direction of the signal transmitter on the planned path.

[0117] In some embodiments, the apparatus further includes: a retransmission module, configured to determine whether the second device has successfully performed an action; after determining that the second device has not successfully performed an action, to re-determine the transmission position and / or transmission direction of the signal transmitter; based on the re-determined transmission position and / or transmission direction, to control the chassis to move and / or adjust the posture of the robotic arm so that the signal transmitter is in the re-determined transmission position and / or transmission direction; and to retransmit a control signal carrying the control information to the second device through the signal transmitter.

[0118] In some embodiments, the retransmission module is further configured to receive a feedback signal from the second device; and determine whether the second device has successfully performed the action based on the feedback signal.

[0119] In some embodiments, the retransmission module is further configured to determine that the second device has failed to execute the action if the control information for the second device is received again within a target time after the signal transmitter sends the control signal.

[0120] In some embodiments, the retransmission module is further configured to: collect first state information of the second device before the signal transmitter sends the control signal; collect second state information of the second device after the signal transmitter sends the control signal; determine the actual change state of the second device based on the first state information and the second state information; and determine whether the action has been successfully executed based on whether the actual change state of the second device is consistent with the reference change state after the action is executed.

[0121] In some embodiments, the robotic arm is equipped with a detection sensor, and the device further includes: a mapping module, used to adjust the pose of the robotic arm during the movement of the chassis within the target area; during the pose adjustment of the robotic arm, spatial information of the target area is collected through the detection sensor, and at least one device within the target area is identified; after identifying the device, the location information of the identified device is determined, wherein the second device is one of the at least one device; and a three-dimensional map of the target area is constructed based on the spatial information and the location information.

[0122] In some embodiments, after determining the location information of the identified device in space, the device further includes: a signal matching module, configured to control the chassis to move and / or adjust the pose of the robotic arm according to the location of the identified device in space, so that the signal transmitter is within the signal receiving range of the identified device; and to send a matching signal to the identified device through the signal transmitter so that the identified device and the signal transmitter are paired.

[0123] In some embodiments, the first device is a cleaning robot, and the second device is a home appliance.

[0124] It should be noted that the description of the apparatus in this application embodiment is similar to the description of the method embodiment above, and has similar beneficial effects as the method embodiment, so it will not be repeated.

[0125] This application provides an electronic device, including: a chassis and a robotic arm mounted on the chassis, the robotic arm having a signal transmitter and the chassis having a control unit.

[0126] The control unit is configured to, in response to receiving control information for the second device, determine the first spatial location of the second device based on a pre-built three-dimensional map;

[0127] Based on the first spatial position, control the chassis to move and / or adjust the posture of the robotic arm so that the signal transmitter is within the signal receiving range of the second device;

[0128] The signal transmitter sends a control signal carrying the control information to the second device, so that the second device can perform an action based on the control information.

[0129] This application provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the device control method described in this application.

[0130] This application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored and when executed by a processor, they will cause the processor to execute the device control method provided in this application.

[0131] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0132] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0133] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., files that store one or more modules, subroutines, or code sections).

[0134] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0135] In summary, the embodiments of this application enable users to conveniently and accurately control devices located in a larger spatial area.

[0136] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A device control method, executed by a first device, wherein, The first device includes a chassis and a robotic arm mounted on the chassis, the robotic arm being equipped with a signal transmitter, and the method includes: In response to receiving control information for the second device, control the chassis to move and / or adjust the pose of the robotic arm; The signal transmitter sends a control signal carrying the control information to the second device, so that the second device can perform an action based on the control information.

2. The method according to claim 1, wherein, The control of the chassis movement and / or the adjustment of the robot arm's pose includes: The first spatial location of the second device is determined based on a pre-built 3D map; Based on the first spatial position, control the chassis to move and / or adjust the posture of the robotic arm so that the signal transmitter is within the signal receiving range of the second device.

3. The method according to claim 2, wherein, The step of controlling the chassis to move and / or adjusting the pose of the robotic arm based on the first spatial position, so that the signal transmitter is within the signal receiving range of the second device, includes: Based on the first spatial location, the transmission position and transmission direction of the signal transmitter are determined; Based on the launch position and the launch direction, control the chassis to move and / or adjust the posture of the robotic arm so that the signal transmitter is in the launch position and launch direction.

4. The method according to claim 3, wherein, Determining the transmission position and transmission direction of the signal transmitter based on the first spatial position includes: Determine the second spatial location of the first device; Based on the second spatial location and the first spatial location, path planning is performed on the first device to obtain the planned path; Determine the transmission position and direction of the signal transmitter on the planned path.

5. The method according to claim 3 or 4, wherein, The method further includes: Determine whether the second device has successfully executed the action; After determining that the second device has failed to perform the action, the transmission position and / or transmission direction of the signal transmitter are re-determined; Based on the redefined launch position and / or launch direction, control the chassis to move and / or adjust the pose of the robotic arm so that the signal transmitter is in the redefined launch position and / or launch direction; The control signal carrying the control information is retransmitted to the second device via the signal transmitter.

6. The method according to claim 5, wherein, Determining whether the second device has successfully executed the action includes: Receive feedback signals from the second device; The second device is determined to have successfully performed the action based on the feedback signal.

7. The method according to claim 5, wherein, Determining whether the second device has successfully executed the action includes: If, within a target time after the signal transmitter sends the control signal, the control information for the second device is received again, it is determined that the second device has failed to execute the action.

8. The method according to claim 5, wherein, Determining whether the second device has successfully executed the action includes: Before the signal transmitter sends the control signal, the first status information of the second device is collected; After the signal transmitter sends the control signal, the second status information of the second device is collected; Based on the first state information and the second state information, the actual change state of the second device is determined, and whether the action is successfully executed is determined based on whether the actual change state of the second device is consistent with the control change state corresponding to the control information.

9. The method according to claim 2, wherein, The robotic arm is equipped with a detection sensor, and the method further includes: As the chassis moves within the target area, the position and posture of the robotic arm are adjusted. During the robotic arm pose adjustment process, the spatial information of the target area is collected by the detection sensor, and at least one device in the target area is identified; After the device is identified, the location information of the identified device in space is determined, and the second device is one of the at least one device; Based on the spatial information and the location information, a three-dimensional map of the target area is constructed.

10. The method according to claim 9, wherein, After determining the location information of the identified device in space, the method further includes: Based on the spatial location of the identified device, control the chassis to move and / or adjust the posture of the robotic arm so that the signal transmitter is within the signal receiving range of the identified device; The signal transmitter sends a matching signal to the identified device so that the identified device can pair with the signal transmitter.

11. The method according to any one of claims 1-10, wherein, The first device is a cleaning robot, and the second device is a home appliance.

12. A device control apparatus, applied to a first device, wherein, The first device includes a chassis and a robotic arm mounted on the chassis. The robotic arm is equipped with a signal transmitter. The device includes: A response module is used to control the chassis to move and / or adjust the posture of the robotic arm in response to receiving control information for the second device. The signal transmitting module is used to send a control signal carrying the control information to the second device through the signal transmitter, so that the second device can perform an action based on the control information.

13. An electronic device, wherein, include: A chassis and a robotic arm mounted on the chassis, the robotic arm being equipped with a signal transmitter, and a control unit being located inside the chassis; The control unit is configured to, in response to receiving control information for the second device, control the chassis to move and / or adjust the posture of the robotic arm; and send a control signal carrying the control information to the second device via the signal transmitter, so that the second device performs an action based on the control information.

14. A computer-readable storage medium, wherein, It stores executable instructions for causing the processor to execute, thereby implementing the device control method as described in any one of claims 1-11.

15. A computer program product, wherein, The device contains a computer program that, when executed by a processor, implements the device control method as described in any one of claims 1-11.

Images