A remote control method, apparatus, device and storage medium

Abstract

This disclosure provides a remote control method, apparatus, device, and storage medium. The method includes: acquiring a scene image corresponding to a preset area and a recognition result corresponding to the scene image; the recognition result includes device information of each of at least one controllable device in the preset area; sending the scene image to a terminal; the terminal is used to display the scene image and receive input operation instructions; receiving the operation instructions sent by the terminal, and based on the operation instructions and the device information of each controllable device, determining a target controllable device and a corresponding control instruction; sending the control instruction to the target controllable device, the control instruction instructing the target controllable device to perform the action corresponding to the control instruction.

Description

Technical Field

[0001] This disclosure relates to the field of data processing, and more particularly to a remote control method, apparatus, device, and storage medium. Background Technology

[0002] Current intelligent monitoring is mostly unidirectional, with video streams transmitted from cameras to display screens or stored on hard drives for human viewing or program analysis. In practical applications, such as high-temperature environments, cold storage facilities, laboratories with stringent environmental requirements, and hazardous environments, managers often need to be fully equipped with protective gear to reach the site and perform related operations. This approach typically consumes more manpower and resources and carries certain risks. Therefore, a remote control solution is urgently needed to address these technical challenges. Summary of the Invention

[0003] This disclosure provides a remote control method, apparatus, device, and storage medium.

[0004] Firstly, a remote control method is provided, including:

[0005] Acquire a scene image corresponding to a preset region and a recognition result corresponding to the scene image; the recognition result includes device information of each of the at least one device to be controlled in the preset region.

[0006] The scene image is sent to the terminal; the terminal is used to display the scene image and receive input operation commands.

[0007] The system receives the operation instruction sent by the terminal and, based on the operation instruction and the device information of each device to be controlled, determines the target device to be controlled and the control instruction corresponding to the target device to be controlled.

[0008] The control command is sent to the target device to be controlled, and the control command is used to instruct the target device to perform the action corresponding to the control command.

[0009] In some embodiments, the operation instruction includes a selection sub-instruction. Receiving the operation instruction sent by the terminal and determining the target device to be controlled based on the operation instruction and device information of each device to be controlled includes:

[0010] Receive the selection sub-instruction sent by the terminal;

[0011] Parse the selection sub-instruction to obtain selection information;

[0012] Based on the selection information and the device information of each of the devices to be controlled, the target device to be controlled is determined from the at least one device to be controlled.

[0013] In some embodiments, the selection information includes input selection coordinates of the target device to be controlled, the device information includes relative coordinates of the device to be controlled, and determining the target device to be controlled from the at least one device to be controlled based on the selection information and the device information of each device to be controlled includes:

[0014] Based on the selected coordinates of the target device to be controlled and the relative coordinates of each of the devices to be controlled, the target device to be controlled is determined among the at least one device to be controlled.

[0015] In some embodiments, the selected coordinates include first selected coordinates of the target controllable device based on the image coordinate system, and the relative coordinates include first relative coordinates of the controllable device based on the image coordinate system; determining the target controllable device among the at least one controllable device based on the selected coordinates of the target controllable device and the relative coordinates of each controllable device includes:

[0016] The target device to be controlled is determined among the at least one device to be controlled based on the first selected coordinates of the target device to be controlled based on the image coordinate system and the first relative coordinates of each device to be controlled based on the image coordinate system.

[0017] In some embodiments, the selected coordinates include a second selected coordinate of the target controllable device based on the regional coordinate system; determining the target controllable device among the at least one controllable device based on the selected coordinates of the target controllable device and the relative coordinates of each controllable device includes:

[0018] The target device to be controlled is determined among the at least one device to be controlled based on the second selected coordinates of the target device to be controlled based on the regional coordinate system and the second relative coordinates of each device to be controlled based on the regional coordinate system.

[0019] In some embodiments, the method further includes: based on the coordinate transformation relationship between the image coordinate system and the region coordinate system corresponding to the preset region, converting the first relative coordinate of each device to be controlled based on the image coordinate system into the second relative coordinate of each device to be controlled based on the region coordinate system;

[0020] Sending the scene image to the terminal includes: sending the scene image and a second relative coordinate of each of the devices to be controlled based on the regional coordinate system to the terminal; the second relative coordinate is used to indicate the actual position of each device to be controlled in the preset area during the display of the scene image on the terminal.

[0021] In some embodiments, the selection information includes an input target identifier of the target device to be controlled, the relative coordinates include a first relative coordinate of the device to be controlled based on the image coordinate system, and determining the target device to be controlled from the at least one device to be controlled based on the selection information and device information of each device to be controlled includes:

[0022] Based on the first relative coordinates corresponding to each of the devices to be controlled, and the first identifier coordinates corresponding to at least one first visual identifier obtained by identifier detection of the scene image, the first visual identifier corresponding to each of the devices to be controlled is determined.

[0023] Based on the target identifier and the first visual identifier corresponding to each of the devices to be controlled, the target device to be controlled is determined among the at least one device to be controlled.

[0024] In some embodiments, determining the first visual identifier corresponding to each of the controlled devices based on the first relative coordinates corresponding to each of the controlled devices and the first identifier coordinates corresponding to at least one first visual identifier obtained by identifier detection of the scene image includes any one of the following:

[0025] Based on a preset identifier recognition model, the scene image is detected to obtain the first identifier coordinates of each visual identifier in at least one visual identifier corresponding to the scene image; based on the first identifier coordinates of each first visual identifier and the first relative coordinates of each device to be controlled, the first visual identifier corresponding to each device to be controlled is determined.

[0026] Based on the first identifier coordinates corresponding to each of the devices to be controlled, the scene image is segmented to obtain a device sub-image corresponding to each device to be controlled; based on a preset identifier recognition model, the identifier of each device to be controlled is detected to determine the first visual identifier corresponding to each device to be controlled.

[0027] In some embodiments, determining the target device to be controlled among the at least one device to be controlled based on the target identifier and the first visual identifier corresponding to each device to be controlled includes:

[0028] When the target identifier is a target visual identifier, the device to be controlled corresponding to the first visual identifier that matches the target visual identifier is determined as the target device to be controlled;

[0029] When the target identifier is a target device identifier, based on the mapping relationship between device identifiers and visual identifiers, the device to be controlled corresponding to the first visual identifier that matches the target device identifier is determined as the target device to be controlled.

[0030] In some embodiments, the operation instruction includes an action sub-instruction; receiving and parsing the operation instruction sent by the terminal to determine the control instruction corresponding to the target controllable device includes:

[0031] Receive the action sub-instruction sent by the terminal;

[0032] Parse the action sub-instruction to obtain action information;

[0033] Based on the action information, control commands corresponding to the target controllable device are generated.

[0034] In some embodiments, generating control commands corresponding to the target controllable device based on the motion information includes:

[0035] When the motion information is based on the image coordinate system, the motion information is converted into target motion information based on the transformation relationship between the image coordinate system and the region coordinate system; when the motion information is based on the region coordinate system, the motion information is used as the target motion information.

[0036] Based on the target motion information, control commands corresponding to the target controllable device are generated.

[0037] In some embodiments, the action sub-instruction includes at least one of the following: a movement sub-instruction, a turning sub-instruction, and a work sub-instruction; wherein,

[0038] The movement sub-command carries movement information, which includes at least one of the following: target movement position, movement speed, and movement trajectory.

[0039] The steering sub-command carries action information that is steering information, which includes at least one of the following: target steering angle, steering speed, and steering trajectory;

[0040] The action information carried by the sub-instruction is the task information, which may include at least one of the following: task status, task objective, task time, and task content.

[0041] In some embodiments, obtaining the scene image corresponding to the preset region and the recognition result corresponding to the scene image includes any one of the following:

[0042] The system receives the scene image and the corresponding recognition result sent by the camera device; the recognition result is generated by the camera device based on the scene image.

[0043] Receive the scene image sent by the camera device; generate the recognition result based on the scene image.

[0044] In some embodiments, the method further includes:

[0045] Based on the mapping relationship between device identifier and communication address, obtain the communication address corresponding to the target device to be controlled;

[0046] Sending the control command to the target device to be controlled includes:

[0047] Based on the communication address corresponding to the target device to be controlled, the control command is sent to the target device to be controlled.

[0048] Secondly, a remote control device is provided, comprising:

[0049] The acquisition module is used to acquire a scene image corresponding to a preset area and a recognition result corresponding to the scene image; the recognition result includes device information of each of the at least one device to be controlled in the preset area.

[0050] The first sending module is used to send the scene image to the terminal; the terminal is used to display the scene image and receive input operation instructions.

[0051] A receiving module is configured to receive the operation instruction sent by the terminal, and determine the target device to be controlled and the control instruction corresponding to the target device to be controlled based on the operation instruction and the device information of each device to be controlled.

[0052] The second sending module is used to send the control command to the target device to be controlled, and the control command is used to instruct the target device to perform the action corresponding to the control command.

[0053] Thirdly, a remote control device is provided, comprising: a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the computer program to implement the steps in the above-described method.

[0054] Fourthly, a computer storage medium is provided, which stores one or more programs that can be executed by one or more processors to implement the steps in the above method.

[0055] In this embodiment, by sending the scene image corresponding to the preset area to the terminal and displaying it to the user, and simultaneously receiving the user's real-time operation instructions based on the scene image, and sending corresponding control instructions to the device to be controlled in the preset area, this disclosure achieves a remote control solution for the device to be controlled, which not only reduces material costs and saves manpower costs, but also avoids the risks of management personnel entering the site, compared to the traditional approach where managers need to be fully equipped to enter the site to perform related operations. Furthermore, since the control instructions generated based on the recognition results corresponding to the scene image and the operation instructions received by the terminal can be used to determine the target device to be controlled from multiple devices to be controlled in the preset area, the control accuracy of the remote control method in this embodiment is improved. Through the control instructions generated based on the operation instructions in this embodiment, the target device to be controlled can perform actions corresponding to the user's real-time operations, achieving a more user-friendly interaction method. Attached Figure Description

[0056] Figure 1 A flowchart illustrating a remote control method provided in an embodiment of this disclosure;

[0057] Figure 2 A flowchart illustrating a remote control method provided in an embodiment of this disclosure;

[0058] Figure 3 A flowchart illustrating a remote control method provided in an embodiment of this disclosure;

[0059] Figure 4 A flowchart illustrating a remote control method provided in an embodiment of this disclosure;

[0060] Figure 5 A flowchart illustrating a remote control method provided in an embodiment of this disclosure;

[0061] Figure 6 A flowchart illustrating a remote control method provided in an embodiment of this disclosure;

[0062] Figure 7 A schematic diagram of an optional implementation scenario provided by this disclosure;

[0063] Figure 8 An optional information flow diagram provided for an embodiment of this disclosure;

[0064] Figure 9 This is a schematic diagram of the composition structure of a remote control device provided in an embodiment of the present disclosure;

[0065] Figure 10 This is a schematic diagram of the hardware entity of a remote control device provided in an embodiment of the present disclosure. Detailed Implementation

[0066] The technical solutions of this disclosure will be described in detail below through embodiments and with reference to the accompanying drawings. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0067] It should be noted that in the embodiments of this disclosure, terms such as "first" and "second" are used to distinguish similar objects and are not necessarily used to describe the order or sequence of the targets. Furthermore, the technical solutions described in the embodiments of this disclosure can be arbitrarily combined without conflict.

[0068] Figure 1 This is a flowchart illustrating a remote control method provided in an embodiment of the present disclosure. The method includes:

[0069] S101. Obtain the scene image corresponding to the preset area and the recognition result corresponding to the scene image; the recognition result includes device information of each of the at least one device to be controlled in the preset area.

[0070] The preset area is a region in a real-world scene, such as a warehouse, laboratory, or plaza. In this embodiment, real-time scene images corresponding to the preset area can be acquired using camera devices positioned within the preset area. The camera device can include at least one camera component, and each camera component can be positioned in different sub-areas of the preset area. All camera components can cover the entire preset area. For example, if the preset area is a warehouse, due to obstacles such as goods and shelves, a single camera component may not be able to cover the entire preset area; therefore, multiple camera components can be set to cover the entire preset area. If the preset area is a plaza, due to the open field of view, only one camera component can be set to cover the entire preset area.

[0071] In some embodiments, at least one device to be controlled exists in the preset area, and correspondingly, the acquired scene images also include real-time images of each device to be controlled.

[0072] In some embodiments, the above-mentioned S101 can be implemented by receiving the scene image and the recognition result corresponding to the scene image sent by the camera device; the recognition result is generated by the camera device based on the scene image.

[0073] In the process of acquiring scene images corresponding to a preset area, the camera device will identify the scene image based on the recognition algorithm stored locally on the camera device to obtain the device information corresponding to each device to be controlled. Then, the camera will package the scene image and the device information corresponding to each device to be controlled in the scene image and send it to the server.

[0074] In some embodiments, the above-mentioned S101 can also be implemented by: receiving the scene image sent by the camera device; generating the recognition result based on the scene image.

[0075] After acquiring the scene image corresponding to the preset area, the camera device directly sends the scene image to the server as a video stream. After receiving the scene image, the server will identify the scene image based on the recognition algorithm stored locally on the server to obtain the device information corresponding to each device to be controlled.

[0076] S102. Send the scene image to the terminal; the terminal is used to display the scene image and receive input operation instructions.

[0077] In some embodiments, the terminal is a user-side device. After receiving the scene image corresponding to the preset area, the server can directly send the scene image to the terminal. During actual deployment, the terminal can display the scene image to the user through a preset display device. Simultaneously, the terminal is also equipped with an input device, which can receive user input commands based on the scene image while the terminal is displaying it.

[0078] In some embodiments, the input device may be a touch screen. In this embodiment, the input device and the display device are the same device. During the display of the scene image on the touch screen, the user's touch operation can be received through the touch screen, and the touch operation can be used as the user's input operation command. For example, the touch operation may be a click operation, a drag operation, a selection operation, etc.

[0079] In some embodiments, the input device may be a mouse. In this embodiment, during the display of the scene image on the display device, the user's mouse operations can be received through the mouse, and these mouse operations can be used as user input commands. For example, the mouse operation may be a click operation, a drag operation, a selection box operation, etc.

[0080] In some embodiments, the input device may be a voice input device. During the display of the scene image on the display device, the user's control voice can be received through the voice input device, and the control voice can be converted into control text based on speech recognition technology. Then, a corresponding operation command is generated based on the control text. For example, the user can say "Device A starts working," and after the terminal converts the control voice into control text, it can determine the target device to be controlled in the control text as "Device A" based on a preset device library, and the corresponding control command is "Start working."

[0081] S103. Receive the operation instruction sent by the terminal, and based on the operation instruction and the device information of each device to be controlled, determine the target device to be controlled and the control instruction corresponding to the target device to be controlled.

[0082] In some embodiments, after receiving an operation instruction input by the user based on the scene image, the terminal sends the operation instruction to the server. Accordingly, after receiving the operation instruction, the server parses the operation instruction and determines the target control device and the control instruction corresponding to the target control device based on the device information of each control device.

[0083] In some embodiments, the operation instruction may include at least one of the following: a selection sub-instruction and an action sub-instruction. The selection sub-instruction may be used to determine a target device to be controlled from the at least one device to be controlled described above; the action sub-instruction may be used to determine the action that the target device to be controlled needs to perform.

[0084] S104. Send the control command to the target device to be controlled, wherein the control command is used to instruct the target device to be controlled to perform the action corresponding to the control command.

[0085] In this embodiment, by sending the scene image corresponding to the preset area to the terminal and displaying it to the user, and simultaneously receiving the user's real-time operation instructions based on the scene image, and sending corresponding control instructions to the device to be controlled in the preset area, this disclosure achieves a remote control solution for the device to be controlled, which not only reduces material costs and saves manpower costs, but also avoids the risks of management personnel entering the site, compared to the traditional approach where managers need to be fully equipped to enter the site to perform related operations. Furthermore, since the control instructions generated based on the recognition results corresponding to the scene image and the operation instructions received by the terminal can be used to determine the target device to be controlled from multiple devices to be controlled in the preset area, the control accuracy of the remote control method in this embodiment is improved. Through the control instructions generated based on the operation instructions in this embodiment, the target device to be controlled can perform actions corresponding to the user's real-time operations, achieving a more user-friendly interaction method.

[0086] Figure 2 This is a flowchart illustrating a remote control method provided in an embodiment of the present disclosure, as shown below. Figure 2 As shown, based on Figure 1 S103 may include S201 to S203, which will combine Figure 2 The steps shown are explained.

[0087] S201. Receive the selection sub-instruction sent by the terminal.

[0088] S202. Parse the selection sub-instruction to obtain selection information.

[0089] S203. Based on the selection information and the device information of each of the devices to be controlled, determine the target device to be controlled from the at least one device to be controlled.

[0090] In some embodiments, the selection information includes the selection coordinates of the target controllable device input by the user, and the device information includes the relative coordinates of the controllable device. The above-described determination of the target controllable device from the at least one controllable device based on the selection information and the device information of each controllable device can be achieved through S2031.

[0091] S2031. Based on the selected coordinates of the target controllable device and the relative coordinates of each of the controllable devices, determine the target controllable device among the at least one controllable device.

[0092] In some embodiments, the selected coordinates include first selected coordinates of the target controllable device based on the image coordinate system, and the relative coordinates include first relative coordinates of the controllable device based on the image coordinate system; determining the target controllable device among the at least one controllable device based on the selected coordinates of the target controllable device and the relative coordinates of each controllable device includes:

[0093] The target device to be controlled is determined among the at least one device to be controlled based on the first selected coordinates of the target device to be controlled based on the image coordinate system and the first relative coordinates of each device to be controlled based on the image coordinate system.

[0094] In some embodiments, the selected coordinates include a second selected coordinate of the target controllable device based on the regional coordinate system; determining the target controllable device among the at least one controllable device based on the selected coordinates of the target controllable device and the relative coordinates of each controllable device includes:

[0095] The target device to be controlled is determined among the at least one device to be controlled based on the second selected coordinates of the target device to be controlled based on the regional coordinate system and the second relative coordinates of each device to be controlled based on the regional coordinate system.

[0096] In some embodiments, the selection information includes a target identifier of the target controllable device input by the user, and the relative coordinates include a first relative coordinate of the controllable device based on the image coordinate system. The above-described determination of the target controllable device from the at least one controllable device based on the selection information and device information of each controllable device can be implemented via S2032 to S2033, including:

[0097] S2032. Based on the first relative coordinates corresponding to each of the devices to be controlled and the first identifier coordinates corresponding to at least one first visual identifier obtained by identifier detection of the scene image, determine the first visual identifier corresponding to each of the devices to be controlled.

[0098] The above S2032 can be achieved by: performing identifier detection on the scene image based on a preset identifier recognition model to obtain the first identifier coordinates corresponding to each of the at least one visual identifiers corresponding to the scene image; and determining the first visual identifier corresponding to each of the devices to be controlled based on the first identifier coordinates corresponding to each of the first visual identifiers and the first relative coordinates corresponding to each device to be controlled.

[0099] S2032 can also be implemented in the following way: based on the first identifier coordinates corresponding to each of the devices to be controlled, the scene image is segmented to obtain a device sub-image corresponding to each of the devices to be controlled; based on a preset identifier recognition model, the identifier of each device to be controlled is detected to determine the first visual identifier corresponding to each device to be controlled.

[0100] S2033. Based on the target identifier and the first visual identifier corresponding to each of the devices to be controlled, determine the target device to be controlled among the at least one device to be controlled.

[0101] Wherein, if the target identifier is a target visual identifier, the device to be controlled corresponding to the first visual identifier that matches the target visual identifier is determined as the target device to be controlled.

[0102] In the case where the target identifier is a target device identifier, based on the mapping relationship between device identifiers and visual identifiers, the device to be controlled corresponding to the first visual identifier that matches the target device identifier is determined as the target device to be controlled.

[0103] In this embodiment, by using the user-inputted selection coordinates of the target controllable device and the relative coordinates of each controllable device obtained from the scene image recognition, the target controllable device is determined from among multiple controllable devices in the scene image, achieving high precision in remote control. Furthermore, by using the user-inputted target identifier of the target controllable device and the relative coordinates of each controllable device obtained from the scene image recognition, the target controllable device can be determined from among multiple controllable devices even when the user only inputs identifier information, achieving high precision in remote control. In this embodiment, based on various information of the target controllable device input by the user (first selection coordinates, second selection coordinates, visual identifiers, and device identifiers), the target controllable device can be determined from among multiple controllable devices, providing rich interactive methods for remote control while ensuring precise control.

[0104] See Figure 3 , Figure 3 This is an optional flowchart illustrating a remote control method provided in an embodiment of this disclosure. Based on any of the above embodiments, it is based on... Figure 1 For example, the method further includes S301, and S102 can be updated to S302, combining... Figure 3 The steps shown are explained.

[0105] S301. Based on the coordinate transformation relationship between the image coordinate system and the region coordinate system corresponding to the preset region, the first relative coordinate of each device to be controlled based on the image coordinate system is converted into the second relative coordinate of each device to be controlled based on the region coordinate system.

[0106] In some embodiments, the image coordinate system is a coordinate system established based on the scene image, and the first relative coordinate represents the relative position of the device to be controlled in the scene image; the region coordinate system is a coordinate system established based on the preset region, and the second relative coordinate represents the relative position of the device to be controlled in the preset region.

[0107] S302, Send the scene image and the second relative coordinates of each of the devices to be controlled based on the regional coordinate system to the terminal; the second relative coordinates are used to indicate the real position of each device to be controlled in the preset area during the process of displaying the scene image on the terminal.

[0108] In some embodiments, the server sends the scene image to the terminal along with the second relative coordinates corresponding to each controllable device in the scene image. The terminal can display the actual position of each controllable device within a preset area while displaying the scene image, allowing the user to obtain the actual positions of all controllable devices and thus provide accurate control commands.

[0109] In this embodiment of the disclosure, based on the coordinate transformation relationship between the established image coordinate system and the region coordinate system, the first relative coordinates corresponding to each device to be controlled are converted into second relative coordinates representing the real position of the device to be controlled, and sent to the terminal at the same time as the scene image. This allows the user to grasp the real position of all devices to be controlled, thereby improving the efficiency of remote control.

[0110] See Figure 4 , Figure 4 This is an optional flowchart illustrating a remote control method provided in an embodiment of this disclosure. Based on any of the above embodiments, it is based on... Figure 1 For example, S103 may include S401 to S403, which will combine Figure 4 The steps shown are explained.

[0111] S401. Receive the action sub-instruction sent by the terminal.

[0112] In some embodiments, the action sub-instruction includes at least one of the following: movement sub-instruction, turning sub-instruction, and operation sub-instruction.

[0113] The movement sub-instruction carries movement information, which includes at least one of the following: target movement position, movement speed, and movement trajectory; the turning sub-instruction carries turning information, which includes at least one of the following: target turning angle, turning speed, and turning trajectory; the operation sub-instruction carries operation information, which may include at least one of the following: operation status, operation target, operation time, and operation content.

[0114] S402. Parse the action sub-instruction to obtain action information.

[0115] S403. Based on the action information, generate control commands corresponding to the target controllable device.

[0116] In some embodiments, the above-mentioned generation of control commands corresponding to the target controllable device based on the action information can be achieved through S4031 to S4032.

[0117] S4031. When the action information is based on the image coordinate system, the action information is converted into target action information based on the transformation relationship between the image coordinate system and the region coordinate system; when the action information is based on the region coordinate system, the action information is used as the target action information.

[0118] In some embodiments, when a user inputs an action sub-instruction based on the scene image, the action information carried in the action sub-instruction is based on the image coordinate system. For example, during the display of the scene image on the touch screen, after the user performs a touch operation on the scene image of the controllable device, the terminal can receive an action sub-instruction based on the scene image, i.e., the action information carried is based on the image coordinate system. As another example, during the display of the scene image, after the user uses a mouse to control the corresponding mouse cursor to perform mouse operations on the scene image, the terminal can receive an action sub-instruction based on the scene image, i.e., the action information carried is based on the image coordinate system. In this case, based on the transformation relationship between the image coordinate system and the region coordinate system, the action information can be converted into target action information based on the region coordinate system. The control command generated based on this target action information can be recognized by the controllable device in the real scene, thereby enabling it to perform a corresponding action.

[0119] In some embodiments, when a user inputs an action sub-command based on a preset area, the action information carried in the action sub-command is based on the coordinate system of that area. For example, a voice input device can receive control voice based on the preset area; for instance, the control voice could be "Device A moves to the center of the area" or "Device A turns to due south." In this case, the action information can be directly used as the target action information, and the control command generated based on the target action information can be recognized by the device to be controlled in the real scene, thereby performing the corresponding action.

[0120] S4032. Generate control commands corresponding to the target controllable device based on the target motion information.

[0121] In this embodiment of the disclosure, since the operation instruction also includes an action sub-instruction, the server, upon receiving the action sub-instruction, can generate a corresponding control instruction based on the action information carried in the action sub-instruction to achieve precise control of the target device to be controlled.

[0122] See Figure 5 , Figure 5 This is an optional flowchart illustrating a remote control method provided in an embodiment of this disclosure. Based on any of the above embodiments, it is based on... Figure 1 For example, the method further includes S501, and S104 can be updated to S502, combining... Figure 5The steps shown are explained.

[0123] S501. Based on the mapping relationship between device identifier and communication address, obtain the communication address corresponding to the target device to be controlled.

[0124] In some embodiments, the server pre-configures a device identifier corresponding to each device to be controlled and a communication address corresponding to each device identifier. After determining the target device to be controlled, the communication address of the target device to be controlled can be determined based on the device identifier corresponding to the target device to be controlled. The communication address can be the IP address, MAC address, etc., of the device to be controlled.

[0125] S502. Based on the communication address corresponding to the target device to be controlled, send the control command to the target device to be controlled.

[0126] In this embodiment of the disclosure, the control command can be accurately sent to the target device through the communication address corresponding to the target device to be controlled. Compared with the broadcast method, this reduces channel occupation and avoids the situation where other devices to be controlled receive the command incorrectly, thereby improving system stability.

[0127] Please see Figure 6 , Figure 6 This is an optional flowchart illustrating a remote control method provided in an embodiment of this disclosure, which will be combined with... Figure 6 The steps shown are explained.

[0128] S601, The camera device sends the captured scene images to the server.

[0129] S602. The server receives the scene image and identifies the scene image to obtain device information for each device to be controlled in the scene image.

[0130] After receiving the scene image, the server can use a preset object recognition model to detect the controllable devices in the scene image, thereby obtaining the detection box corresponding to each controllable device. Based on the detection box corresponding to each controllable device, the first relative coordinates of each controllable device in the scene image can be determined.

[0131] In another embodiment, based on the detection frame corresponding to each device to be controlled, the real-time sub-image corresponding to each device to be controlled can be determined. By using a preset identifier recognition model to identify the real-time sub-image corresponding to each device to be controlled, the visual identifier corresponding to each device to be controlled can be obtained. Based on the preset mapping relationship between the visual identifier and the device identifier, the device identifier corresponding to each device to be controlled can be obtained.

[0132] In another embodiment, the scene image can be directly identified based on a preset identifier recognition model to obtain at least one visual identifier and the identifier position corresponding to each visual identifier; based on the identifier position corresponding to each visual identifier and the first relative coordinate of each device to be controlled in the scene image, the visual identifier corresponding to each device to be controlled is determined; based on the preset mapping relationship between visual identifiers and device identifiers, the device identifier corresponding to each device to be controlled can be obtained.

[0133] S603. The server sends the scene image and the first relative coordinates of each of the devices to be controlled in the scene image to the terminal.

[0134] S604. The terminal receives the scene image and the first relative coordinates of each of the controlled devices in the scene image, and displays them.

[0135] S605. During the process of displaying the scene image, the terminal receives operation instructions input for the target control device.

[0136] In some embodiments, the operation instruction includes a selection sub-instruction for the target controllable device, the selection sub-instruction carrying selection information of the target controllable device; the selection information may be a device identifier / visual identifier of the target controllable device, a first selection coordinate of the target controllable device in the scene image, or a second selection coordinate of the target controllable device in a preset area.

[0137] In some embodiments, the operation instruction further includes an action sub-instruction for the target controllable device, the action sub-instruction carrying action information of the target controllable device. The action sub-instruction may be a movement instruction, the corresponding action information of which may be movement information corresponding to the target controllable device, and the movement information may include at least one of the following: target movement position, movement speed, and movement trajectory. Alternatively, the action sub-instruction may be a turning instruction, the corresponding action information of which may be turning information corresponding to the target controllable device, and the turning information may include at least one of the following: target turning angle, turning speed, and turning trajectory. Finally, the action sub-instruction may be a work instruction, the corresponding action information of which may be work information corresponding to the target controllable device, and the work information may include at least one of the following: work status, work target, work time, and work content.

[0138] S606. The terminal sends the operation command to the server.

[0139] In some embodiments, the terminal may send a selection sub-instruction and an action sub-instruction to the server. The selection sub-instruction is used to determine the target device to be controlled, and the action sub-instruction is used to determine the corresponding action to be performed on the target device to be controlled.

[0140] In some embodiments, the terminal can send action sub-instructions to the server. When no target device to be controlled is identified, all devices to be controlled within a preset area are considered as the target devices. In practical applications, this embodiment can be used to control the operating state of all devices to be controlled, such as switching all devices from an energy-saving state to a high-efficiency state, or switching all devices from an operating state to a sleep state.

[0141] S607. The server receives the operation instruction and determines the target device to be controlled and the control instruction corresponding to the target device to be controlled.

[0142] In some embodiments, when the operation instruction includes a selection sub-instruction, the server can determine the device identifier of the target controllable device based on the selection information of the target controllable device carried in the selection sub-instruction; then, based on a preset mapping relationship between device identifiers and communication addresses, the communication address of the target controllable device is determined. Wherein, when the selection information is the visual identifier of the target controllable device, the device identifier of the target controllable device can be determined based on the conversion relationship between the visual identifier and the device identifier; when the selection information is the first selection coordinate of the target controllable device in the scene image, based on the first relative coordinate of each controllable device in the scene image, the controllable device corresponding to the first relative coordinate matching the first selection coordinate is designated as the target controllable device. When the selection information is the second selection coordinate of the target controllable device in a preset area, based on the second relative coordinates of every two controllable devices in the preset area, the controllable device corresponding to the second relative coordinate matching the second selection coordinate is designated as the target controllable device.

[0143] In some embodiments, if the operation instruction does not include a selection sub-instruction, all devices to be controlled in the preset area can be identified as target devices to be controlled.

[0144] In some embodiments, when the operation instruction includes an action sub-instruction, the action sub-instruction is converted into the control instruction. For example, when the action sub-instruction is a movement instruction, a corresponding control instruction can be generated based on the information type of the movement information and the action sub-instruction. Wherein, when the information type of the movement information is based on the image coordinate system, the movement information can be converted based on the transformation relationship between the image coordinate system and the region coordinate system, and the converted movement information is used as the control instruction.

[0145] In some embodiments, the operation instruction further includes an action sub-instruction for the target controllable device, the action sub-instruction carrying action information of the target controllable device. The action sub-instruction may be a movement instruction, the corresponding action information of which may be movement information corresponding to the target controllable device, and the movement information may include at least one of the following: target movement position, movement speed, and movement trajectory. Alternatively, the action sub-instruction may be a turning instruction, the corresponding action information of which may be turning information corresponding to the target controllable device, and the turning information may include at least one of the following: target turning angle, turning speed, and turning trajectory. Finally, the action sub-instruction may be a work instruction, the corresponding action information of which may be work information corresponding to the target controllable device, and the work information may include at least one of the following: work status, work target, work time, and work content.

[0146] S608. The server sends a control command generated based on the operation information to the target device to be controlled.

[0147] S609. The target device to be controlled executes the action corresponding to the control command.

[0148] In this embodiment, by sending the scene image corresponding to the preset area to the terminal and displaying it to the user, and simultaneously receiving the user's real-time operation instructions based on the scene image, and sending corresponding control instructions to the device to be controlled in the preset area, the effect of remote control of the device to be controlled can be achieved. Furthermore, since the control instructions generated based on the recognition result corresponding to the scene image and the operation instructions received by the terminal can be used to determine the target device to be controlled from multiple devices to be controlled in the preset area, the control accuracy of the remote control method in this embodiment is improved. Through the control instructions generated based on the operation instructions in this embodiment, the target device to be controlled can perform actions corresponding to the user's real-time operation, achieving a more user-friendly interaction method.

[0149] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0150] Current intelligent monitoring is mostly unidirectional, with video streams transmitted from cameras to display screens or stored on hard drives for human viewing or program analysis. In practical applications, there are areas where it is inconvenient for people to directly enter, such as high-temperature environments, cold storage facilities, laboratories with stringent environmental requirements, and hazardous environments. To manage these areas, it is necessary not only to observe the conditions within these areas but also to perform operations such as moving or retrieving certain items located within them without human access.

[0151] To achieve the above effects, embodiments of this disclosure can combine the object detection function of monitoring equipment with a remotely controllable moving device pre-set within the area, enabling a WYSIWYG touch-screen remote control device. The remotely controllable moving device can be an identifiable vehicle, etc.

[0152] In related technologies, a single target is often controlled via a single control button or joystick, with the monitoring video only providing a visual display for the operator to observe the target. If there are multiple targets in the area, multiple joysticks or more complex control methods are required. To solve this problem, the embodiments of this disclosure utilize intelligent monitoring to identify each device to be controlled from the monitoring screen and obtain the location of each device. During the transmission of the monitoring screen to a touchscreen interface for display, the user can select the target device to be controlled and perform the corresponding operation. This achieves a WYSIWYG (What You See Is What You Get) effect and provides a more user-friendly interaction.

[0153] Please see Figure 7 , Figure 7 The illustration shows an optional implementation scenario provided by an embodiment of this disclosure. Figure 7 In the shown preset area A10, at least one controllable device A11 is distributed. This controllable device A11 can be connected to the server wirelessly / wiredly. At least one camera component A12 is also provided in the preset area A10. The camera component A12 can acquire scene images corresponding to the preset area A10. By analyzing the scene images, the relative position of each controllable device A11 in the scene image can be determined. Based on the coordinate mapping relationship between the scene image and the preset area, the relative position of each controllable device A11 within the preset area can be determined. In this embodiment, during the process of identifying the controllable devices contained in the scene image, each controllable device A11 can be distinguished by identifying a unique visual identifier set in each controllable device. For example, the visual identifier set in the controllable device can be a QR code, text, graphics, etc.

[0154] In some scenarios, the device to be controlled can be used to carry target objects, to divide the preset area, or to be combined with other devices to be controlled to obtain target devices of any shape.

[0155] Please see Figure 8 , Figure 8 This illustration shows an optional information flow diagram provided by an embodiment of the present disclosure. (To be combined with...) Figure 7 The schematic diagram of the implementation scenario illustrates the information transmission process in the embodiments of this disclosure.

[0156] In some embodiments, the at least one camera component A12 can send multiple consecutively acquired scene images as a video stream to the server A13. The server can then transmit the video stream carrying the multiple scene images to the terminal A14. The terminal A14 can display the video stream to the user and accept control commands from the user for any device to be controlled. These control commands include at least movement commands to change the location of the device to be controlled, turning commands to change the relative direction of the device to be controlled, and operation commands to change the target of the device to be controlled.

[0157] In some embodiments, after acquiring a scene image including the device A11 to be controlled, the camera component A12 directly analyzes the scene image to obtain the relative position of the device A11 to be controlled in the preset area, and sends the original scene image and the identified relative position of the device to be controlled to the server in the form of structured data, and forwards it to the terminal; or, in another embodiment, the camera component A12 can directly send the original scene image to the server, which analyzes the scene image to obtain the relative position of the device A11 to be controlled in the preset area, and sends the original scene image and the identified relative position of the device to be controlled to the terminal.

[0158] In the process of analyzing the scene image, each device to be controlled can be distinguished by identifying the visual identifiers set on each device. For example, if there are a first device and a second device to be controlled in a preset area, the scene image can be used to determine that the first device to be controlled has a first visual identifier and the second device to be controlled has a second visual identifier. Based on the preset correspondence between the visual identifiers and the devices to be controlled, the IDs of the first and second devices to be controlled can be determined. Based on the preset correspondence between the IDs and communication addresses, the communication address corresponding to each device to be controlled can be obtained, and communication between the target devices to be controlled can be achieved based on the communication address.

[0159] In some embodiments, after receiving the video stream carrying multiple frames of scene images, the terminal can display the video stream through a display component. Then, the terminal can receive a user's selection instruction for a target controllable device among the multiple controllable devices included in the scene images, and determine that target controllable device as the device the user needs to control at that moment.

[0160] The aforementioned display component is a touch screen with touch functionality. The terminal can receive touch operations from the user through the touch screen and, based on the touch operations, determine the target controllable device among multiple controllable devices included in the scene image. The terminal may also include an input component, such as a mouse, keyboard, or voice control device. The terminal can receive selection instructions from the user through the input component and, based on the selection instructions, determine the target controllable device among multiple controllable devices included in the scene image.

[0161] In some embodiments, the selection instruction received by the terminal carries a selection position for the display interface. The terminal can send the selection position to the server. Based on the selection position and the relative position of each device to be controlled, the server determines the target device to be controlled selected by the user, and then obtains the communication address corresponding to the target device to be controlled.

[0162] In some implementation scenarios, after a user selects the target device to be controlled, the terminal can receive the user's control operations on the target device to be controlled, obtain the corresponding control information, and send the control information to the server. The server forwards the control information to the target device to be controlled based on the communication address corresponding to the target device to be controlled, and the target device to be controlled can perform the corresponding operation based on the control information.

[0163] Figure 9 This is a schematic diagram of the composition structure of a remote control device provided in an embodiment of the present disclosure, as shown below. Figure 9 As shown, the remote control device 900 includes:

[0164] The acquisition module 901 is used to acquire a scene image corresponding to a preset area and a recognition result corresponding to the scene image; the recognition result includes device information of each of the at least one device to be controlled in the preset area.

[0165] The first sending module 902 is used to send the scene image to the terminal; the terminal is used to display the scene image and receive input operation instructions;

[0166] The receiving module 903 is used to receive the operation instruction sent by the terminal, and determine the target device to be controlled and the control instruction corresponding to the target device to be controlled based on the operation instruction and the device information of each device to be controlled.

[0167] The second sending module 904 is used to send the control command to the target device to be controlled, and the control command is used to instruct the target device to be controlled to perform the action corresponding to the control command.

[0168] In some embodiments, the operation instruction includes a selection sub-instruction, and the receiving module 903 is further configured to: receive the selection sub-instruction sent by the terminal; parse the selection sub-instruction to obtain selection information; and determine the target device to be controlled from the at least one device to be controlled based on the selection information and the device information of each device to be controlled.

[0169] In some embodiments, the selection information includes the input selection coordinates of the target device to be controlled, and the device information includes the relative coordinates of the device to be controlled. The receiving module 903 is further configured to: determine the target device to be controlled among the at least one device to be controlled based on the selection coordinates of the target device to be controlled and the relative coordinates of each device to be controlled.

[0170] In some embodiments, the selected coordinates include a first selected coordinate of the target device to be controlled based on the image coordinate system, and the relative coordinates include a first relative coordinate of the device to be controlled based on the image coordinate system; the receiving module 903 is further configured to: determine the target device to be controlled among the at least one device to be controlled based on the first selected coordinate of the target device to be controlled based on the image coordinate system and the first relative coordinate of each device to be controlled based on the image coordinate system.

[0171] In some embodiments, the selected coordinates include a second selected coordinate of the target device to be controlled based on the regional coordinate system; the receiving module 903 is further configured to: determine the target device to be controlled among the at least one device to be controlled based on the second selected coordinate of the target device to be controlled based on the regional coordinate system and the second relative coordinate of each device to be controlled based on the regional coordinate system.

[0172] In some embodiments, the device further includes a conversion module, which is configured to convert the first relative coordinates of each controllable device based on the image coordinate system into a second relative coordinate of each controllable device based on the region coordinate system, based on the coordinate transformation relationship between the image coordinate system and the region coordinate system corresponding to the preset region. The second sending module 904 is further configured to: send the scene image and the second relative coordinates of each controllable device based on the region coordinate system to the terminal; the second relative coordinates are used to indicate the true position of each controllable device in the preset region during the display of the scene image on the terminal.

[0173] In some embodiments, the selection information includes a target identifier of the target controllable device input by the user, and the relative coordinates include a first relative coordinate of the controllable device based on the image coordinate system. The receiving module 903 is further configured to: determine a first visual identifier corresponding to each controllable device based on the first relative coordinates corresponding to each controllable device and the first identifier coordinates corresponding to at least one first visual identifier obtained by identifier detection of the scene image; and determine the target controllable device among the at least one controllable devices based on the target identifier and the first visual identifier corresponding to each controllable device.

[0174] In some embodiments, the receiving module 903 is further configured to: perform identifier detection on the scene image based on a preset identifier recognition model to obtain a first identifier coordinate corresponding to each of the at least one visual identifiers corresponding to the scene image; determine a first visual identifier corresponding to each of the devices to be controlled based on the first identifier coordinate corresponding to each of the first visual identifiers and a first relative coordinate corresponding to each device to be controlled; or, segment the scene image based on the first identifier coordinate corresponding to each device to be controlled to obtain a device sub-image corresponding to each device to be controlled; and perform identifier detection on the device to be controlled corresponding to each device to be controlled based on the preset identifier recognition model to determine a first visual identifier corresponding to each device to be controlled.

[0175] In some embodiments, the receiving module 903 is further configured to: when the target identifier is a target visual identifier, determine the device to be controlled corresponding to the first visual identifier that matches the target visual identifier as the target device to be controlled; when the target identifier is a target device identifier, determine the device to be controlled corresponding to the first visual identifier that matches the target device identifier as the target device to be controlled based on the mapping relationship between the device identifier and the visual identifier.

[0176] In some embodiments, the operation instruction includes an action sub-instruction; the receiving module 903 is further configured to: receive the action sub-instruction sent by the terminal; parse the action sub-instruction to obtain action information; and generate a control instruction corresponding to the target controllable device based on the action information.

[0177] In some embodiments, the receiving module 903 is further configured to: convert the motion information into target motion information based on the transformation relationship between the image coordinate system and the region coordinate system when the motion information is based on the image coordinate system; use the motion information as the target motion information when the motion information is based on the region coordinate system; and generate control instructions corresponding to the target controllable device based on the target motion information.

[0178] In some embodiments, the action sub-instruction includes at least one of the following: a movement sub-instruction, a turning sub-instruction, and a work sub-instruction; wherein, the movement sub-instruction carries movement information, which includes at least one of the following: target movement position, movement speed, and movement trajectory; the turning sub-instruction carries movement information, which includes at least one of the following: target turning angle, turning speed, and turning trajectory; the work sub-instruction carries movement information, which may include at least one of the following: work status, work target, work time, and work content.

[0179] In some embodiments, the acquisition module 901 is further configured to: receive the scene image and the recognition result corresponding to the scene image sent by the camera device; the recognition result is generated by the camera device based on the scene image; or, receive the scene image sent by the camera device; and generate the recognition result based on the scene image.

[0180] In some embodiments, the conversion module is further configured to: obtain the communication address corresponding to the target device to be controlled based on the mapping relationship between the device identifier and the communication address; the second sending module 904 is further configured to: send the control command to the target device to be controlled based on the communication address corresponding to the target device to be controlled.

[0181] The description of the above device embodiments is similar to that of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this disclosure, please refer to the description of the method embodiments of this disclosure for understanding.

[0182] It should be noted that, in the embodiments of this disclosure, if the above-described remote control method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a device to execute all or part of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a magnetic disk, or an optical disk. Thus, the embodiments of this disclosure are not limited to any specific hardware and software combination.

[0183] Figure 10 This is a schematic diagram of the hardware entity of a remote control device provided in an embodiment of the present disclosure, such as... Figure 10As shown, the hardware entity of the remote control device 1000 includes a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program that can run on the processor 1001, and the processor 1001 executes the program to implement the steps in the method of any of the above embodiments.

[0184] The memory 1002 stores computer programs that can run on the processor. The memory 1002 is configured to store instructions and applications that can be executed by the processor 1001. It can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 1001 and the various modules in the remote control device 1000. It can be implemented by flash memory or random access memory (RAM).

[0185] The processor 1001 executes the program to implement the steps of any of the above-mentioned remote control methods. The processor 1001 typically controls the overall operation of the remote control device 1000.

[0186] This disclosure provides a computer storage medium storing one or more programs that can be executed by one or more processors to implement the steps of the remote control method as described in any of the above embodiments.

[0187] It should be noted that the descriptions of the storage medium and device embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0188] The aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this disclosure does not specifically limit the specific implementation.

[0189] The aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0190] It should be understood that the terms "an embodiment," "an embodiment," "an embodiment of this disclosure," "the foregoing embodiments," or "some embodiments" throughout the specification mean that the target features, structures, or characteristics related to the embodiment are included in at least one embodiment of this disclosure. Therefore, the phrases "in an embodiment," "an embodiment," "an embodiment of this disclosure," "the foregoing embodiments," or "some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these target features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this disclosure, the sequence numbers of the above processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above embodiments of this disclosure are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0191] Unless otherwise specified, any step in the embodiments of this disclosure can be performed by the processor of the remote control device. Unless otherwise specified, the embodiments of this disclosure do not limit the order in which the remote control device performs the following steps. Furthermore, the methods used to process data in different embodiments may be the same or different methods. It should also be noted that any step in the embodiments of this disclosure can be performed independently by the remote control device; that is, when the remote control device performs any step in the above embodiments, it may not depend on the execution of other steps.

[0192] In the several embodiments provided in this disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0193] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0194] In addition, each functional unit in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0195] The methods disclosed in the several method embodiments provided in this disclosure can be arbitrarily combined without conflict to obtain new method embodiments.

[0196] The features disclosed in the several product embodiments provided in this disclosure can be combined arbitrarily without conflict to obtain new product embodiments.

[0197] The features disclosed in the several method or device embodiments provided in this disclosure can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0198] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0199] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this disclosure, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a remote control device, or a network device, etc.) to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0200] In the embodiments of this disclosure, descriptions of the same steps and contents in different embodiments can be referred to each other. In the embodiments of this disclosure, the term "and" does not affect the order of steps.

[0201] The above description is merely an embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A remote control method, characterized in that, The method includes: Acquire a scene image corresponding to a preset region and a recognition result corresponding to the scene image; the recognition result includes device information of each of the at least one device to be controlled in the preset region. The scene image is sent to the terminal; the terminal is used to display the scene image and receive input operation instructions; the operation instructions include selection sub-instructions; The system receives the operation command sent by the terminal, and determines the target device to be controlled and the corresponding control command based on the operation command and the device information of each device to be controlled; the device information includes the relative coordinates of the device to be controlled. The control command is sent to the target device to be controlled, and the control command is used to instruct the target device to perform the action corresponding to the control command. The step of receiving the operation command sent by the terminal and determining the target device to be controlled based on the operation command and the device information of each device to be controlled includes: Receive a selection sub-instruction sent by the terminal; parse the selection sub-instruction to obtain selection information; the selection information includes the target identifier of the target device to be controlled; based on the selection information and the device information of each device to be controlled, determine the target device to be controlled from the at least one device to be controlled; The relative coordinates include the first relative coordinates of the device to be controlled based on the image coordinate system. The step of determining the target device to be controlled from the at least one device to be controlled based on the selection information and the device information of each device to be controlled includes: Based on the first relative coordinates corresponding to each of the devices to be controlled and the first identifier coordinates corresponding to at least one first visual identifier obtained by identifier detection of the scene image, a first visual identifier corresponding to each of the devices to be controlled is determined; based on the target identifier and the first visual identifier corresponding to each device to be controlled, the target device to be controlled is determined among the at least one device to be controlled.

2. The method according to claim 1, characterized in that, The step of determining the target device to be controlled from the at least one device to be controlled based on the selection information and the device information of each device to be controlled further includes: Based on the selected coordinates of the target device to be controlled and the relative coordinates of each of the devices to be controlled, the target device to be controlled is determined among the at least one device to be controlled.

3. The method according to claim 2, characterized in that, The selected coordinates include first selected coordinates of the target controllable device based on the image coordinate system, and the relative coordinates include first relative coordinates of the controllable device based on the image coordinate system; determining the target controllable device among the at least one controllable device based on the selected coordinates of the target controllable device and the relative coordinates of each controllable device includes: The target device to be controlled is determined among the at least one device to be controlled based on the first selected coordinates of the target device to be controlled based on the image coordinate system and the first relative coordinates of each device to be controlled based on the image coordinate system.

4. The method according to claim 3, characterized in that, The selected coordinates include a second selected coordinate based on the regional coordinate system of the target control device; determining the target control device among the at least one control device based on the selected coordinates of the target control device and the relative coordinates of each control device includes: The target device to be controlled is determined among the at least one device to be controlled based on the second selected coordinates of the target device to be controlled based on the regional coordinate system and the second relative coordinates of each device to be controlled based on the regional coordinate system.

5. The method according to claim 4, characterized in that, The method further includes: based on the coordinate transformation relationship between the image coordinate system and the region coordinate system corresponding to the preset region, converting the first relative coordinate of each device to be controlled based on the image coordinate system into the second relative coordinate of each device to be controlled based on the region coordinate system; Sending the scene image to the terminal includes: sending the scene image and a second relative coordinate of each of the devices to be controlled based on the regional coordinate system to the terminal; the second relative coordinate is used to indicate the actual position of each device to be controlled in the preset area during the display of the scene image on the terminal.

6. The method according to claim 1, characterized in that, The step of determining the first visual identifier corresponding to each of the controlled devices based on the first relative coordinates corresponding to each of the controlled devices and the first identifier coordinates corresponding to at least one first visual identifier obtained by identifier detection of the scene image includes any one of the following: Based on a preset identifier recognition model, the scene image is detected to obtain the first identifier coordinates of each visual identifier in at least one visual identifier corresponding to the scene image; based on the first identifier coordinates of each first visual identifier and the first relative coordinates of each device to be controlled, the first visual identifier corresponding to each device to be controlled is determined. Based on the first identifier coordinates corresponding to each of the controlled devices, the scene image is segmented to obtain a device sub-image corresponding to each of the controlled devices. Based on a preset identifier recognition model, identifier detection is performed on each device to be controlled to determine the first visual identifier corresponding to each device to be controlled.

7. The method according to claim 6, characterized in that, The step of determining the target device to be controlled among the at least one device to be controlled based on the target identifier and the first visual identifier corresponding to each device to be controlled includes: When the target identifier is a target visual identifier, the device to be controlled corresponding to the first visual identifier that matches the target visual identifier is determined as the target device to be controlled; When the target identifier is a target device identifier, based on the mapping relationship between device identifiers and visual identifiers, the device to be controlled corresponding to the first visual identifier that matches the target device identifier is determined as the target device to be controlled.

8. The method according to any one of claims 1 to 7, characterized in that, The operation instruction includes action sub-instructions; receiving and parsing the operation instruction sent by the terminal to determine the control instruction corresponding to the target controllable device includes: Receive the action sub-instruction sent by the terminal; Parse the action sub-instruction to obtain action information; Based on the action information, control commands corresponding to the target controllable device are generated.

9. The method according to claim 8, characterized in that, The step of generating control commands corresponding to the target controllable device based on the action information includes: When the motion information is based on the image coordinate system, the motion information is converted into target motion information based on the transformation relationship between the image coordinate system and the region coordinate system; when the motion information is based on the region coordinate system, the motion information is used as the target motion information. Based on the target motion information, control commands corresponding to the target controllable device are generated.

10. The method according to claim 8, characterized in that, The action sub-instructions include at least one of the following: movement sub-instructions, turning sub-instructions, and operation sub-instructions; wherein... The movement sub-command carries movement information, which includes at least one of the following: target movement position, movement speed, and movement trajectory. The steering sub-command carries action information that is steering information, which includes at least one of the following: target steering angle, steering speed, and steering trajectory; The action information carried by the sub-instruction is the task information, which includes at least one of the following: task status, task objective, task time, and task content.

11. The method according to any one of claims 1 to 7, characterized in that, The step of obtaining the scene image corresponding to the preset region and the recognition result corresponding to the scene image includes any one of the following: The system receives the scene image and the corresponding recognition result sent by the camera device; the recognition result is generated by the camera device based on the scene image. Receive the scene image sent by the camera device; The recognition result is generated based on the scene image.

12. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Based on the mapping relationship between device identifier and communication address, obtain the communication address corresponding to the target device to be controlled; Sending the control command to the target device to be controlled includes: Based on the communication address corresponding to the target device to be controlled, the control command is sent to the target device to be controlled.

13. A remote control device, characterized in that, include: The acquisition module is used to acquire the scene image corresponding to the preset area and the recognition result corresponding to the scene image; The identification result includes device information for each of the at least one device to be controlled in the preset area; The first sending module is used to send the scene image to the terminal; The terminal is used to display the scene image and receive input operation instructions; the operation instructions include selection sub-instructions; A receiving module is configured to receive the operation command sent by the terminal, and determine the target device to be controlled and the control command corresponding to the target device to be controlled based on the operation command and the device information of each device to be controlled; the device information includes the relative coordinates of the device to be controlled; The second sending module is used to send the control command to the target device to be controlled, and the control command is used to instruct the target device to be controlled to perform the action corresponding to the control command. The receiving module is further configured to receive a selection sub-instruction sent by the terminal; parse the selection sub-instruction to obtain selection information; the selection information includes the target identifier of the target device to be controlled input. Based on the selection information and the device information of each of the devices to be controlled, the target device to be controlled is determined from the at least one device to be controlled; The relative coordinates include the first relative coordinates of the device to be controlled based on the image coordinate system. The receiving module is further configured to determine the first visual identifier corresponding to each device to be controlled based on the first relative coordinates corresponding to each device to be controlled and the first identifier coordinates corresponding to at least one first visual identifier obtained by identifier detection of the scene image. Based on the target identifier and the first visual identifier corresponding to each of the devices to be controlled, the target device to be controlled is determined among the at least one device to be controlled.

14. A remote control device, characterized in that, include: Memory and processor The memory stores computer programs that can run on the processor. When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 12.

15. A computer storage medium, characterized in that, The computer storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the method according to any one of claims 1 to 12.

Images