Robot guiding task point control method and related device

Abstract

The invention discloses a control method and a related device for guiding task points by a robot holder, and relates to the field of information processing, and the method comprises the steps: obtaining a field image which is an image used for constructing a target three-dimensional scene; obtaining a target image acquisition pose of a virtual robot corresponding to a target task point in the target three-dimensional scene; obtaining a target image corresponding to the target image acquisition pose according to the field image; and correction information is obtained according to the target image, and the correction information is used for correcting an image acquisition pose of a real robot when the real robot detects the target task point in a real scene.

Description

Technical Field

[0001] The present application relates to the field of information processing, and in particular to a control method and related devices for robot guidance task points. Background Art

[0002] A pan-tilt head is a device used to fix and adjust the position of a robot or other equipment. In the field of robotics, pan-tilt heads are widely used in various service, industrial and scientific research robots to adjust the robot's posture and position so that it can better complete tasks.

[0003] As an emerging technology, the metaverse can integrate a variety of new technologies to produce new virtual-real Internet applications and social forms, and use the metaverse to build a platform to achieve the task of controlling the robot gimbal guidance.

[0004] The robot gimbal guidance task process under the Metaverse construction platform can include: using LiDAR (Light Detection And Ranging) to scan the on-site environment; editing the robot task points on the Metaverse construction platform, and using optional gimbal guidance tasks to increase fault tolerance; fine-tuning according to actual conditions, on-site real-machine debugging; and real-machine deployment.

[0005] In the above process, the position, orientation and virtual gimbal parameters of the virtual robot twin body need to be adjusted to appropriate parameters according to the virtual environment objects in the Metaverse platform. However, due to the errors between the environment constructed by virtual reality and the real reality environment, as well as the positioning of the robot when performing tasks and the errors in the operation of the mechanical structure, the deployed robot cannot reach the absolute point and configuration parameters of the task editing, resulting in a low task completion rate. Summary of the invention

[0006] The first aspect of the present application provides a control method for a robot guiding a task point, comprising:

[0007] Obtaining a scene image, wherein the scene image is an image used to construct a target three-dimensional scene;

[0008] Obtaining a target image acquisition posture of the virtual robot corresponding to a target task point in the target three-dimensional scene;

[0009] Obtaining a target image corresponding to the target image acquisition posture according to the scene image;

[0010] Correction information is obtained according to the target image, and the correction information is used to correct the image acquisition posture of the real robot when the real robot detects the target task point in a real scene.

[0011] In a possible implementation, the method further includes:

[0012] In response to receiving the issued instruction, the target image acquisition posture and the correction information are sent to the real robot, and the real robot can obtain a first image in a real scene based on the target image acquisition posture, and correct the image acquisition posture according to the first image and the correction information.

[0013] In a possible implementation, the scene image includes all images used to construct the target three-dimensional scene.

[0014] In a possible implementation, obtaining the on-site image includes:

[0015] The scene image corresponding to the target task point is obtained from a scene image set, where the scene image set is a set of images used to construct the target three-dimensional scene.

[0016] In a possible implementation, obtaining a target image corresponding to the target image acquisition posture according to the scene image includes:

[0017] Importing the scene image into the target three-dimensional scene according to the shooting posture of the scene image to obtain a scene image three-dimensional scene;

[0018] Determining the image acquisition range of the virtual robot in the target three-dimensional scene according to the target image acquisition posture;

[0019] The target image is obtained by rendering an image corresponding to the image acquisition range according to the three-dimensional scene of the on-site image.

[0020] In a possible implementation, obtaining a target image corresponding to the target image acquisition posture according to the scene image includes:

[0021] Obtaining a shooting posture of the scene image;

[0022] Based on the scene image and the shooting posture, establishing a corresponding relationship between the target image acquisition posture and the shooting posture;

[0023] According to the corresponding relationship, determining the interception range corresponding to the target image acquisition posture in the scene image;

[0024] According to the interception range, a target image is intercepted in the scene image.

[0025] In a possible implementation, the target image acquisition posture includes:

[0026] The position coordinates of the virtual robot in the target three-dimensional scene and the orientation angle of the virtual robot pan / tilt.

[0027] A second aspect of the present application provides a control device for a robot guiding a task point, comprising:

[0028] A scene image acquisition module, used to acquire a scene image, wherein the scene image is an image used to construct a target three-dimensional scene;

[0029] An acquisition posture acquisition module is used to obtain a target image acquisition posture of the virtual robot corresponding to a target task point in the target three-dimensional scene;

[0030] A target image acquisition module, used to obtain a target image corresponding to the target image acquisition posture according to the scene image;

[0031] A correction information acquisition module is used to obtain correction information based on the target image, and the correction information is used to correct the image acquisition posture of the real robot when the real robot detects the target task point in a real scene.

[0032] The third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the control method of the robot guiding task point of the above-mentioned first aspect or any implementation method of the first aspect.

[0033] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0034] The memory is used to store computer programs;

[0035] The processor is used to execute the computer program so that the electronic device can implement the control method of the robot guiding task point of the above-mentioned first aspect or any implementation method of the first aspect.

[0036] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the control method of the robot guiding task point of the above-mentioned first aspect or any implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0038] Figure 1 It is a structural schematic diagram of the control system of the robot guidance task point provided by the present application;

[0039] Figure 2 A schematic diagram of a flow chart of a control method for a robot guiding a task point provided in an embodiment of the present application;

[0040] Figure 3 is a schematic diagram of a target image and correction information provided in an embodiment of the present application;

[0041] Figure 4 is a first image schematic diagram provided in an embodiment of the present application;

[0042] Figure 5 It is a schematic diagram of a process of obtaining a target image corresponding to the target image acquisition posture according to the scene image provided by an embodiment of the present application;

[0043] Figure 6 is a schematic diagram of a virtual robot provided in an embodiment of the present application determining an image acquisition range in a target three-dimensional scene;

[0044] Figure 7 is a schematic diagram of an image corresponding to an image acquisition range provided in an embodiment of the present application;

[0045] Figure 8 It is a schematic diagram of a process of obtaining a target image corresponding to the target image acquisition posture according to the scene image provided by an embodiment of the present application;

[0046] Fig. 9 It is a structural schematic diagram of a control device for a robot guiding a task point provided in an embodiment of the present application;

[0047] Fig.10 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0049] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0050] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0051] In order to facilitate understanding of the solution of the present application, the control system of the robot guidance task point of the present application is first introduced below.

[0052] Figure 1 is a schematic diagram of the structure of a control system for a robot-guided task point provided by the present application. The system of this embodiment may include: a platform device 101 and a real robot 102;

[0053] Among them, the platform equipment can use on-site images to build a virtual three-dimensional scene, and build a virtual robot in the virtual three-dimensional scene. By collecting postures for the deployment of the virtual robot at the task point in the virtual three-dimensional scene, the collected posture information is deployed to the real robot.

[0054] The real robot 102 is provided with a moving device 1021 and a pan-tilt platform 1022 . The moving device can drive the real robot to move according to the acquisition posture, and the pan-tilt platform can acquire images of the surrounding environment of the real robot.

[0055] In the present application, correction information is determined using on-site images, and the correction information can instruct a real robot to correct its image acquisition posture when performing detection in a real scene to ensure that the image it acquires matches the task point.

[0056] Figure 2 A flow chart of a control method for a robot guiding a task point provided in an embodiment of the present application may include steps 201 to 204, and these steps are described in detail below.

[0057] 201. Obtain a scene image, where the scene image is an image used to construct a target three-dimensional scene;

[0058] Among them, a control method for a robot guiding task point provided in this embodiment is applied to an electronic device, which serves as a robot control platform, for example, a metaverse construction platform can be adopted.

[0059] The on-site image can construct a target three-dimensional scene, which is a virtualization of the real scene.

[0060] The on-site image may be obtained by scanning using LiDAR (Light Detection And Ranging, also known as optical radar) technology and processed by the platform as visual data or non-visual data.

[0061] As an example, the visualization data may be in FBX (Filmbox) format.

[0062] The on-site image may be a plurality of frames of images used to stitch a panoramic image.

[0063] When constructing the target three-dimensional scene, the multiple frames of on-site images can be used to stitch a panoramic image, and the panoramic image can be used to construct the target three-dimensional scene.

[0064] In a possible implementation, the acquired on-site image includes all images used to construct the target three-dimensional scene.

[0065] Among them, all images of the target three-dimensional scene are obtained, and then the target image is obtained by using the all images.

[0066] In a possible implementation, obtaining a scene image includes: obtaining the scene image corresponding to the target task point from a scene image set, where the scene image set is a set of images used to construct the target three-dimensional scene.

[0067] Among them, the scene image set includes several frames of images obtained by scanning the real scene. In the scene image set, only some of the images may have relevant content of the task point. Therefore, in this embodiment, the scene image obtained can only obtain the scene image corresponding to the target task point to reduce the amount of image processing in the subsequent process of obtaining the target image.

[0068] Among them, a partial image of the target three-dimensional scene is obtained, and then the partial image is used to obtain the target image.

[0069] 202. Obtain a target image acquisition posture of the virtual robot corresponding to a target task point in the target three-dimensional scene;

[0070] After constructing the target three-dimensional scene, the platform sets the image acquisition posture set for the virtual robot for each task point in the target three-dimensional scene.

[0071] The image acquisition posture is the posture adopted by the virtual robot to acquire images at corresponding task points in the target three-dimensional scene.

[0072] Among them, the image acquisition posture is the acquisition posture of the virtual robot corresponding to the target task point in the target three-dimensional scene. Correspondingly, in the real scene, the acquisition posture of the real robot at the task point is the posture corresponding to the image acquisition posture.

[0073] In a possible implementation, the target image acquisition posture includes: the position coordinates of the virtual robot in the target three-dimensional scene and the orientation angle of the virtual robot gimbal.

[0074] The position coordinates may be XYθ, where XY is the position of the environment ground coordinate system, and θ represents the rotation angle.

[0075] The orientation angle may adopt PTZ, which is the abbreviation of Pan / Tilt / Zoom, representing the all-round (up and down, left and right) movement of the pan / tilt head and the zoom and zoom control of the lens.

[0076] In a possible implementation, the field data obtained by scanning the real scene is processed by the platform to obtain visualization data, and the visualization data is loaded by the engine through rendering and imported into the virtual environment under the visualization of the scanned content, and the collection posture of the virtual robot for each task point is generated in the virtual environment. Specifically, the posture XYθ of the virtual robot and the PTZ value of the virtual gimbal on the virtual robot are adjusted in the virtual environment (so that the virtual camera on the virtual robot will display the FBX screen seen), and the gimbal screen is adjusted to the appropriate position to complete the process of editing the task point.

[0077] In practical applications, for a certain scene, multiple, dozens or even hundreds of task points can be set, and the control method of the embodiment of the present application can be executed for each task point to obtain correction information for each task point.

[0078] 203. Obtaining a target image corresponding to the target image acquisition posture according to the scene image;

[0079] The on-site image is an image of a constructed target three-dimensional scene. Due to the influence of the accuracy of the constructed model, there will be a certain error between the target three-dimensional scene and the real scene.

[0080] Among them, since the target three-dimensional scene is constructed by the on-site image, the image content that can be captured by the virtual robot using the target image acquisition posture in the target three-dimensional scene can be captured in the on-site image according to the target image acquisition posture by utilizing the correspondence between the target three-dimensional scene and the on-site image.

[0081] Among them, the virtual robot uses the target image posture to perform image acquisition in the target three-dimensional scene. The acquisition range of the virtual robot corresponds to an area. The target image can correspond to the on-site image, and the image of the area corresponding to the acquisition range of the virtual robot is captured.

[0082] Among them, the target image is obtained by intercepting the scene image and is a part of the scene image. The target image is the image corresponding to the target task point in the scene image. The target task point is also arranged accordingly in the target three-dimensional scene. The target image is the part of the scene image in the real scene corresponding to the target task point in the target three-dimensional scene.

[0083] As an example, the task performed by the target task point is to collect detection data of a flow meter. The target image may be an image corresponding to the target task point, including the flow meter and the installation environment of the flow meter.

[0084] In a possible implementation, the live image is first imported into the target three-dimensional scene to obtain the live image three-dimensional scene, and the image acquisition range of the virtual robot in the target three-dimensional scene is determined, and the target image is obtained by using the image corresponding to the live image three-dimensional scene and the image acquisition range. Figure 5 The process of obtaining the target image is described in detail.

[0085] In a possible implementation, a corresponding relationship between the shooting posture of the scene image and the acquisition posture of the target image can be established first, and the target image can be intercepted from the scene image according to the corresponding relationship. Figure 8 The process of obtaining the target image is described in detail.

[0086] 204. Obtain correction information according to the target image, where the correction information is used to correct the image acquisition posture of the real robot when the real robot detects the target task point in the real scene.

[0087] Among them, when the real robot detects the target task point in the real scene, by collecting images of the target task point, the real robot may not be able to reach the absolute point position and configuration parameters of the task editing. The image collection posture of the real robot is not the point position of the task editing, and the configuration parameters implemented are inconsistent with the configuration parameters of the task editing. Therefore, the image taken by the real robot cannot support the completion of the task. The correction information is information for correcting the image collection posture of the real robot during detection.

[0088] Among them, the correction information corresponds to the points and configuration parameters of the task editing. When the real robot collects images for the target task points, if it reaches the absolute points and configuration parameters of the task editing, the collected images match the correction information.

[0089] Among them, the matching includes: the corresponding content of the correction information is in the central area of ​​the image captured by the real robot, the image captured by the real robot only contains the corresponding content of the correction information, etc. The specific matching method is not limited in this application.

[0090] As an example, the correction information may be specific information of the mission point, such as a mission nameplate set in the mission point.

[0091] Among them, the target image is captured from the on-site image, which contains the information of the target task point in the real scene. A part of the target image is captured as correction information, which can be used to compare with the image captured in the real scene to correct the image acquisition posture of the real robot.

[0092] Among them, the platform configures the target image acquisition posture determined in the target three-dimensional scene to the real robot, and the real robot uses the configured acquisition posture to reach the corresponding position in the real scene and then performs image acquisition.

[0093] The target image contains information about the real scene of the target task point in the on-site image, and a portion is cut out from the target image as correction information.

[0094] In a possible implementation, selection information may be received, where the selection information is for selecting a capture range of a target image, and the selection information is used to capture a portion of the content within the capture range of the target image, and correction information is captured.

[0095] The selection information may be a range manually input by a technician; or may be a range automatically selected for a portion of the target image that contains the characteristic information of the target task point based on the characteristic information of the target task point input in advance.

[0096] Figure 3 30 is a schematic diagram of a target image and correction information provided by an embodiment of the present application, wherein the target image 301 includes a target task point and surrounding environment, and the correction information 302 includes the target task point. The correction information is obtained by intercepting a part of the target image.

[0097] Among them, the Figure 3The target task point is a power cabinet (35kV1# station transformer cabinet). The target image 301 is an image including the power cabinet and the surrounding environment. Part of the content in the target image 301 is captured as correction information. The correction information 302 is the display area image of the power cabinet.

[0098] In this embodiment, a scene image for constructing a target three-dimensional scene is obtained, a target image acquisition posture of a virtual robot corresponding to a target task point in the target three-dimensional scene is obtained, a target image corresponding to the target image acquisition posture is obtained according to the scene image, correction information is obtained according to the target image, and the correction information is used to correct the image acquisition posture of the real robot when the real robot detects the target task point in the real scene, and the scene image for constructing the target three-dimensional scene is intercepted by using the target image acquisition posture of the virtual robot for the target task point in the target three-dimensional scene to obtain the target image, which is a part of the scene image, which is a part of the scene image corresponding to the target task point in the target three-dimensional scene in the real scene, and then correction information is obtained from the target image, the correction information corresponds to the real scene and is related to the target image acquisition posture, and the correction information can be used to realize the correction of the image acquisition posture of the real robot in the real scene when detecting the target task point, so as to ensure that the real robot can reach the absolute point position and configuration parameters of the task, thereby improving the task completion rate.

[0099] In a possible implementation, after obtaining the correction information, the method further includes:

[0100] In response to receiving the issued instruction, the target image acquisition posture and the correction information are sent to the real robot, and the real robot can obtain a first image in the real scene based on the target image acquisition posture, and correct the image acquisition posture according to the first image and the correction information.

[0101] Among them, the real robot can set up a recognition algorithm, and can use the correction information to train the recognition algorithm in the robot, so as to use the recognition algorithm to identify the first image taken, identify the relevant features of the target task point therein, and determine whether the relevant features of the target task point meet specific task completion conditions. If not, determine the correction parameters of the real robot.

[0102] Among them, another recognition algorithm can be set in the real robot, and the correction information and the first image obtained by shooting are used for recognition and comparison. When it is determined that the two correspond, it is judged whether the first image meets the task completion conditions. If not, the correction parameters of the real robot are determined using the correction information and the first image.

[0103] The recognition algorithm may be an image recognition algorithm, which can recognize features of an object in an image to obtain the image features of the object.

[0104] In a possible implementation, if the first image captured for a certain task point matches the corresponding correction information and the first image meets the task completion conditions, there is no need to adjust the image capture posture of the task point; if the first image captured for a certain task point matches the corresponding correction information, but the first image does not meet the task completion conditions, it is necessary to adjust the image capture posture of the task point; if the first image captured for a certain task point does not match the corresponding correction information, it is necessary to adjust the image capture posture of the task point.

[0105] It should be noted that due to the errors between the constructed target three-dimensional scene and the real environment, the positioning of the real robot when performing the task, and the errors in the operation of the mechanical structure, even if the two are superimposed, the error will generally not be large, and the first image collected by the real robot at a certain task point will not match the corresponding correction information. Therefore, generally, the first image collected for a certain task point matches the corresponding correction information, but the first image does not meet the task completion conditions, and the image collection posture of the task point needs to be adjusted.

[0106] As an example, the task completion condition is that the area image corresponding to the target task point in the first image is located in the central area.

[0107] Among them, the image corresponding to the task point in the first image can be at any position in the first image. If it is in the central area of ​​the first image, it is determined that the task completion conditions are met. Otherwise, the task completion conditions are not met. Of course, the first image does not contain the image corresponding to the task point, and it is determined that the task completion conditions are not met.

[0108] Among them, the task issuing instruction is generated when the operator of the operating platform triggers the issuing button. The purpose of the issuing instruction is to send the image acquisition posture corresponding to each task point configured for the virtual robot in the target three-dimensional scene to the real robot, so that the real robot can use the received image acquisition posture to perform image acquisition on the corresponding task point in the real scene to complete the task.

[0109] The image acquisition posture corresponding to each task point and its corresponding correction information are respectively sent to the real robot, so that the real robot can use the image acquisition posture to perform image acquisition on the corresponding task point to obtain the first image.

[0110] Figure 4It is a schematic diagram of the first image provided in an embodiment of the present application, wherein the first image 401 includes an image 402 corresponding to the task point, and the image of the task point includes a display area image of the 35kV1# station power cabinet. In (a), image 402 is located on the left side of the first image 401, and the task completion condition is not met; in (b), image 402 is located in the upper right corner of the first image 401, and the task completion condition is not met; in (c), image 402 is located in the center of the first image 401, and the task completion condition is met; in (d), the image 402 only includes a partial area of ​​the task point, and the image 402 is located on the left side of the first image, and the task completion condition is not met.

[0111] In one possible implementation, after the real robot adjusts its image acquisition posture, the adjusted image acquisition posture can be fed back to the platform so that the platform can refer to the adjusted image acquisition posture when subsequently constructing the target three-dimensional scene, thereby improving the accuracy of the target three-dimensional scene construction.

[0112] In a possible implementation, after the real robot adjusts its image acquisition posture, the adjusted image acquisition posture can be recorded. When the task point is subsequently executed, the adjusted image acquisition posture is used to reduce the data processing amount of the real robot.

[0113] In one possible implementation, the real robot uses its configured image acquisition posture and correction information to determine whether a correction process is needed each time the corresponding task point is executed, and performs correction when necessary. When the number of times that correction is not needed reaches a certain number, the image acquisition posture can be used directly to execute the corresponding task point without having to determine whether a correction process is needed, thereby reducing the data processing amount of the real robot.

[0114] In this embodiment, after obtaining the correction information and the target image acquisition posture, in response to receiving the issued instruction, the target image acquisition posture and the correction information are sent to the real robot. The real robot can capture a first image in the real scene based on the target image acquisition posture, and correct the image acquisition posture according to the first image and the correction information. In the real scene, the real robot automatically adjusts its image acquisition posture without the need for technical personnel to arrive at the site, thereby improving the efficiency of task completion.

[0115] Figure 5 It is a flowchart of obtaining a target image corresponding to the target image acquisition posture based on the scene image provided by an embodiment of the present application, which may include steps 501 to 503, and these steps are described in detail below.

[0116] 501. Importing the scene image into the target three-dimensional scene according to the shooting posture of the scene image to obtain a three-dimensional scene of the scene image;

[0117] The shooting posture is the shooting posture adopted by the image acquisition device when shooting live images in a real scene.

[0118] In a possible implementation, during the shooting process of each scene image, the shooting posture is recorded, and the shooting posture includes the position coordinates and orientation angle of the shooting device in the real scene. The position coordinates may include XYθ, and the orientation angle may adopt PTZ.

[0119] The shooting posture can be saved in the platform together with the on-site image.

[0120] Among them, according to the shooting posture of the scene image, it is imported into the target three-dimensional scene to obtain the scene image three-dimensional scene, which is the three-dimensional scene of the corresponding part of the scene image.

[0121] The three-dimensional scene of the on-site image corresponds to the real scene of the on-site image.

[0122] In this embodiment, the target three-dimensional scene may be imported only for a portion of the scene images corresponding to the task point in the scene image set to obtain the three-dimensional scene of the scene image.

[0123] 502. Determine the image acquisition range of the virtual robot in the target three-dimensional scene according to the target image acquisition posture;

[0124] The target image acquisition posture is an image acquisition posture set by the virtual robot corresponding to the target task point in the target three-dimensional scene.

[0125] In the process of determining the target image acquisition posture in the target three-dimensional scene, the image acquisition parameters of the pan / tilt in the real robot are combined. In this embodiment, the image acquisition parameters are further used to determine the image acquisition range of the virtual robot in the target three-dimensional scene according to the determined target image acquisition posture.

[0126] The image acquisition range may be a range determined by the virtual robot in the target three-dimensional scene according to the target image acquisition posture.

[0127] Among them, the virtual robot adopts a virtual pan-tilt camera, and the image acquisition range is the acquisition range corresponding to the fov (Field of View, the range that the lens can cover) parameter of the virtual pan-tilt camera.

[0128] 503. Render an image corresponding to the image acquisition range according to the three-dimensional scene of the on-site image to obtain the target image.

[0129] Among the multiple frames of scene images, the partial images corresponding to the image acquisition range are determined, and a certain task point may correspond to two or even more frames of scene images.

[0130] Among them, according to the above-determined three-dimensional scene of the on-site image, the image corresponding to the image acquisition range is rendered to obtain the target image, and the target image contains all the content information of the corresponding task point.

[0131] As an example, task point A corresponds to a cabinet-type equipment. The shooting posture of each frame of the on-site image is imported into the target three-dimensional scene to obtain the on-site image three-dimensional scene of task point A. The image acquisition range of the virtual robot for task point A in the target three-dimensional scene is determined, and the image acquisition range corresponds to two frames of on-site images. The two frames of images are rendered according to the on-site image three-dimensional scene to obtain the target image.

[0132] Figure 6 6 is a schematic diagram of a virtual robot determining an image acquisition range in a target three-dimensional scene provided by an embodiment of the present application. In the target three-dimensional scene 601, the image acquisition range 603 corresponding to the virtual robot 602 is shown. Figure 6 The image acquisition range is indicated by a dotted line. In the target three-dimensional scene, the area corresponding to the image acquisition parameters of the virtual robot belongs to the image acquisition range.

[0133] Figure 7 : This is a schematic diagram of the image acquisition range corresponding to the image provided in the embodiment of the present application. In this schematic diagram, the image acquisition range corresponds to three frames of field images 701-703. In this schematic diagram, the image acquisition range is for the task point 35kV1# station power cabinet. The image acquisition range for the task point corresponds to the right half of the field image 701, the entire part of the field image 702, and the left half of the field image 703. The three frames of images are rendered according to the three-dimensional scene of the field image to obtain the target image 704, which contains the right half of the field image 701, the entire field image 702, and the left half of the field image 703.

[0134] In this embodiment, the scene image is imported into the target three-dimensional scene by utilizing the shooting posture of the scene image to obtain the scene image three-dimensional scene, thereby realizing the construction of the scene image three-dimensional scene of the target task point; according to the aforementioned determined target image acquisition posture, the image acquisition range of the virtual robot in the target three-dimensional scene is determined; according to the scene image three-dimensional scene, the image corresponding to the image acquisition range is rendered to obtain the target image, thereby realizing obtaining the target image corresponding to the target image acquisition posture according to the scene image, and the target image is an image containing the target task point and its surrounding environment, which provides a basis for the subsequent determination of correction information.

[0135] Figure 8It is a flow chart of obtaining a target image corresponding to the target image acquisition posture based on the scene image provided by an embodiment of the present application, which may include steps 801 to 804, and these steps are described in detail below.

[0136] 801. Obtaining a shooting posture of a scene image;

[0137] The shooting posture is the shooting posture adopted by the image acquisition device when shooting live images in a real scene.

[0138] During the shooting process, each scene image records its shooting posture, which includes the position coordinates and orientation angle of the shooting device in the real scene. The position coordinates may include XYθ, and the orientation angle may adopt PTZ.

[0139] The shooting posture can be saved in the platform together with the on-site image.

[0140] 802. Based on the scene image and the shooting posture, establish a corresponding relationship between the target image acquisition posture and the shooting posture;

[0141] Among them, since there is a one-to-one correspondence between the spatial positions of the real three-dimensional scene and the virtual three-dimensional scene, the shooting posture of the scene image scanned in the real three-dimensional scene can be established with the target image acquisition posture in the virtual three-dimensional scene.

[0142] Generally, when scanning a real three-dimensional scene to obtain a scene image, each task point needs to be photographed intensively. Therefore, the shooting posture corresponding to each scene image will have a shooting posture corresponding to each task point, and the shooting posture is the posture facing the task point.

[0143] Among them, the corresponding relationship between the target image acquisition posture and the shooting posture represents the difference between the acquisition angle of the virtual gimbal of the virtual robot and the acquisition angle of the scanning device in the real scene, and the difference between the two is fixed.

[0144] Among them, the target image acquisition posture is the acquisition posture for the target task point, then for the target task point, the corresponding shooting posture of the corresponding scene image is determined, and the shooting posture corresponding to the scene image is the shooting posture corresponding to the target image acquisition posture in the real three-dimensional scene.

[0145] In a specific implementation, for each task point in the target three-dimensional scene, each shooting posture corresponding to its target image acquisition posture can be determined, and a corresponding relationship between the image acquisition posture and the shooting posture of each task point can be established.

[0146] 803. Determine, based on the corresponding relationship, a capture range corresponding to the target image acquisition posture in the scene image;

[0147] The corresponding relationship represents the difference between the acquisition angle of the virtual pan-tilt platform of the virtual robot and the acquisition angle of the scanning device in the real scene, and the difference between the two is fixed.

[0148] Therefore, based on the corresponding relationship, in the scene image, the interception range corresponding to the target image acquisition posture is determined, and the interception range can correspond to one or more frames of the scene image, and can include part or all of a certain frame of image.

[0149] In the process of determining the target image acquisition posture in the target three-dimensional scene, the image acquisition parameters of the pan / tilt in the real robot are combined. In this embodiment, the image acquisition parameters are further used in combination with the target image acquisition posture to determine the corresponding image acquisition range in the scene image.

[0150] 804. A target image is captured from the scene image according to the capture range.

[0151] The interception range may correspond to part or all of the scene image.

[0152] When the interception range is a part of the scene image, a corresponding part of the scene image is intercepted to obtain a target image.

[0153] As an example, the capture range is the left half of the scene image, and the left half of the scene image is captured as the target image.

[0154] When the captured range is the entire scene image, the scene image is used as the target image.

[0155] If the interception range involves multiple frames of on-site images, the intercepted images can be spliced ​​together in a set order to obtain a target image.

[0156] In a possible implementation, the splicing may be performed in a rendering manner.

[0157] In this embodiment, the shooting posture of the scene image is obtained; based on the scene image and the shooting posture, a corresponding relationship between the target image acquisition posture and the shooting posture is established; based on the corresponding relationship, the interception range corresponding to the target image acquisition posture in the scene image is determined; based on the interception range, the target image is intercepted in the scene image. In this process, the spatial position correspondence characteristics of the real three-dimensional scene and the target three-dimensional scene are used to establish a corresponding relationship between the shooting posture of shooting the scene image in the real environment and the target image acquisition posture, so as to use the corresponding relationship to determine the interception range corresponding to the target image acquisition posture in the scene image, and then the target image is intercepted in the scene image based on the interception range, thereby achieving the goal of obtaining the target image corresponding to the target image acquisition posture according to the scene image. The target image is an image containing the target task point and its surrounding environment, which provides a basis for the subsequent determination of correction information.

[0158] A control method for a robot guidance task point provided in an embodiment of the present application is introduced above, and a device for executing the control method for the robot guidance task point will be introduced below.

[0159] Fig. 9 Schematic diagram of a control device for guiding a robot task point provided in an embodiment of the present application. Fig. 9 As shown, the control device 900 of the robot guiding task point includes:

[0160] A scene image acquisition module 901 is used to acquire a scene image, which is an image used to construct a target three-dimensional scene;

[0161] The acquisition posture acquisition module 902 is used to obtain the target image acquisition posture of the virtual robot corresponding to the target task point in the target three-dimensional scene;

[0162] A target image acquisition module 903 is used to obtain a target image corresponding to the target image acquisition posture according to the scene image;

[0163] The correction information acquisition module 904 is used to obtain correction information according to the target image. The correction information is used to correct the image acquisition posture of the real robot when the real robot detects the target task point in the real scene.

[0164] In a possible implementation, the method further includes:

[0165] The response module is used to send the target image acquisition posture and the correction information to the real robot in response to receiving the issued instruction. The real robot can collect a first image in a real scene according to the target image acquisition posture, and correct the image acquisition posture according to the first image and the correction information.

[0166] In a possible implementation, the scene image includes all images used to construct the target three-dimensional scene.

[0167] In a possible implementation, the on-site image acquisition module is specifically used to:

[0168] The scene image corresponding to the target task point is obtained from the scene image set, and the scene image set is a set of images used to construct the target three-dimensional scene.

[0169] In a possible implementation, the target image acquisition module includes:

[0170] An importing unit, used for importing the scene image into the target three-dimensional scene according to the shooting posture of the scene image to obtain the scene image three-dimensional scene;

[0171] A first determining unit, configured to determine an image acquisition range of the virtual robot in the target three-dimensional scene according to the target image acquisition posture;

[0172] The rendering unit is used to render the image corresponding to the image acquisition range according to the three-dimensional scene of the on-site image to obtain the target image.

[0173] In a possible implementation, the target image acquisition module includes:

[0174] An obtaining unit, used for obtaining the shooting posture of the scene image;

[0175] An establishing unit, used to establish a corresponding relationship between the target image acquisition posture and the shooting posture based on the scene image and the shooting posture;

[0176] A second determining unit is used to determine the interception range corresponding to the target image acquisition posture in the scene image according to the corresponding relationship;

[0177] The interception unit is used to intercept and obtain a target image from the scene image according to the interception range.

[0178] In a possible implementation, the target image acquisition posture includes:

[0179] The position coordinates of the virtual robot in the target three-dimensional scene and the orientation angle of the virtual robot pan / tilt.

[0180] It should be noted that for the functional explanation of the structure of each component in the control device of a robot guiding task point provided in this embodiment, please refer to the explanation in the aforementioned method embodiment, which will not be repeated in this embodiment.

[0181] In this embodiment, a scene image for constructing a target three-dimensional scene is obtained, a target image acquisition posture of a virtual robot corresponding to a target task point in the target three-dimensional scene is obtained, a target image corresponding to the target image acquisition posture is obtained according to the scene image, correction information is obtained according to the target image, and the correction information is used to correct the image acquisition posture of the real robot when the real robot detects the target task point in the real scene, and the scene image for constructing the target three-dimensional scene is intercepted by using the target image acquisition posture of the virtual robot for the target task point in the target three-dimensional scene to obtain the target image, which is a part of the scene image, which is a part of the scene image corresponding to the target task point in the target three-dimensional scene in the real scene, and then correction information is obtained from the target image, the correction information corresponds to the real scene and is related to the target image acquisition posture, and the correction information can be used to realize the correction of the image acquisition posture of the real robot in the real scene when detecting the target task point, so as to ensure that the real robot can reach the absolute point position and configuration parameters of the task, thereby improving the task completion rate.

[0182] The present application also provides an electronic device in an embodiment. Fig.10 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Fig.10 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0183] like Fig.10 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 to a random access memory (RAM) 1003. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 1003. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0184] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a memory card, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig.10 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0185] Also provided in an embodiment of the present application is a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any method for controlling a robot guiding task point provided in an embodiment of the present application.

[0186] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any robot guidance task point control method provided in an embodiment of the present application.

[0187] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0188] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0189] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0190] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A control method for a robot guiding a task point, comprising: Obtaining a scene image, wherein the scene image is an image used to construct a target three-dimensional scene; Obtaining a target image acquisition posture of the virtual robot corresponding to a target task point in the target three-dimensional scene; Obtaining a target image corresponding to the target image acquisition posture according to the scene image; Correction information is obtained according to the target image, and the correction information is used to correct the image acquisition posture of the real robot when the real robot detects the target task point in a real scene.

2. The control method of the robot guiding task point according to claim 1, further comprising: In response to receiving the issued instruction, the target image acquisition posture and the correction information are sent to the real robot, and the real robot can obtain a first image in a real scene based on the target image acquisition posture, and correct the image acquisition posture according to the first image and the correction information.

3. According to the control method of the robot guidance task point according to claim 1, the on-site image includes all images used to construct the target three-dimensional scene.

4. The control method for guiding a robot task point according to claim 1, wherein obtaining a scene image comprises: The scene image corresponding to the target task point is obtained from a scene image set, where the scene image set is a set of images used to construct the target three-dimensional scene.

5. The control method for a robot-guided task point according to claim 1, wherein the step of obtaining a target image corresponding to the target image acquisition posture according to the on-site image comprises: Importing the scene image into the target three-dimensional scene according to the shooting posture of the scene image to obtain a scene image three-dimensional scene; Determining the image acquisition range of the virtual robot in the target three-dimensional scene according to the target image acquisition posture; The target image is obtained by rendering an image corresponding to the image acquisition range according to the three-dimensional scene of the on-site image.

6. The control method for guiding a robot task point according to claim 1, wherein the step of obtaining a target image corresponding to the target image acquisition posture according to the scene image comprises: Obtaining a shooting posture of the scene image; Based on the scene image and the shooting posture, establishing a corresponding relationship between the target image acquisition posture and the shooting posture; According to the corresponding relationship, determining the interception range corresponding to the target image acquisition posture in the scene image; According to the interception range, a target image is intercepted in the scene image.

7. The control method for guiding a robot task point according to claim 1, wherein the target image acquisition posture comprises: The position coordinates of the virtual robot in the target three-dimensional scene and the orientation angle of the virtual robot pan / tilt.

8. A control device for a robot guiding a task point, comprising: A scene image acquisition module, used to acquire a scene image, wherein the scene image is an image used to construct a target three-dimensional scene; An acquisition posture acquisition module is used to obtain a target image acquisition posture of the virtual robot corresponding to a target task point in the target three-dimensional scene; A target image acquisition module, used to obtain a target image corresponding to the target image acquisition posture according to the scene image; A correction information acquisition module is used to obtain correction information according to the target image, and the correction information is used to correct the image acquisition posture of the real robot when the real robot detects the target task point in a real scene.

9. A computer program product, characterized in that It comprises computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the control method for a robot-guided task point as claimed in any one of claims 1 to 7.

10. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the control method of the robot guiding task point as described in any one of claims 1 to 7.

Images