Positioning methods, devices, computer equipment, and software products for inspection robots

Abstract

This application relates to a positioning method, apparatus, computer device, computer-readable storage medium, and computer program product for an inspection robot. The method includes: acquiring a coarse positioning pose of the inspection robot relative to the target cabinet door when the inspection robot moves to a preset distance from the target cabinet door; if the error of the coarse positioning pose does not meet a preset accuracy requirement, acquiring a local image of the target cabinet door; extracting a set of natural structural features from the local image; determining the deviation of the inspection robot's natural structural features based on the set of natural structural features and a preset template; generating a correction command for the inspection robot based on the deviation, and correcting the inspection robot based on the correction command; after correction, returning to the coarse positioning pose acquisition step, continuously correcting the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirement. This method can improve the accuracy of inspection robot positioning.

Description

Technical Field

[0001] This application relates to the field of robot servo control technology, and in particular to a positioning method, device, computer equipment, computer-readable storage medium, and computer program product for an inspection robot. Background Technology

[0002] When substation inspection robots automatically operate equipment such as terminal boxes and switchgear, the end effector of the robotic arm needs to be precisely aligned with the locks, handles, or pressing areas on the cabinet doors. In existing technologies, robots mainly rely on navigation and positioning systems (such as SLAM, LiDAR, inertial navigation, etc.) to obtain the coarse positioning pose of the robot body relative to the cabinet door.

[0003] However, due to factors such as uneven ground, changes in the robot's posture, sensor errors, and cabinet installation deviations, there is often a centimeter-level deviation between the coarse positioning result and the actual operating point. If the robotic arm moves directly according to the pre-stored coordinates, it is easy to cause misalignment or collision, making it difficult to meet the requirements of fine operation.

[0004] In related technologies, existing solutions use QR codes, manual markers, or depth camera point clouds for assisted positioning. However, these marker-dependent solutions require additional installation and maintenance on the cabinet door. In outdoor environments, markers are easily soiled or detached, resulting in high maintenance costs, and they are not suitable for cabinets that are already in operation and cannot be modified. Furthermore, single-shot visual positioning methods lack a closed-loop fine-tuning mechanism based on end-image feedback, making it impossible to continuously correct errors during the approach process, leading to inaccurate positioning. Summary of the Invention

[0005] Therefore, it is necessary to provide a positioning method, device, computer equipment, computer-readable storage medium, and computer program product for inspection robots that can improve the positioning accuracy of inspection robots, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a positioning method for an inspection robot, comprising:

[0007] When the inspection robot moves to a preset distance from the target cabinet door, obtain the coarse positioning pose of the inspection robot relative to the target cabinet door;

[0008] If the error of the coarse positioning pose does not meet the preset accuracy requirements, a local image of the target cabinet door is acquired.

[0009] Extract the set of natural structural features from the local image;

[0010] Based on the set of natural structural features and the preset template, the deviation of the natural structural features of the inspection robot is determined;

[0011] Based on the deviation, a correction instruction for the inspection robot is generated, and the inspection robot is corrected based on the correction instruction;

[0012] After the correction is completed, return to the coarse positioning pose acquisition step and continuously correct the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirements.

[0013] In one embodiment, the set of natural structural features includes at least one of the following: lock outline corner point of the target cabinet door, handle end point, cabinet door gap line, hinge edge line, nameplate edge, or screw hole center.

[0014] In one embodiment, determining the deviation of the natural structural features of the inspection robot based on the set of natural structural features and the preset template includes:

[0015] Each natural structural feature in the set of natural structural features is compared with the corresponding preset template, and the pixel deviation between the current coordinates of each natural structural feature and the target coordinates in the preset template is calculated.

[0016] The pixel deviation is taken as the deviation of the natural structural features of the inspection robot.

[0017] In one embodiment, generating correction instructions for the inspection robot based on the deviation, and correcting the inspection robot based on the correction instructions, includes:

[0018] Each deviation of a natural structural feature is assigned a preset weight, and a weighted sum is performed to obtain the image error of the inspection robot.

[0019] The image error is converted into a correction command for the inspection robot;

[0020] Based on the correction instructions, generate small step speed or displacement correction instructions for the end effector of the inspection robot arm.

[0021] The inspection robot is corrected based on the small step speed or the displacement correction command.

[0022] In one embodiment, the method further includes:

[0023] If a natural structural feature is reflected, dirty, or occluded, the preset weight of the corresponding natural structural feature is reduced, and the preset weight of the remaining natural structural features is increased.

[0024] In one embodiment, the method further includes:

[0025] When the inspection robot is a preset robot, depth information of natural structural features within a preset range is obtained;

[0026] Based on the depth information, the image error is converted into the end-effector pose correction amount of the inspection robot;

[0027] The inspection robot is corrected based on the aforementioned end-effector pose correction amount.

[0028] Secondly, this application also provides a positioning device for an inspection robot, comprising:

[0029] The acquisition module is used to acquire the coarse positioning pose of the inspection robot relative to the target cabinet door when the inspection robot moves to a preset distance from the target cabinet door.

[0030] The acquisition module is used to acquire a local image of the target cabinet door when the error of the coarse positioning pose does not meet the preset accuracy requirements.

[0031] An extraction module is used to extract a set of natural structural features from the local image;

[0032] The determination module is used to determine the deviation of the natural structural features of the inspection robot based on the set of natural structural features and the preset template;

[0033] The correction module is used to generate correction instructions for the inspection robot based on the deviation, and to correct the inspection robot based on the correction instructions;

[0034] The correction module is used to return to the coarse positioning pose acquisition step after the correction is completed, and continuously correct the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirements.

[0035] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0036] When the inspection robot moves to a preset distance from the target cabinet door, obtain the coarse positioning pose of the inspection robot relative to the target cabinet door;

[0037] If the error of the coarse positioning pose does not meet the preset accuracy requirements, a local image of the target cabinet door is acquired.

[0038] Extract the set of natural structural features from the local image;

[0039] Based on the set of natural structural features and the preset template, the deviation of the natural structural features of the inspection robot is determined;

[0040] Based on the deviation, a correction instruction for the inspection robot is generated, and the inspection robot is corrected based on the correction instruction;

[0041] After the correction is completed, return to the coarse positioning pose acquisition step and continuously correct the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirements.

[0042] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0043] When the inspection robot moves to a preset distance from the target cabinet door, obtain the coarse positioning pose of the inspection robot relative to the target cabinet door;

[0044] If the error of the coarse positioning pose does not meet the preset accuracy requirements, a local image of the target cabinet door is acquired.

[0045] Extract the set of natural structural features from the local image;

[0046] Based on the set of natural structural features and the preset template, the deviation of the natural structural features of the inspection robot is determined;

[0047] Based on the deviation, a correction instruction for the inspection robot is generated, and the inspection robot is corrected based on the correction instruction;

[0048] After the correction is completed, return to the coarse positioning pose acquisition step and continuously correct the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirements.

[0049] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0050] When the inspection robot moves to a preset distance from the target cabinet door, obtain the coarse positioning pose of the inspection robot relative to the target cabinet door;

[0051] If the error of the coarse positioning pose does not meet the preset accuracy requirements, a local image of the target cabinet door is acquired.

[0052] Extract the set of natural structural features from the local image;

[0053] Based on the set of natural structural features and the preset template, the deviation of the natural structural features of the inspection robot is determined;

[0054] Based on the deviation, a correction instruction for the inspection robot is generated, and the inspection robot is corrected based on the correction instruction;

[0055] After the correction is completed, return to the coarse positioning pose acquisition step and continuously correct the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirements.

[0056] The aforementioned positioning method, device, computer equipment, computer-readable storage medium, and computer program product for the inspection robot first acquire a coarse positioning pose of the inspection robot relative to the target cabinet door when the inspection robot moves to a preset distance from the target cabinet door. If the error of the coarse positioning pose does not meet the preset accuracy requirements, a local image of the target cabinet door is acquired. A set of natural structural features of the local image is extracted. Based on the set of natural structural features and a preset template, the deviation of the natural structural features of the inspection robot is determined. Based on the deviation, a correction command for the inspection robot is generated, and the inspection robot is corrected based on the correction command. After the correction is completed, the process returns to the step of acquiring the coarse positioning pose, and the inspection robot is continuously corrected until the error of the coarse positioning pose meets the preset accuracy requirements. Thus, through a mechanism of coarse positioning, visual fine positioning, and closed-loop iteration, the inspection robot achieves high-precision pose convergence in cabinet door operation scenarios. Positioning can be completed using only the inherent natural structural features of the cabinet door itself, significantly reducing deployment and maintenance costs. At the same time, through end-effector visual servo closed-loop control, centimeter-level coarse positioning deviations caused by uneven ground, changes in robot posture, or sensor errors can be effectively compensated, avoiding robot arm pressing misalignment or collisions. In addition, through multiple iterations and corrections until the accuracy requirements are met, it is ensured that the end-effector operation point can stably reach the precise alignment state of target operation points such as locks and handles, significantly improving the success rate, safety, and environmental adaptability of the robot's automatic operation. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is an application environment diagram of the positioning method for an inspection robot in one embodiment;

[0059] Figure 2 This is a flowchart illustrating the positioning method of an inspection robot in one embodiment;

[0060] Figure 3 This is a structural schematic diagram of the set of natural structural features of a target cabinet door in one embodiment;

[0061] Figure 4 This is a structural block diagram of the positioning device of an inspection robot in one embodiment;

[0062] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0065] The positioning method for inspection robots provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located on the cloud or other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0066] In one exemplary embodiment, such as Figure 2 As shown, a positioning method for an inspection robot is provided, which can be applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps 202 to 212. Wherein:

[0067] Step 202: When the inspection robot moves to a preset distance from the target cabinet door, obtain the coarse positioning pose of the inspection robot relative to the target cabinet door.

[0068] For example, when the inspection robot moves to a preset distance from the target cabinet door, the coarse positioning pose of the inspection robot relative to the target cabinet door is obtained.

[0069] Among them, the coarse positioning pose is the initial position and attitude information of the inspection robot relative to the target cabinet door. The initial position is the three-dimensional spatial coordinates between the inspection robot and the target cabinet door, and the attitude information is the orientation angle of the inspection robot, such as yaw angle, pitch angle and roll angle.

[0070] Step 204: If the coarse positioning pose error does not meet the preset accuracy requirements, acquire a local image of the target cabinet door.

[0071] Optionally, the error of the coarse positioning pose is determined. If the error of the coarse positioning pose meets the preset accuracy, the robotic arm of the inspection robot enters the operation preparation state. If the error of the coarse positioning pose does not meet the preset accuracy requirements, a local image of the target cabinet door is acquired by the end-effector camera of the robotic arm of the inspection robot.

[0072] Step 206: Extract the set of natural structural features of the local image.

[0073] For example, a set of natural structural features is extracted from a local image of the target cabinet door.

[0074] Among them, natural structural features are the geometric or textural features that are inherent in the target cabinet door itself, do not require additional artificial markers, and can be used for visual recognition and positioning.

[0075] Step 208: Based on the set of natural structural features and the preset template, determine the deviation of the natural structural features of the inspection robot.

[0076] Among them, the preset templates are pre-calibrated benchmark templates, historical same-cabinet type templates, or online construction templates.

[0077] Optionally, the set of natural structural features of the target cabinet door can be compared with a preset template to determine the deviation of the natural structural features of the inspection robot.

[0078] Step 210: Based on the deviation, generate correction instructions for the inspection robot, and correct the inspection robot based on the correction instructions.

[0079] For example, correction instructions for the inspection robot are generated based on the deviations in the natural structural features of the inspection robot, and the robotic arm of the inspection robot is corrected based on the correction instructions.

[0080] Step 212: After the correction is completed, return to the coarse positioning pose acquisition step and continuously correct the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirements.

[0081] Optionally, after the correction is completed, if the error of the coarse positioning pose of the inspection robot still does not meet the preset accuracy requirements, the process returns to the coarse positioning pose acquisition step and continuously corrects the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirements.

[0082] The coarse positioning pose error includes image error, estimation spatial error, and end-effector pose error.

[0083] In some embodiments, if the error of the coarse positioning pose of the inspection robot meets the preset accuracy requirements, the final end pose, target operation point, and error evaluation results are sent to the display unit for visualization.

[0084] In the aforementioned positioning method for the inspection robot, when the inspection robot moves to a preset distance from the target cabinet door, a coarse positioning pose of the inspection robot relative to the target cabinet door is acquired; if the error of the coarse positioning pose does not meet the preset accuracy requirement, a local image of the target cabinet door is acquired; a set of natural structural features of the local image is extracted; based on the set of natural structural features and a preset template, the deviation of the natural structural features of the inspection robot is determined; based on the deviation, a correction command for the inspection robot is generated, and the inspection robot is corrected based on the correction command; after the correction is completed, the process returns to the step of acquiring the coarse positioning pose, and the inspection robot is continuously corrected until the error of the coarse positioning pose meets the preset accuracy requirement. Thus, through a mechanism of coarse positioning, visual fine positioning, and closed-loop iteration, the inspection robot achieves high-precision pose convergence in cabinet door operation scenarios. Positioning can be completed using only the inherent natural structural features of the cabinet door itself, significantly reducing deployment and maintenance costs. At the same time, through end-effector visual servo closed-loop control, centimeter-level coarse positioning deviations caused by uneven ground, changes in robot posture, or sensor errors can be effectively compensated, avoiding robot arm pressing misalignment or collisions. In addition, through multiple iterations and corrections until the accuracy requirements are met, it is ensured that the end-effector operation point can stably reach the precise alignment state of target operation points such as locks and handles, significantly improving the success rate, safety, and environmental adaptability of the robot's automatic operation.

[0085] In an exemplary embodiment, the set of natural structural features includes at least one of the following: lock outline corner point, handle end point, cabinet door gap line, hinge edge line, nameplate edge, or screw hole center of the target cabinet door.

[0086] In practice, the set of natural structural features includes at least one of the following: the corner point of the lock outline of the target cabinet door, the end point of the handle, the gap line of the cabinet door, the edge line of the hinge, the edge of the nameplate, or the center of the screw hole.

[0087] In some embodiments, a structural schematic diagram of the target cabinet door's natural structural feature set is shown below. Figure 3 As shown.

[0088] In some embodiments, when locating the target operation point of the target cabinet door based on natural structural features, at least one natural structural feature directly related to the target operation point is included, or two or more natural structural features that can jointly determine the local coordinate system of the target cabinet door are included.

[0089] In some embodiments, natural structural features are preferred to be structures that are stable in the long term and do not require additional modification to the target cabinet door. For example, the cabinet door gap line is used to estimate the cabinet door plane orientation; the handle end point is used to determine the center of the operating area; the lock outline corner point is used to determine the position of the lock relative to the cabinet door; the hinge edge line is used to estimate the cabinet door pivot direction; and the nameplate edge and screw hole center can be used as auxiliary features.

[0090] In the above embodiments, the inherent and long-term stable geometric structure and texture information of the cabinet door are fully utilized, eliminating the need for additional QR codes, labels or any artificial markers. This fundamentally reduces the burden of retrofitting cabinet doors in operational substations and the long-term operation and maintenance costs. At the same time, these features cover various types such as point and line features, which are redundant and complementary to each other. When a single feature becomes unreliable due to reflection, obstruction or dirt, the system can still rely on other available features to complete the positioning and deviation calculation. This significantly improves the robustness and applicability of the visual servo positioning method in complex outdoor environments, and can adapt to most common cabinet door structures, exhibiting good versatility and portability.

[0091] In an exemplary embodiment, the deviation of the natural structural features of the inspection robot is determined based on the set of natural structural features and the preset template, including: comparing each natural structural feature in the set of natural structural features with the corresponding preset template, calculating the pixel deviation between the current coordinates of each natural structural feature and the target coordinates in the preset template; and using the pixel deviation as the deviation of the natural structural features of the inspection robot.

[0092] In practice, each natural structural feature in the set of natural structural features of the target cabinet door is compared with the corresponding preset template, and the pixel deviation between the current coordinates of each natural structural feature and the target coordinates in the preset template is calculated; the pixel deviation is used as the deviation of the natural structural features of the inspection robot.

[0093] In the above embodiments, by comparing each feature in the set of natural structural features with the preset template point by point, calculating the pixel deviation between the current coordinates and the target coordinates of the template, and using the pixel deviation directly as the deviation of the target operation point, the deviation calculation is made intuitive and lightweight, without the need for complex 3D reconstruction or iterative optimization, which greatly reduces the computational overhead and response latency of the visual servo control system.

[0094] In an exemplary embodiment, a correction instruction for the inspection robot is generated based on the deviation, and the inspection robot is corrected based on the correction instruction, including: assigning a preset weight to the deviation of each natural structural feature, performing a weighted summation to obtain the image error of the inspection robot; converting the image error into a correction instruction for the inspection robot; generating a small step speed or displacement correction instruction for the end effector of the inspection robot based on the correction instruction; and correcting the inspection robot based on the small step speed or displacement correction instruction.

[0095] In practice, each deviation of the natural structural feature is assigned a preset weight and weighted summation is performed to obtain the image error of the inspection robot; the image error is converted into a correction command for the inspection robot; based on the correction command, a small step speed or displacement correction command is generated at the end of the inspection robot's robotic arm; the inspection robot is corrected based on the small step speed or displacement correction command.

[0096] In some embodiments, correction instructions for the inspection robot can be generated based on weighted image errors, camera calibration relationships, depth or scale information.

[0097] In the above embodiments, by assigning a preset weight to the deviation of each natural structural feature and performing a weighted summation, the interference of unreliable features is effectively suppressed, highlighting the dominant role of high-confidence features, thereby obtaining a more accurate and robust overall image error. On this basis, the weighted image error is converted into a correction command, and further small step speed or displacement correction commands are generated at the end of the robotic arm. This ensures the accuracy of the correction direction and magnitude, and avoids the risk of oscillation or collision caused by excessive single-step correction through the small step strategy. This allows the robotic arm to smoothly and safely approach the target operation point step by step, significantly improving the convergence stability, safety, and environmental adaptability of the end-effector visual servo precision positioning.

[0098] In an exemplary embodiment, the positioning method of the inspection robot further includes: reducing the preset weight of the corresponding natural structural feature and increasing the preset weight of the remaining natural structural features when the natural structural feature is reflective, dirty or obstructed.

[0099] In practice, if a certain natural structural feature is reflected, dirty, or obscured, the recognition result will be unreliable. Therefore, the preset weight of the corresponding natural structural feature will be reduced, and the preset weight of the remaining natural structural features will be increased.

[0100] In some embodiments, in outdoor environments, reflections, shadows, water stains, and dust can cause instability in the recognition of some features. The system calculates the reliability weight of each feature based on the number of features, edge strength, matching residual, historical stability, and occlusion ratio. If the handle end is occluded, the system can temporarily rely on the cabinet door gap line, hinge edge, and lock profile for pose estimation.

[0101] In the above embodiments, by actively reducing the preset weight of the affected features and simultaneously increasing the weight of the remaining reliable features when natural structural features are reflective, dirty, or occluded, adaptive dynamic adjustment of feature weights is achieved. This enables the visual servoing system to "choose the best" when faced with local feature degradation, avoiding unreliable features from leading the end-effector to deviate from its guiding direction. This mechanism significantly improves the robustness and fault tolerance of the positioning method in complex outdoor environments, ensuring that even if some features temporarily fail, the system can still rely on other available features such as cabinet door gap lines and hinge edges to continue to complete precise positioning, thereby maintaining the continuity and safety of the operation process and reducing the risk of positioning failure or operation interruption due to environmental interference.

[0102] In an exemplary embodiment, the localization method for the inspection robot further includes: when the inspection robot is a preset robot, acquiring depth information within a preset range of natural structural features; converting image errors into end-effector pose correction amounts based on the depth information; and correcting the inspection robot based on the end-effector pose correction amounts.

[0103] In practical implementation, when the inspection robot is a preset robot, depth information within a preset range of natural structural features is acquired; based on the depth information, image errors are converted into end-effector pose corrections for the inspection robot; and the inspection robot is corrected based on the end-effector pose corrections.

[0104] Among them, the robot is required to have an RGB-D camera, a binocular camera or a structure camera at the end of the robot arm, which can directly obtain depth information near natural structural features.

[0105] In some embodiments, when the end-point camera is a monocular camera, instead of directly assuming precise depth, the scale is estimated using cabinet door plane constraints, known cabinet door structural dimensions, historical template dimensions of handles or locks, and small-step closed-loop probing. If the confidence level of the scale estimation is insufficient, the single fine-tuning step size is reduced, and multiple image feedbacks are required to change in the same convergence direction before continuing to approach.

[0106] In the above embodiments, by targeting a pre-defined robot with depth perception capabilities (such as an RGB-D camera, a binocular camera, or a structured light camera), depth information within a preset range of natural structural features is directly acquired, and image errors are explicitly converted into end-effector pose correction amounts. This achieves a precise mapping from two-dimensional pixel deviations to three-dimensional spatial correction commands, avoiding estimation errors and conservative control caused by scale uncertainty in monocular cameras. This scheme can fully utilize depth information to calculate a reasonable correction step size in one go, reducing the number of closed-loop iterations and accelerating the convergence speed of fine positioning. At the same time, it makes the end-effector correction closer to the real geometric constraints, improving positioning accuracy and operation success rate. It is especially suitable for robot autonomous operation scenarios with high requirements for accuracy and efficiency.

[0107] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0108] Based on the same inventive concept, this application also provides a positioning device for an inspection robot to implement the positioning method of the inspection robot described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more positioning device embodiments of inspection robots provided below can be found in the limitations of the positioning method of inspection robots above, and will not be repeated here.

[0109] In one exemplary embodiment, such as Figure 4 As shown, a positioning device for an inspection robot is provided, comprising: an acquisition module 401, a collection module 402, an extraction module 403, a determination module 404, and a correction module 405, wherein:

[0110] The acquisition module is used to acquire the coarse positioning pose of the inspection robot relative to the target cabinet door when the inspection robot moves to a preset distance from the target cabinet door.

[0111] The acquisition module is used to acquire local images of the target cabinet door when the error of the coarse positioning pose does not meet the preset accuracy requirements.

[0112] The extraction module is used to extract a set of natural structural features from the local image.

[0113] The determination module is used to determine the deviation of the natural structural features of the inspection robot based on the set of natural structural features and the preset template.

[0114] The correction module is used to generate correction instructions for the inspection robot based on the deviation, and to correct the inspection robot based on the correction instructions.

[0115] The correction module is used to return to the coarse positioning pose acquisition step after the correction is completed, and continuously correct the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirements.

[0116] In one exemplary embodiment, the set of natural structural features includes at least one of the following: lock outline corner point of the target cabinet door, handle end point, cabinet door gap line, hinge edge line, nameplate edge, or screw hole center.

[0117] In one exemplary embodiment, the determining module is further configured to:

[0118] Each natural structural feature in the set of natural structural features is compared with the corresponding preset template, and the pixel deviation between the current coordinates of each natural structural feature and the target coordinates in the preset template is calculated.

[0119] Pixel deviation is used as a deviation of the natural structural features of the inspection robot.

[0120] In one exemplary embodiment, the above-described correction module is further configured to:

[0121] Each deviation of a natural structural feature is assigned a preset weight, and a weighted sum is performed to obtain the image error of the inspection robot.

[0122] Convert image errors into correction instructions for the inspection robot;

[0123] Based on the correction instructions, generate small step speed or displacement correction instructions for the end effector of the inspection robot arm.

[0124] The inspection robot is corrected based on small step speed or displacement correction commands.

[0125] In one exemplary embodiment, the above-described apparatus further includes an adjustment module for:

[0126] If a natural structural feature is reflected, dirty, or occluded, the preset weight of the corresponding natural structural feature is reduced, and the preset weight of the remaining natural structural features is increased.

[0127] In one exemplary embodiment, the above-described correction module is further configured to:

[0128] When the inspection robot is a preset robot, depth information of natural structural features within a preset range is obtained;

[0129] Based on depth information, image errors are converted into end-effector pose correction values ​​for the inspection robot.

[0130] The inspection robot is corrected based on the end-effector pose correction.

[0131] The various modules in the positioning device of the aforementioned inspection robot can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0132] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a positioning method for an inspection robot.

[0133] The display unit of this computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of this computer device can be a touch layer covering the display screen, or buttons, a trackball, or a touchpad set on the casing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0134] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0135] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0136] When the inspection robot moves to a preset distance from the target cabinet door, obtain the coarse positioning pose of the inspection robot relative to the target cabinet door;

[0137] If the error of the coarse positioning pose does not meet the preset accuracy requirements, a local image of the target cabinet door is acquired.

[0138] Extract the set of natural structural features from the local image;

[0139] Based on the set of natural structural features and the preset template, the deviation of the natural structural features of the inspection robot is determined;

[0140] Based on the deviation, a correction instruction for the inspection robot is generated, and the inspection robot is corrected based on the correction instruction;

[0141] After the correction is completed, return to the coarse positioning pose acquisition step and continuously correct the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirements.

[0142] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0143] When the inspection robot moves to a preset distance from the target cabinet door, obtain the coarse positioning pose of the inspection robot relative to the target cabinet door;

[0144] If the error of the coarse positioning pose does not meet the preset accuracy requirements, a local image of the target cabinet door is acquired.

[0145] Extract the set of natural structural features from the local image;

[0146] Based on the set of natural structural features and the preset template, the deviation of the natural structural features of the inspection robot is determined;

[0147] Based on the deviation, a correction instruction for the inspection robot is generated, and the inspection robot is corrected based on the correction instruction;

[0148] After the correction is completed, return to the coarse positioning pose acquisition step and continuously correct the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirements.

[0149] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0150] When the inspection robot moves to a preset distance from the target cabinet door, obtain the coarse positioning pose of the inspection robot relative to the target cabinet door;

[0151] If the error of the coarse positioning pose does not meet the preset accuracy requirements, a local image of the target cabinet door is acquired.

[0152] Extract the set of natural structural features from the local image;

[0153] Based on the set of natural structural features and the preset template, the deviation of the natural structural features of the inspection robot is determined;

[0154] Based on the deviation, a correction instruction for the inspection robot is generated, and the inspection robot is corrected based on the correction instruction;

[0155] After the correction is completed, return to the coarse positioning pose acquisition step and continuously correct the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirements.

[0156] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0157] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0158] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0159] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A positioning method of a patrol robot, characterized by, The method includes: When the inspection robot moves to a preset distance from the target cabinet door, obtain the coarse positioning pose of the inspection robot relative to the target cabinet door; If the error of the coarse positioning pose does not meet the preset accuracy requirements, a local image of the target cabinet door is acquired. Extract the set of natural structural features from the local image; Based on the set of natural structural features and the preset template, the deviation of the natural structural features of the inspection robot is determined; Based on the deviation, a correction instruction for the inspection robot is generated, and the inspection robot is corrected based on the correction instruction; After the correction is completed, return to the coarse positioning pose acquisition step and continuously correct the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirements.

2. The method of claim 1, wherein, The set of natural structural features includes at least one of the following: lock outline corner point, handle end point, cabinet door gap line, hinge edge line, nameplate edge, or screw hole center of the target cabinet door.

3. The method of claim 1, wherein, The determination of deviations in the natural structural features of the inspection robot based on the set of natural structural features and the preset template includes: Each natural structural feature in the set of natural structural features is compared with the corresponding preset template, and the pixel deviation between the current coordinates of each natural structural feature and the target coordinates in the preset template is calculated. The pixel deviation is taken as the deviation of the natural structural features of the inspection robot.

4. The method according to claim 1, characterized in that, The process of generating correction instructions for the inspection robot based on the deviation, and correcting the inspection robot based on the correction instructions, includes: Each deviation of a natural structural feature is assigned a preset weight, and a weighted sum is performed to obtain the image error of the inspection robot. The image error is converted into a correction command for the inspection robot; Based on the correction instructions, generate small step speed or displacement correction instructions for the end effector of the inspection robot arm. The inspection robot is corrected based on the small step speed or the displacement correction command.

5. The method according to claim 4, characterized in that, The method further includes: If a natural structural feature is reflected, dirty, or occluded, the preset weight of the corresponding natural structural feature is reduced, and the preset weight of the remaining natural structural features is increased.

6. The method according to claim 4, characterized in that, The method further includes: When the inspection robot is a preset robot, depth information of natural structural features within a preset range is obtained; Based on the depth information, the image error is converted into the end-effector pose correction amount of the inspection robot; The inspection robot is corrected based on the aforementioned end-effector pose correction amount.

7. A positioning device for an inspection robot, characterized in that, The device includes: The acquisition module is used to acquire the coarse positioning pose of the inspection robot relative to the target cabinet door when the inspection robot moves to a preset distance from the target cabinet door. The acquisition module is used to acquire a local image of the target cabinet door when the error of the coarse positioning pose does not meet the preset accuracy requirements. An extraction module is used to extract a set of natural structural features from the local image; The determination module is used to determine the deviation of the natural structural features of the inspection robot based on the set of natural structural features and the preset template; The correction module is used to generate correction instructions for the inspection robot based on the deviation, and to correct the inspection robot based on the correction instructions; The correction module is used to return to the coarse positioning pose acquisition step after the correction is completed, and continuously correct the inspection robot until the error of the coarse positioning pose meets the preset accuracy requirements.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images