Robot repositioning method and device and storage medium

Abstract

The invention discloses a robot repositioning method and device and a storage medium, and belongs to the technical field of robots. The method comprises the steps of obtaining at least two groups of visual coding feature images collected by a robot when a relocation condition is met, resolving robot poses corresponding to the groups of visual coding feature images, fusing the robot poses corresponding to the groups of visual coding feature images to obtain fused poses, and generating a relocation result according to the fused poses. According to the method and the device, the multiple groups of visual coding features of the robot are collected, the corresponding robot poses are calculated and fused, and the fused poses are obtained, so that the situation that the accuracy of repositioning of the robot is affected due to robot pose deviation calculated by single-frame visual coding features of the robot is avoided.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a robot repositioning method, device, and storage medium. Background Art

[0002] To improve the robot's navigation and operational accuracy in complex environments, it uses relocalization to adjust its motion path to avoid accidents such as falls. However, if the robot fails to localize or its positioning accuracy is insufficient, it can easily lead to mission failure.

[0003] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a robot repositioning method, device and storage medium, aiming to solve the technical problem of insufficient robot positioning accuracy.

[0005] To achieve the above object, the present application provides a robot repositioning method, the method comprising the following steps: When the relocation condition is met, obtaining at least two sets of visual coding feature images collected by the robot; Calculating the robot pose corresponding to each group of visual coding feature images; Fusing the robot postures corresponding to the groups of visual encoding feature images to obtain a fused posture; A relocalization result is generated according to the fused pose.

[0006] In one embodiment, before the step of acquiring at least two sets of visual coding feature images collected by the robot when the relocation condition is met, the step further includes: Obtaining a visually estimated pose and a laser-estimated pose of the robot respectively, and calculating a pose difference between the visually estimated pose and the laser-estimated pose; When the posture difference is greater than a posture difference threshold, determining that a relocation condition is met; The robot is controlled to switch to a stationary state, and in the stationary state, the steps of obtaining at least two groups of visual coding feature images collected by the robot and subsequent steps are performed.

[0007] In one embodiment, before the steps of respectively obtaining the visually estimated pose and the laser estimated pose of the robot and calculating the pose difference between the visually estimated pose and the laser estimated pose, the method further includes: Obtaining, by means of a visual sensor, a target visual coding feature image used by the robot to determine whether a relocation condition is satisfied; Extracting target visual coding features of the target visual coding feature image based on image processing; Determining, in spatial coordinates, feature point coordinates of a visual coding feature point in the target visual coding feature; The visually estimated pose of the robot is calculated according to the feature point coordinates.

[0008] In one embodiment, before the steps of respectively obtaining the visually estimated pose and the laser estimated pose of the robot and calculating the pose difference between the visually estimated pose and the laser estimated pose, the method further includes: Collecting the robot's environmental point cloud through a laser radar; Matching the environmental point cloud with the robot's environmental map and calculating the occupancy probability of the matching result; The environmental point cloud is iterated according to the occupancy probability, and the laser estimated pose of the robot is calculated according to the iteration result.

[0009] In one embodiment, the step of fusing the robot postures corresponding to the groups of visually encoded feature images to obtain a fused posture comprises: Obtaining position information and direction information of each group of robot postures, and determining the position distance between the position information and the angular difference between the direction information; Calculating a center position of the position information based on the position distance, and calculating a central tendency of the direction information based on the angle difference; generating the target position information according to the central position, and generating the target direction information according to the central trend; The fused pose is obtained based on the target position information and the target direction information.

[0010] In one embodiment, the step of obtaining the fused pose based on the target position information and the target direction information includes: Determining the horizontal coordinate and the vertical coordinate of the current posture according to the target position information, and determining the angle of the current position according to the target direction information; The fused pose is generated based on the abscissa, the ordinate, and the angle.

[0011] In one embodiment, the step of calculating the robot pose corresponding to each group of visually encoded feature images includes: Extracting the visual coding features of multiple frames of static visual coding feature images through image processing; Determining, in spatial coordinates, feature point coordinates of the visual coding feature points in the visual coding feature; The robot posture is calculated according to the feature point coordinates.

[0012] In one embodiment, after the step of generating a relocalization result according to the fused pose, the method further includes: Determining a current pose of the robot in the relocalization result; Planning an adjustment path of the robot according to the current posture and the initial motion path; According to the adjustment path, the robot is controlled to perform a path adjustment action.

[0013] In addition, to achieve the above-mentioned purpose, the present application also provides a robot repositioning device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is configured to implement the steps of the robot repositioning method as described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the robot repositioning method as described above are implemented.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: The present application simultaneously collects multiple sets of visual feature codes of the robot and solves them separately during repositioning, and fuses the robot postures obtained by solving multiple sets of visual coding features, and determines the repositioning result based on the fused posture, thereby reducing the degree to which the robot is affected by abnormal factors when repositioning with single-frame visual coding features during the repositioning process, and improving the accuracy and stability of the robot repositioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 This is a flow chart of the first embodiment of the robot repositioning method of the present application; Figure 2 This is a schematic diagram of robot repositioning involved in the first embodiment of the robot repositioning method of the present application; Figure 3 This is a schematic diagram of position clustering involved in the first embodiment of the robot repositioning method of the present application; Figure 4 This is a schematic diagram of direction clustering involved in the first embodiment of the robot relocalization method of the present application; Figure 5 This is a schematic diagram of the robot posture clustering process involved in the first embodiment of the robot relocalization method of the present application; Figure 6 This is a flow chart of a second embodiment of the robot repositioning method of the present application; Figure 7 This is a schematic diagram of the repositioning process involved in the second embodiment of the robot repositioning method of the present application; Figure 8 This is a flow chart of a third embodiment of the robot repositioning method of the present application; Figure 9 It is a structural diagram of a robot repositioning device in a hardware operating environment involved in an embodiment of the present application.

[0019] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0021] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0022] The main solution of the embodiment of the present application is: when the relocation conditions are met, at least two groups of visual coding feature images collected by the robot are obtained; the robot posture corresponding to each group of the visual coding feature images is solved; the robot posture corresponding to each group of the visual coding feature images is fused to obtain a fused posture; and a relocation result is generated according to the fused posture.

[0023] The robot uses visual sensors such as cameras to capture image information of the surrounding environment, extract visual feature points to build an environmental map, and achieve repositioning by comparing the current visual features with the features in the map. In the process, it relies on the capture and analysis of single-frame visual coding features to calculate the current positioning. Therefore, when facing a dynamically changing environment or after running for a long time, the robot will cause visual feature drift, making it difficult to adapt to dynamic influencing factors such as changes in environmental lighting and object movement, resulting in a decrease in the robot's positioning accuracy.

[0024] The present application simultaneously collects multiple sets of visual feature codes of the robot and solves them separately during repositioning, and fuses the robot postures obtained by solving multiple sets of visual coding features, and determines the repositioning result based on the fused posture, thereby reducing the degree to which the robot is affected by abnormal factors when repositioning with single-frame visual coding features during the repositioning process, and improving the accuracy and stability of the robot repositioning.

[0025] To better understand the above technical solutions, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0026] It should be noted that the execution subject of this embodiment can be a robot control system, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of performing the above functions, a robot repositioning device, etc., and this embodiment does not specifically limit this. The following uses a robot control system as an example to illustrate this embodiment and the following embodiments.

[0027] Based on this, the embodiment of the present application provides a robot repositioning method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the robot repositioning method of the present application.

[0028] In this embodiment, the robot repositioning method includes steps S10 to S40: Step S10: When the relocation condition is met, obtaining at least two sets of visual coding feature images collected by the robot; In this embodiment, the robot control system uses a positioning device to determine its current position in a preset environmental map and moves based on a preset route. The positioning device includes a visual sensor that calculates the robot's position by collecting visually encoded feature images. The robot's position consists of its position information and posture information. The position information is used to indicate the robot's specific location in space and is typically described using a three-dimensional vector. The posture information indicates the robot's orientation in space and includes information such as pitch angle, yaw angle, roll angle, and / or rotation matrix.

[0029] Specifically, based on preset relocalization rules, the robot can determine whether relocalization conditions are met based on information such as the environment map, the preset movement path, and the current position and posture. If the relocalization conditions are met, the robot control system will obtain at least two sets of visual coding feature images through the visual sensor. By solving and fusing multiple sets of visual coding feature images, it recalculates the current position and posture of the robot, thus achieving robot relocalization.

[0030] For example, users can set the robot's relocalization rules based on the robot's operating environment or work requirements. During movement, the robot can trigger relocalization when it moves to a new location and cannot determine its current location, when the deviation from a preset path or target location exceeds a certain threshold, or when it detects an environmental change such as an obstacle moving or a change in lighting conditions, and calibrates its current position through relocalization.

[0031] For example, Figure 2 As shown, Figure 2 This is a schematic diagram of the robot relocation process involved in this embodiment. The robot is equipped with an industrial camera as a video camera. The z-axis of the industrial camera coordinate system (Camera) is oriented vertically and horizontally upward. Visual coding features are affixed to the top of the operating environment, such as a corridor ceiling. The robot relocates by recognizing these visual coding features.

[0032] Step S20: Calculating the robot posture corresponding to each group of visual coding feature images; In this embodiment, the visual coding feature image is a feature image captured by a camera and used to calculate the robot's pose using visual coding features. The robot control system converts the original image, which serves as the visual coding feature image, into feature data through feature coding and extracts visual coding features from the visual coding feature image. The visual coding feature image can then be processed by the pose calculation algorithm in the robot control system to calculate the robot's pose.

[0033] Specifically, based on the fusion of multiple sets of robot postures during the relocalization process, the robot control system needs to solve at least two sets of visual coding feature images captured by the robot's camera equipment, and obtain the corresponding robot postures respectively, so as to reduce the interference of environmental factors on the visual coding feature image solving process.

[0034] As an optional implementation method for calculating the robot posture, step S20 includes steps S21 to S23: Step S21: extracting the visual coding features of multiple frames of static visual coding feature images through image processing; Step S22: determining the feature point coordinates of the visual coding feature points in the visual coding feature in the spatial coordinates; Step S23: Calculate the robot posture according to the feature point coordinates.

[0035] In this embodiment, when the robot control system determines that the relocation conditions are met and relocation is performed, it stops the robot's movement, collects multiple frames of static visual coding feature images in a static state, and extracts the corresponding visual coding feature images. Alternatively, the robot control system can also select the image of the target frame from the video stream captured by the camera device as the visual coding feature image.

[0036] Specifically, the robot control system extracts the coding features of the captured image through image processing methods to obtain a series of 3D-2D matching points. The coordinates of the visual coding feature points in space are P i =[X i , Y i , Z i ] T When the corresponding 2D projection point pixel coordinate is p i =[u i , v i ] T , then

[0037] Among them, S i is a non-zero scale factor, K is the intrinsic parameter of the camera device, [RT] is the coordinate transformation matrix from the visual encoding feature in space to the camera device, R is the rotation parameter, and T is the translation parameter.

[0038] Furthermore, due to the noise in the robot's posture and observation points, there is an error in the equation, so a least squares problem is often constructed to optimize the robot's posture.

[0039]

[0040] Among them, Δε represents the minimum sum of squared errors. By calculating Δε, the corresponding 2D projection point pixel coordinates pi=[ui,vi] with the minimum sum of squared errors can be determined. T , thereby determining the robot's posture.

[0041] For example, Figure 2 As shown, based on the position of the industrial camera set on the robot, the robot control system transforms the robot posture matrix ( ), establish a matching relationship between the visual coding feature image captured by the camera device and the 2D projection point pixel coordinates of the robot projection coordinate system (Robat).

[0042] Step S30: fusing the robot postures corresponding to the visual coding feature images of each group to obtain a fused posture; In this embodiment, after the robot control system completes the solution of the robot posture of each group of visual coding feature images, it fuses the robot postures of each group through clustering, thereby reducing the impact of the environment on the robot repositioning when a single group of visual coding features is affected by the environment.

[0043] In one embodiment, the robot posture is recorded and stored in the form of a QR code posture. The position information of the QR code posture is represented by coordinates on a two-dimensional plane, which can be absolute coordinates relative to a fixed point or relative coordinates relative to a reference point. The direction information of the QR code posture is the rotation angle of the QR code relative to a preset reference direction, which can be the tilt angle of the QR code relative to the horizontal plane or the rotation angle of the QR code relative to the camera's line of sight. The robot control system obtains the position information of the QR code by detecting feature points such as corner points of the QR code, and determines the rotation angle of the QR code by detecting and analyzing the feature points of the QR code and based on the relative position relationship between the feature points.

[0044] For example, Figure 2 As shown, based on the 3D-2D matching points of the industrial camera coordinate system and the robot projection coordinate system, the robot control system transforms the QR code matrix ( ) method, the 3D-2D matching points are transformed into a QR code posture, namely the Quick Response Tag (QR Tag).

[0045] Furthermore, step S30 includes steps S31 to S34: Step S31: obtaining the position information and direction information of each group of robot postures, and determining the position distance between the position information and the angle difference between the direction information; In this implementation, the static multi-frame robot pose fusion process primarily utilizes a clustering approach. Because the QR code pose consists of both position and orientation information, the robot control system fuses both position and orientation information separately, generating a fused pose based on the combined position and orientation information.

[0046] Step S32: calculating the center position of the position information based on the position distance, and calculating the central tendency of the direction information according to the angle difference; In this embodiment, the robot control system clusters each group of acquired position information and each group of acquired direction information, determines the center position by clustering the position information, and determines the center trend by clustering the direction information.

[0047] In one example, referring to Figure 3 and Figure 4 , Figure 3This is a schematic diagram of location clustering involved in this embodiment. Figure 4 This is a schematic diagram of the direction clustering involved in this embodiment. Figure 3 and Figure 4 As shown, to prevent abnormal data from interfering with the clustering process, the robot control system determines the distance between the position information and other position information, and determines the maximum distance between them. If the maximum distance exceeds a set maximum distance threshold, the position information and / or the robot pose corresponding to it is deemed abnormal and removed. Simultaneously, the robot control system calculates the angular difference between the direction information and other direction information, and if the maximum angular difference exceeds a set maximum angular difference threshold, the direction information and / or the robot pose corresponding to it is removed.

[0048] Step S33: generating the target position information according to the center position, and generating the target direction information according to the center trend; Step S34: obtaining the fused pose based on the target position information and the target direction information.

[0049] In this embodiment, the fused pose is composed of target position information and target direction information. The robot control system can use the fused center position as the target position information and the center trend as the target direction information, and generate the fused pose based on the target position information and target direction information.

[0050] Specifically, the robot control system can determine the horizontal and vertical coordinates of the current pose based on the target position information, and determine the angle of the current position based on the target direction information. Based on the horizontal and vertical coordinates and the angle, the robot control system can determine the information of the fused pose and generate the fused pose.

[0051] For example, refer to Figure 5 , Figure 5 The figure below is a schematic diagram of the robot pose clustering process involved in this embodiment. The robot control system uses a camera to capture n sets of visually encoded feature images of the robot, extracts the encoded features from each image, and decodes the visual encoding. Based on the decoding results, the robot control system determines the matching relationship between 3D points in space and 2D points in the environment map and generates the corresponding pose. The robot control system then clusters and fuses multiple poses to determine the optimal robot pose.

[0052] Step S40: generating a relocalization result according to the fused pose.

[0053] In this embodiment, the robot control system may optionally use the fused posture directly as the relocalization result, or determine the position and orientation of the robot in the environment map and / or action route to determine the relocalization result.

[0054] For example, Figure 2 As shown, the robot control system completes the fusion of robot postures through multiple groups of QR tags, generates fusion postures, and then transforms the postures through the map matrix ( ) maps the fused position to the environment map, allowing the robot to determine whether it is currently on the correct action path based on the relocalization results.

[0055] The embodiment of the present application simultaneously collects multiple sets of visual feature codes of the robot and solves them separately during repositioning, and fuses the robot postures obtained by solving the multiple sets of visual coding features, and determines the repositioning result based on the fused posture, thereby reducing the degree to which the robot is affected by abnormal factors when repositioning with single-frame visual coding features during the repositioning process, and improving the accuracy and stability of the robot repositioning.

[0056] Based on the same inventive concept, this application also provides a second embodiment, referring to Figure 6 , Figure 6 This is a flow chart of the second embodiment of the robot repositioning method of the present application.

[0057] In this embodiment, the repositioning of the robot includes steps S11 to S13: Step S11: respectively obtaining a visually estimated pose and a laser-estimated pose of the robot, and calculating a pose difference between the visually estimated pose and the laser-estimated pose; Step S12: When the posture difference is greater than the posture difference threshold, determining that a relocation condition is met; In this embodiment, if Figure 2 As shown, the robot is also equipped with a lidar (LiDAR), which continuously emits laser signals to its surroundings and receives reflected laser information, which is a point cloud of the environment. Based on the recognition results of the point cloud, the robot control system determines the distance between the robot and objects in the environment and determines the robot's laser-estimated position in the lidar coordinate system.

[0058] Furthermore, when the frequencies of the visual camera and lidar pose estimates are out of sync, the lag of one sensor can introduce new pose deviations, reducing the accuracy of the robot's pose calculation during movement. Therefore, the robot control system obtains the visually estimated pose from the robot's camera and the laser-estimated pose from the lidar, calculates the pose difference between the visually estimated and laser-estimated poses, and determines that relocalization conditions are met and executes relocalization when the pose difference exceeds a preset threshold.

[0059] In a specific implementation, during the robot's motion, the robot control system uses a visual sensor at preset time intervals to obtain a target visual coding feature image used by the robot to determine whether relocation conditions are met. By performing image processing on the target visual coding feature image, the target visual coding features of the target visual coding feature image are extracted, and the feature point coordinates of the visual coding feature points in the target visual coding features are determined in spatial coordinates. Based on the feature point coordinates, the robot's visually estimated pose is calculated. The robot control system can process the target visual feature image during the robot's motion using the same calculation method used to calculate the robot's pose from a single-frame visual coding feature image during the robot's relocation process.

[0060] At the same time, the robot control system uses LiDAR to collect a point cloud of the robot's environment, matches it with the robot's environment map, and calculates the occupancy probability of the matching result. The point cloud is iterated based on the occupancy probability, and the robot's laser-estimated pose is calculated based on the iteration results.

[0061] Specifically, 2D radar estimates the robot's pose mainly by acquiring the environment point cloud scan matching offline map to achieve pose estimation. Scan matching is the process of aligning the lidar point cloud with the environment map in the form of a raster map, and it mainly uses the Gauss-Newton method to iteratively search and solve. For continuous map coordinates Si, M(Si) is used to represent the probability of the point occupying the environment map, so there is

[0062] Where is the position of the robot in the map, and is the coordinate of the environment map point of the i-th laser point.

[0063] Step S13: Control the robot to switch to a stationary state, and in the stationary state, execute the step of acquiring at least two sets of visual coding feature images collected by the robot and subsequent steps.

[0064] In this embodiment, when the robot control system determines to perform a repositioning action, it controls the robot to stop the current action and collect multiple sets of visual coding feature images in a stationary state.

[0065] For example, Figure 7 As shown, Figure 7 The figure below is a schematic diagram of the relocalization process involved in this embodiment. The robot control system obtains the robot's visually estimated pose and laser-estimated pose, compares them, and if the difference is large, collects multiple sets of visually estimated poses again. The poses estimated from these multiple sets of visual encoding features are fused to complete relocalization.

[0066] The embodiment of the present application calculates the posture difference between the visually estimated posture and the laser-estimated posture, and when the posture difference is greater than a preset threshold, determines that the repositioning conditions are met and performs repositioning, thereby avoiding the situation where the visually estimated posture and the laser-estimated posture are out of sync, and further improves the stability of the robot's positioning during movement.

[0067] Since the system described in Example 2 of this application is the system used to implement the method of Example 1 of this application, those skilled in the art will be able to understand the specific structure and variations of the system based on the method described in Example 1 of this application, and therefore, no further description is given here. All systems used in the method of Example 1 of this application fall within the scope of protection to be provided by this application.

[0068] Based on the same inventive concept, this application also provides a third embodiment, referring to Figure 8 , Figure 8 This is a flow chart of the third embodiment of the robot repositioning method of the present application.

[0069] In this embodiment, the repositioning of the robot includes steps S41 to S43: Step S41: determining the current posture of the robot in the relocation result; Step S42: planning an adjustment path of the robot according to the current posture and the initial motion path; Step S43: controlling the robot to perform a path adjustment action according to the adjustment path.

[0070] In this embodiment, the robot control system can obtain the current posture of the robot after repositioning in the repositioning result, and obtain the position and posture in the current posture respectively. The robot control system will match the position and posture with the initial motion path of the robot before repositioning, determine whether the current position is offset, and whether the direction of the posture is deviated, and make corrections. For example, if the robot position is offset, the robot control system can re-plan the path based on the end point of the initial motion path, or determine the path to the position point of the initial motion path that is closest to the current position to generate an adjustment path. After determining the adjustment action, the robot control system controls the robot to perform a path adjustment action based on the adjustment path, thereby achieving the expected path movement effect.

[0071] Since the system described in Example 3 of this application is the system used to implement the method of Example 1 of this application, those skilled in the art will be able to understand the specific structure and variations of the system based on the method described in Example 1 of this application, and therefore will not be described in detail here. All systems used in the method of Example 1 of this application fall within the scope of protection to be provided by this application.

[0072] The present application provides a robot repositioning device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the robot repositioning method in the above-mentioned embodiment one.

[0073] Reference below Figure 9 , which shows a schematic structural diagram of a robot repositioning device suitable for implementing embodiments of the present application. The robot repositioning device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9 The robot repositioning device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0074] like Figure 9As shown, the robot's repositioning device may include a processing device 1001 (e.g., a core processor, graphics processor, etc.), which can execute various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the robot's repositioning device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003, such as a magnetic tape or hard disk; and communication device 1009. The communication device 1009 can allow the robot's repositioning device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a robot's repositioning device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have alternatively.

[0075] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0076] The robot repositioning device provided in this application, utilizing the robot repositioning method of the aforementioned embodiment, can resolve the technical issue of insufficient robot repositioning accuracy. Compared to the prior art, the robot repositioning device provided in this application has the same beneficial effects as the robot repositioning method provided in the aforementioned embodiment. Other technical features of the robot repositioning device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0077] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0078] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0079] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the robot repositioning method in the above-mentioned embodiment.

[0080] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0081] The computer-readable storage medium may be included in the repositioning device of the robot; or may exist independently without being assembled into the repositioning device of the robot.

[0082] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the robot's repositioning device, the robot's repositioning device enables: when the repositioning conditions are met, to obtain at least two groups of visual coding feature images collected by the robot; to solve the robot posture corresponding to each group of the visual coding feature images; to fuse the robot posture corresponding to each group of the visual coding feature images to obtain a fused posture; and to generate a repositioning result according to the fused posture.

[0083] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0084] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0085] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0086] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned robot repositioning method, thereby resolving the technical issue of insufficient robot repositioning accuracy. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the robot repositioning method provided in the aforementioned embodiments, and are not further elaborated here.

[0087] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A robot repositioning method, characterized in that: The method comprises the following steps: When the relocation condition is met, obtaining at least two sets of visual coding feature images collected by the robot; Calculating the robot posture corresponding to each group of visual coding feature images; Fusing the robot postures corresponding to each group of visual coding feature images to obtain a fused posture; A relocalization result is generated according to the fused pose.

2. The method according to claim 1, characterized in that Before the step of acquiring at least two sets of visual coding feature images collected by the robot when the relocation condition is met, the step further includes: Respectively obtaining a visually estimated pose and a laser estimated pose of the robot, and calculating a pose difference between the visually estimated pose and the laser estimated pose; When the posture difference is greater than the posture difference threshold, determining that the relocation condition is met; The robot is controlled to switch to a stationary state, and in the stationary state, the steps of acquiring at least two groups of visual coding feature images collected by the robot and subsequent steps are performed.

3. The method according to claim 2, characterized in that Before the steps of respectively acquiring the visual estimated pose and the laser estimated pose of the robot and calculating the pose difference between the visual estimated pose and the laser estimated pose, the method further includes: Obtaining, by means of a visual sensor, a target visual coding feature image used by the robot to determine whether a relocation condition is met; Extracting target visual coding features of the target visual coding feature image based on image processing; Determine, in spatial coordinates, feature point coordinates of a visual coding feature point in the target visual coding feature; The visually estimated pose of the robot is calculated according to the feature point coordinates.

4. The method according to claim 2, characterized in that Before the steps of respectively acquiring the visual estimated pose and the laser estimated pose of the robot and calculating the pose difference between the visual estimated pose and the laser estimated pose, the method further includes: Collecting the environment point cloud of the robot through laser radar; Matching the environmental point cloud with the environmental map of the robot, and calculating the occupancy probability of the matching result; The environmental point cloud is iterated according to the occupancy probability, and the laser estimated position and posture of the robot is calculated according to the iteration result.

5. The method according to claim 1, characterized in that The step of fusing the robot postures corresponding to each group of the visual coding feature images to obtain a fused posture comprises: Obtaining position information and direction information of each group of robot postures, and determining the position distance between the position information and the angle difference between the direction information; Calculating the central position of the position information based on the position distance, and calculating the central tendency of the direction information according to the angle difference; generating the target position information according to the central position, and generating the target direction information according to the central trend; The fused posture is obtained based on the target position information and the target direction information.

6. The method according to claim 5, characterized in that The step of obtaining the fusion posture based on the target position information and the target direction information comprises: Determine the horizontal coordinate and the vertical coordinate of the current position according to the target position information, and determine the angle of the current position according to the target direction information; The fused pose is generated based on the horizontal coordinate, the vertical coordinate and the angle.

7. The method according to claim 1, characterized in that The step of calculating the robot posture corresponding to each group of visual coding feature images comprises: Extracting the visual coding features of multiple frames of static visual coding feature images through image processing; Determine, in spatial coordinates, feature point coordinates of the visual coding feature points in the visual coding feature; The robot posture is calculated according to the feature point coordinates.

8. The method according to claim 1, characterized in that After the step of generating a relocation result according to the fusion posture, the method further includes: Determining a current position and posture of the robot in the relocalization result; Planning an adjustment path of the robot according to the current posture and the initial action path; According to the adjustment path, the robot is controlled to perform a path adjustment action.

9. A robot repositioning device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the robot repositioning method according to any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the robot repositioning method according to any one of claims 1 to 8 are implemented.

Images