A robot and a method for controlling the movement of the robot

Abstract

The present disclosure provides a robot and a method for controlling the movement of the robot, wherein the robot includes a visual sensor and a control component; wherein the visual sensor is configured to acquire a current frame image while the robot is moving; and the control component is configured to, upon detecting that the current frame image includes a target marker, determine a first position of the target marker in a target scene; and control the robot to move toward a target working point based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative positional relationship between a target working point and the target marker. The robot has higher positioning accuracy.

Description

Technical Field

[0001] The present disclosure relates to the field of robot control technology, and in particular to a robot and a driving control method of the robot. Background Art

[0002] With the development of intelligent robot technology, controlling the robot to accurately reach the target position has become the primary task of current intelligent control. The current positioning method based on the contours of multiple reference objects in the target scene has the problem of poor positioning accuracy. Summary of the Invention

[0003] The embodiments of the present disclosure at least provide a robot and a method for controlling the movement of the robot.

[0004] In a first aspect, an embodiment of the present disclosure provides a robot, comprising: an information acquisition component and a control component;

[0005] The information acquisition component is configured to acquire a current frame image during the robot's driving process;

[0006] The positioning component is configured to: when detecting that the current frame image includes a target marker, determine a first position of the target marker in the target scene;

[0007] Based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative position relationship between the target work point and the target marker, the robot is controlled to move toward the target work point.

[0008] In a possible implementation, the target marker is fixedly arranged in the target scene; different working points have different relative positional relationships with the target marker;

[0009] Alternatively, the target marker is movably arranged in the target scene; the working point is set at a different position from the target marker, and different working points are formed as the target marker is located at different positions of the target scene.

[0010] In one possible implementation, the control component, when determining the first position of the target marker in the target scene, is configured to:

[0011] determining, based on the current frame image, a first observation position of the target marker in a robot coordinate system corresponding to the robot;

[0012] Based on the current posture of the robot in the target scene and the first observation position of the target marker in the robot coordinate system, a first position of the target marker in the target scene is determined.

[0013] In a possible implementation, the control component is further configured to determine a current posture of the robot in the target scene based on the current frame image.

[0014] In one possible implementation, when determining the current posture of the robot in the target scene based on the current frame image, the control component is configured to:

[0015] Determining, based on the current frame image, a second observation pose of at least one target reference object in the target scene in a robot coordinate system;

[0016] Determining a third observation pose of the at least one target reference object in the target scene based on the reference pose of the robot in the target scene and the second observation pose of the at least one target reference object in the robot coordinate system;

[0017] Based on the third observation posture and the actual posture of the at least one target reference object in the target scene, the reference posture is adjusted to obtain the current posture of the robot.

[0018] In a possible implementation, the control component is further configured to:

[0019] Performing edge detection processing on the current frame image to obtain observation contour information in the current frame image;

[0020] Matching the observed contour information with contour information corresponding to a plurality of reference objects;

[0021] If any observed contour information matches the contour information of any reference object successfully, the reference object is used as the target reference object included in the current frame image.

[0022] In one possible implementation, the control component, when determining the third observed pose of the at least one target reference object in the target scene based on the reference pose of the robot in the target scene and the second observed pose of the at least one target reference object in the robot coordinate system, is configured to:

[0023] Determining, based on a reference pose of the robot in the target scene, conversion relationship information between the robot coordinate system and the scene coordinate system;

[0024] The conversion relationship information is used to convert the first observation position of the at least one target reference object in the robot coordinate system into a second observation posture in the target scene.

[0025] In one possible implementation, the control component, when adjusting the reference pose based on the third observation pose and the actual pose of the at least one target reference object in the target scene to obtain the current pose of the robot, is configured to:

[0026] Perform at least one iteration cycle, and in each iteration cycle: determine a pose error based on a third observed pose corresponding to the at least one target reference object and an actual pose in the target scene; and adjust a reference pose of a current iteration cycle based on the pose error to obtain an adjusted reference pose;

[0027] In response to an iteration stop condition being met, determining the adjusted reference pose obtained in the last iteration cycle as the current pose of the robot;

[0028] The reference posture of the current iteration cycle is determined based on the reference posture of the previous iteration cycle, or based on the initial reference posture.

[0029] In a possible implementation, the control component is further configured to: determine the initial reference pose based on an odometer installed on the robot.

[0030] In one possible implementation, the iteration stopping condition includes at least one of the following:

[0031] The number of iterations corresponding to the current iteration cycle reaches the preset number of iterations;

[0032] The pose error determined in the current iteration cycle is less than the preset pose error threshold.

[0033] In one possible implementation, the control component, when controlling the robot to travel toward the target work point based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative positional relationship between the target work point and the target marker, is configured to:

[0034] Obtaining a second position of the target working point in the target scene based on the first position of the target marker in the target scene and the relative positional relationship between the target working point and the target marker;

[0035] Generate a target path based on the current posture of the robot and the second position of the target workpoint in the target scene;

[0036] The robot is controlled to travel toward the target working point based on the target path.

[0037] In a second aspect, the present disclosure also provides a method for controlling a robot's movement, including:

[0038] Get the current frame image captured by the visual sensor on the robot;

[0039] When detecting that the current frame image includes a target marker, determining a first position of the target marker in the target scene;

[0040] Based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative position relationship between the target work point and the target marker, the robot is controlled to move toward the target work point.

[0041] In a possible implementation, the target marker is fixedly arranged in the target scene; different working points have different relative positional relationships with the target marker;

[0042] Alternatively, the target marker is movably arranged in the target scene; the working point is set at a different position from the target marker, and different working points are formed as the target marker is located at different positions of the target scene.

[0043] In a possible implementation, determining the first position of the target marker in the target scene includes:

[0044] determining, based on the current frame image, a first observation position of the target marker in a robot coordinate system corresponding to the robot;

[0045] Based on the current posture of the robot in the target scene and the first observation position of the target marker in the robot coordinate system, a first position of the target marker in the target scene is determined.

[0046] In a possible implementation, the method further includes determining a current posture of the robot in the target scene based on the current frame image.

[0047] In one possible implementation, determining the current posture of the robot in the target scene based on the current frame image includes:

[0048] Determining, based on the current frame image, a second observation pose of at least one target reference object in the target scene in a robot coordinate system;

[0049] Determining a third observation pose of the at least one target reference object in the target scene based on the reference pose of the robot in the target scene and the second observation pose of the at least one target reference object in the robot coordinate system;

[0050] Based on the third observation posture and the actual posture of the at least one target reference object in the target scene, the reference posture is adjusted to obtain the current posture of the robot.

[0051] In a possible implementation, the following further includes:

[0052] Performing edge detection processing on the current frame image to obtain observation contour information in the current frame image;

[0053] Matching the observed contour information with contour information corresponding to a plurality of reference objects;

[0054] If any observed contour information matches the contour information of any reference object successfully, the reference object is used as the target reference object included in the current frame image.

[0055] In one possible implementation, determining a third observation pose of the at least one target reference object in the target scene based on the reference pose of the robot in the target scene and the second observation pose of the at least one target reference object in the robot coordinate system includes:

[0056] Determining, based on a reference pose of the robot in the target scene, conversion relationship information between the robot coordinate system and the scene coordinate system;

[0057] The conversion relationship information is used to convert the first observation position of the at least one target reference object in the robot coordinate system into a second observation posture in the target scene.

[0058] In one possible implementation, adjusting the reference pose based on the third observed pose and the actual pose of the at least one target reference object in the target scene to obtain the current pose of the robot includes:

[0059] Perform at least one iteration cycle, and in each iteration cycle: determine a pose error based on a third observed pose corresponding to the at least one target reference object and an actual pose in the target scene; and adjust a reference pose of a current iteration cycle based on the pose error to obtain an adjusted reference pose;

[0060] In response to an iteration stop condition being met, determining the adjusted reference pose obtained in the last iteration cycle as the current pose of the robot;

[0061] The reference posture of the current iteration cycle is determined based on the reference posture of the previous iteration cycle, or based on the initial reference posture.

[0062] In a possible implementation, the method further includes determining the initial reference pose based on an odometer installed on the robot.

[0063] In one possible implementation, the iteration stopping condition includes at least one of the following:

[0064] The number of iterations corresponding to the current iteration cycle reaches the preset number of iterations;

[0065] The pose error determined in the current iteration cycle is less than the preset pose error threshold.

[0066] In one possible implementation, controlling the robot to move toward the target work point based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative positional relationship between the target work point and the target marker includes:

[0067] Obtaining a second position of the target working point in the target scene based on the first position of the target marker in the target scene and the relative positional relationship between the target working point and the target marker;

[0068] Generate a target path based on the current posture of the robot and the second position of the target workpoint in the target scene;

[0069] The robot is controlled to travel toward the target working point based on the target path.

[0070] The robot and the robot driving control method provided by the embodiments of the present disclosure set a target marker in a target scene. When the target marker is included in the current frame image captured by the visual sensor on the robot, the first position of the target marker in the target scene is determined, and based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative positional relationship between the target working point and the target marker, the robot is controlled to drive toward the target working point, thereby achieving high-precision positioning in a changing environment by using the target marker as a reference.

[0071] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0073] Figure 1 A schematic structural diagram of a robot provided by an embodiment of the present disclosure is shown;

[0074] Figure 2 A flowchart of a robot driving control method provided by an embodiment of the present disclosure is shown;

[0075] Figure 3 A flowchart showing a specific method for determining the current position of a robot in a target scene provided by an embodiment of the present disclosure is shown;

[0076] Figure 4 A specific example of feature matching during the driving process of a robot provided by an embodiment of the present disclosure is shown;

[0077] Figure 5 A specific example of detecting a target marker and generating a target path during the driving process of a robot provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0079] Research has found that in logistics scenarios like containers, the scene changes as goods are stacked. Robots need to handle both positioning challenges brought on by these changes and corresponding changes in their workstations. Conventional contour-based positioning methods typically require detecting the outlines of multiple reference objects in the target scene and then using these outlines for positioning. However, due to the constant changes in logistics scenarios and the need for constant changes in workstations, current positioning methods that rely on the outlines of reference objects cannot meet the high-precision positioning requirements in such situations.

[0080] Based on the above research, the present disclosure provides a robot and a driving control method for the robot, which improves the positioning accuracy of the robot in a changing environment by setting a target marker in the target scene as a reference for the robot to move to the target working point.

[0081] The defects in the above solutions are the results obtained by the inventors after practice and careful research. Therefore, the process of discovering the above problems and the solutions proposed by this disclosure for the above problems below should be the contributions made by the inventors to this disclosure during the disclosure process.

[0082] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0083] To facilitate understanding of this embodiment, a detailed description of a robot disclosed in this embodiment is first provided, followed by a detailed description of a robot movement control method disclosed in this embodiment. The robot movement control method provided in this embodiment can be executed by a robot or a server controlling the movement of the robot. In some possible implementations, the robot movement control method can be implemented by a processor invoking computer-readable instructions stored in a memory.

[0084] The robot and the robot movement control method provided by the embodiments of the present disclosure are described below.

[0085] See also Figure 1 FIG. 1 is a schematic diagram of the structure of a robot provided by an embodiment of the present disclosure. The robot includes: a visual sensor 10 and a control component 20;

[0086] The visual sensor 10 is configured to obtain a current frame image during the robot's driving process;

[0087] The control component 20 is configured to: when detecting that the current frame image includes a target marker, determine a first position of the target marker in the target scene;

[0088] Based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative position relationship between the target work point and the target marker, the robot is controlled to move toward the target work point.

[0089] The disclosed embodiment sets a target marker in a target scene. When a control component on a robot includes the target marker in a current frame image acquired based on a visual sensor, the control component determines a first position of the target marker in the target scene. Based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative positional relationship between the target work point and the target marker, the robot is controlled to move toward the target work point. Thus, high-precision positioning is achieved in a changing environment by using the target marker as a reference.

[0090] In a possible implementation, the target marker is fixedly arranged in the target scene; different working points have different relative positional relationships with the target marker;

[0091] Alternatively, the target marker is movably arranged in the target scene; the working point is set at a different position from the target marker, and different working points are formed as the target marker is located at different positions of the target scene.

[0092] In a possible implementation, the control component 20, when determining the first position of the target marker in the target scene, is configured to:

[0093] determining, based on the current frame image, a first observation position of the target marker in a robot coordinate system corresponding to the robot;

[0094] Based on the current posture of the robot in the target scene and the first observation position of the target marker in the robot coordinate system, a first position of the target marker in the target scene is determined.

[0095] In a possible implementation, the control component 20 is further configured to determine a current posture of the robot in the target scene based on the current frame image.

[0096] In one possible implementation, when determining the current posture of the robot in the target scene based on the current frame image, the control component 20 is configured to:

[0097] Determining, based on the current frame image, a second observation pose of at least one target reference object in the target scene in a robot coordinate system;

[0098] Determining a third observation pose of the at least one target reference object in the target scene based on the reference pose of the robot in the target scene and the second observation pose of the at least one target reference object in the robot coordinate system;

[0099] Based on the third observation posture and the actual posture of the at least one target reference object in the target scene, the reference posture is adjusted to obtain the current posture of the robot.

[0100] In a possible implementation, the control component 20 is further configured to:

[0101] Performing edge detection processing on the current frame image to obtain observation contour information in the current frame image;

[0102] Matching the observed contour information with contour information corresponding to a plurality of reference objects;

[0103] If any observed contour information matches the contour information of any reference object successfully, the reference object is used as the target reference object included in the current frame image.

[0104] In one possible implementation, the control component 20, when determining the third observed pose of the at least one target reference object in the target scene based on the reference pose of the robot in the target scene and the second observed pose of the at least one target reference object in the robot coordinate system, is configured to:

[0105] Determining, based on a reference pose of the robot in the target scene, conversion relationship information between the robot coordinate system and the scene coordinate system;

[0106] The conversion relationship information is used to convert the first observation position of the at least one target reference object in the robot coordinate system into a second observation posture in the target scene.

[0107] In one possible implementation, the control component 20, when adjusting the reference pose based on the third observation pose and the actual pose of the at least one target reference object in the target scene to obtain the current pose of the robot, is configured to:

[0108] Perform at least one iteration cycle, and in each iteration cycle: determine a pose error based on a third observed pose corresponding to the at least one target reference object and an actual pose in the target scene; and adjust a reference pose of a current iteration cycle based on the pose error to obtain an adjusted reference pose;

[0109] In response to an iteration stop condition being met, determining the adjusted reference pose obtained in the last iteration cycle as the current pose of the robot;

[0110] The reference posture of the current iteration cycle is determined based on the reference posture of the previous iteration cycle, or based on the initial reference posture.

[0111] In a possible implementation, the control component 20 is further configured to determine the initial reference pose based on an odometer installed on the robot.

[0112] In one possible implementation, the iteration stopping condition includes at least one of the following:

[0113] The number of iterations corresponding to the current iteration cycle reaches the preset number of iterations;

[0114] The pose error determined in the current iteration cycle is less than the preset pose error threshold.

[0115] In one possible implementation, the control component 20, when controlling the robot to travel toward the target work point based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative positional relationship between the target work point and the target marker, is configured to:

[0116] Obtaining a second position of the target working point in the target scene based on the first position of the target marker in the target scene and the relative positional relationship between the target working point and the target marker;

[0117] Generate a target path based on the current posture of the robot and the second position of the target workpoint in the target scene;

[0118] The robot is controlled to travel toward the target working point based on the target path.

[0119] In the embodiment of the present disclosure, the control component 20 controls the specific driving process of the robot. For details, please refer to the following Figure 2 Corresponding robot driving control method.

[0120] Based on the same inventive concept, an embodiment of the present disclosure also provides a robot driving control method corresponding to the robot method.

[0121] Reference Figure 2 FIG. 1 is a flow chart of a robot driving control method provided by an embodiment of the present disclosure, including:

[0122] S201: Acquire the current frame image captured by the visual sensor on the robot;

[0123] S202: When detecting that the current frame image includes a target marker, determining a first position of the target marker in the target scene;

[0124] S203: Based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative position relationship between the target work point and the target marker, control the robot to move toward the target work point.

[0125] The disclosed embodiment sets a target marker in a target scene. When the current frame image captured by a visual sensor on a robot includes the target marker, the first position of the target marker in the target scene is determined. Based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative positional relationship between the target work point and the target marker, the robot is controlled to move toward the target work point, thereby achieving high-precision positioning in a changing environment by using the target marker as a reference.

[0126] The above S201 to S203 are described in detail below.

[0127] With respect to the above S201 , the current frame image captured by the visual sensor is, for example, a current frame image acquired during the robot's driving in the target scene.

[0128] The visual sensor includes, for example, at least one of a depth camera, a fisheye camera, a multi-camera color camera, a monocular color camera, and the like.

[0129] For example, while the robot is navigating within a target scene, it can obtain a video stream through a visual sensor. The video stream includes multiple frames of images. The current image in the video stream captured in real time by the visual sensor can be used as the current frame image. Alternatively, the current frame image can be obtained by sampling from the video stream according to a control cycle for controlling the robot.

[0130] With respect to the above S202 , after obtaining the current frame image, for example, it may be detected whether the current frame image includes the target marker.

[0131] Exemplary target markers include, for example, reflectors, objects with distinctive contours, or visual markers placed within the target scene. The target markers can be fixedly positioned within a specific location within the target scene. By determining the relative positional relationships between different workstations and the target markers, the robot, after determining a first position of the target marker within the target scene, can then, based on the relative positional relationships between the different workstations and the target markers, reach the corresponding workstations. Furthermore, the target markers can also be movably positioned within the target scene; personnel within the target scene can manually adjust the position of the target markers.

[0132] Alternatively, a device for controlling the movement of the target marker, such as a slide rail, a wireless remote-controlled car, etc., is provided in the target scene; the target marker can be controlled to slide on the slide rail and thus moved to different positions in the target scene; or the wireless remote-controlled car is manually controlled to control the target marker mounted on the wireless remote-controlled car to reach different positions in the target scene; in this case, the working point is set at a preset relative position to the target marker, and different working points are formed as the target marker is located at different positions of the target scene; by changing the position of the target marker in the target scene, the robot can be controlled to reach different working points.

[0133] When detecting whether the target marker is included in the current frame image, for example, an edge and contour detection algorithm can be used to process the current frame image to obtain observed contour information of multiple objects in the current frame image; then the contour information is matched with the contour information of the target marker; if any contour information successfully matches the contour information of the target marker, it is determined that the target marker is included in the current frame image.

[0134] When it is detected that the current frame image includes the target marker, a first position of the target marker in the target scene is determined.

[0135] In a specific implementation, for example, the first position of the target marker in the target scene may be determined in the following manner:

[0136] determining, based on the current frame image, a first observation position of the target marker in a robot coordinate system corresponding to the robot;

[0137] Based on the current posture of the robot in the target scene and the first observation position of the target marker in the robot coordinate system, a first position of the target marker in the target scene is determined.

[0138] Here, the robot coordinate system corresponding to the robot is, for example, the camera coordinate system corresponding to the vision sensor. For example, a three-dimensional coordinate system is constructed with the optical center of the vision sensor as the origin, the optical axis of the vision sensor as the depth direction (i.e., the z-axis), and the image projection plane of the vision sensor as the plane containing the x-axis and y-axis. This three-dimensional coordinate system is the camera coordinate system, also known as the robot coordinate system. As the robot's position changes, the conversion relationship between the camera coordinate system and the scene coordinate system also changes at any time.

[0139] When determining the first position of the target marker in the target scene based on the current posture of the robot in the target scene and the first observation position of the target marker in the robot coordinate system, for example, the current posture of the robot in the target scene can be used to determine the conversion relationship between the robot coordinate system and the scene coordinate system, and then the first observation position of the target marker in the robot coordinate system can be converted to the scene coordinate system using the conversion relationship to obtain the first position of the target marker in the target scene.

[0140] In another embodiment of the present disclosure, the method further includes: determining a current posture of the robot in the target scene based on the current frame image.

[0141] See also Figure 3 As shown, the embodiment of the present disclosure provides a specific method for determining the current posture of the robot in the target scene based on the current frame image, including:

[0142] S301: Based on the current frame image, determine a second observation pose of at least one target reference object in the target scene in a robot coordinate system.

[0143] Here, a plurality of reference objects are determined in the target scene, and the reference objects are generally various objects with fixed positions set in the target scene. Specifically, for different target scenes, the reference objects are also different.

[0144] For example, if the target scene is a container, the reference objects include: the texture and edge lines of the container's inner wall, the QR code set on the container's inner wall, etc. If the target scene is a warehouse, the reference objects include: the texture and edge lines of the warehouse's wall, certain fixed facilities in the warehouse, the QR code affixed to the warehouse, etc.

[0145] When determining the second observation pose of at least one target reference object in the target scene in the robot coordinate system based on the current frame image, for example, an edge and contour detection algorithm can be used to perform edge detection processing on the current frame image to obtain observation contour information of multiple objects in the current frame image; then the observation contour information is matched with the contour information corresponding to the multiple reference objects; if any observation contour information successfully matches the contour information of a reference object, the reference object is used as the target reference object included in the current frame image.

[0146] Here, the process of determining the target reference object can be executed synchronously with the above-mentioned process of determining the target marker, that is, edge and contour detection processing is performed on the current frame image, and based on the processing results, it is determined whether there is a target marker and the target reference object is determined.

[0147] After the target reference object is determined, the current frame image is used to determine a second observation pose of the target reference object in the robot coordinate system.

[0148] Here, if the current frame image includes n target reference objects, the second observation poses P1, P2, P3...P of the n target reference objects in the robot coordinate system are determined based on the current frame image. n .

[0149] S302: Based on the reference posture of the robot in the target scene and the second observation posture of the at least one target reference object in the robot coordinate system, determine the third observation posture of the at least one target reference object in the target scene.

[0150] Here, when determining the third observation pose of the target reference object in the target scene, for example, the following method can be used:

[0151] Determining, based on a reference pose of the robot in the target scene, conversion relationship information between the robot coordinate system and the scene coordinate system;

[0152] The conversion relationship information is used to convert the first observation position of the at least one target reference object in the robot coordinate system into a second observation posture in the target scene.

[0153] For example, if the reference pose of the robot in the target scene is represented as: O, then the third observation pose P of the i-th target reference object among the n target reference objects is i 'Satisfied: P i '=OP i Among them, P i represents the second observation pose of the i-th target reference object.

[0154] S303: Based on the third observation posture and the actual posture of the at least one target reference object in the target scene, adjust the reference posture to obtain the current posture of the robot.

[0155] In a specific implementation, for example, the following method can be used to obtain the current posture of the robot:

[0156] Perform at least one iteration cycle, and in each iteration cycle: determine a pose error based on a third observed pose corresponding to the at least one target reference object and an actual pose in the target scene; and adjust a reference pose of a current iteration cycle based on the pose error to obtain an adjusted reference pose;

[0157] In response to an iteration stop condition being met, determining the adjusted reference pose obtained in the last iteration cycle as the current pose of the robot;

[0158] The reference posture of the current iteration cycle is determined based on the reference posture of the previous iteration cycle, or based on the initial reference posture.

[0159] For example, the posture error L satisfies the following formula (1):

[0160]

[0161] Among them, e i represents the position error corresponding to the i-th target reference object among the n target reference objects; and e i Satisfy the following formula (2):

[0162] e i =P ti -1 P i ' (2)

[0163] Among them, P ti represents the actual position of the i-th target reference object in the target scene; P i ' represents the third observation pose of the i-th target reference object.

[0164] The pose error can be determined using the above method. The robot's reference in the target scene is then optimized using a nonlinear optimization method, which processes the reference pose over multiple iterations. In each iteration, the pose error determined in that iteration is used to adjust the reference pose corresponding to that iteration, yielding the adjusted reference pose for that iteration.

[0165] After the iteration stop condition is met, the adjusted reference pose obtained in the last iteration cycle is used as the current pose of the robot.

[0166] Among them, for the first iteration cycle, the corresponding reference pose is the initial reference pose.

[0167] For cycles other than the first cycle, the corresponding reference pose is the adjusted reference pose determined in the previous iteration cycle.

[0168] Here, the initial reference pose of the robot may be determined based on, for example, an odometer installed on the robot.

[0169] The iteration stopping condition includes at least one of the following:

[0170] The number of iterations corresponding to the current iteration cycle reaches the preset number of iterations;

[0171] The pose error determined in the current iteration cycle is less than the preset pose error threshold.

[0172] It should be noted that the initial reference pose is only for the current frame image. For other frames, the corresponding initial reference pose is determined based on the odometer count when acquiring other frames.

[0173] Furthermore, it should be noted that, for example, a preset initial position can be determined for the robot in the target scene; when the robot is not in operation, it is parked at this initial position. When the robot is turned on, it uses this initial position as its first position in the target scene. During subsequent operation, each time the driving control method provided by the disclosed embodiment is executed, its position is updated. The odometer count and the previously updated position are then used to determine the initial position corresponding to the current frame image.

[0174] In this way, the current posture of the robot can be determined based on the current frame image. Then, based on the current posture of the robot in the target scene and the first observed position of the target marker in the robot coordinate system, the first position of the target marker in the target scene is determined.

[0175] Regarding the above S203: when controlling the robot to move toward the target working point based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative positional relationship between the target working point and the target marker, for example, the following method can be used:

[0176] Obtaining a second position of the target working point in the target scene based on the first position of the target marker in the target scene and the relative positional relationship between the target working point and the target marker;

[0177] Generate a target path based on the current posture of the robot and the second position of the target workpoint in the target scene;

[0178] The robot is controlled to travel toward the target working point based on the target path.

[0179] In a specific implementation, when generating a target path based on the current posture of the robot and the second position of the target work point in the target scene, for example, any one of the route planning methods including the quintic curve planning method, the Bezier curve planning method, etc. can be used to generate the target path.

[0180] Taking the quintic curve planning method as an example, the coordinates of the robot's path points can be set as: (x, y); the planning problem is modeled as a quintic polynomial problem, and the acceleration is used as the optimization term to construct the constraint equation By giving the starting and ending position constraints, starting and ending angle constraints, speed constraints, and acceleration constraints, the quintic parametric equation about time can be transformed into a linear quadratic form J=u T Hu, by solving the linear quadratic form, a smooth optimal path connecting the start and end points that satisfies the constraints can be obtained. This optimal path is determined as the target path.

[0181] After obtaining the target path, the robot is controlled to move from the current position to the target working point according to the target path.

[0182] It should be noted here that in the process of the robot moving from the current position to the target working point, the robot driving control method provided by the embodiment of the present disclosure can continue to be executed periodically. In this process, the target path will be continuously optimized according to the actual state of the robot to achieve better control of the robot.

[0183] In another embodiment, after the robot reaches the target work point, it can report the completion of the task to the host computer. After receiving the information reported by the robot, the host computer can perform the next operation, such as controlling the robot to place the object at the location corresponding to the work point, or controlling the robot to move the object placed at the target work point to another location, etc. The specific settings can be set according to actual needs and will not be repeated here.

[0184] See also Figure 4 and Figure 5 As shown, the embodiment of the present disclosure provides a specific example of driving control of a robot, including:

[0185] like Figure 4 As shown in Figure a, the robot is located inside a container containing cargo. The robot's visual sensors capture images of the container's inner walls and perform feature detection to obtain observed contour information. This information is then matched against the contour information of a predetermined reference object to determine the robot's current position within the container. X and Y represent the X and Y axes of the container's coordinate system.

[0186] like Figure 4 As shown in b, the robot adjusts its own posture and forward direction according to the current posture to control the robot to travel along a preset path in the container. The preset path is, for example, the center line of the container.

[0187] See also Figure 5 As shown in (b), if the robot detects a target marker while driving along the preset path, it will replan the path, generate the target path, and drive along the target path to the target work point.

[0188] like Figure 5 As shown in Figure a, if the robot does not detect the target marker while traveling along the preset path, it will continue to travel along the preset path.

[0189] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0190] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0191] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0192] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0193] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the scope of protection of the claims.

Claims

1. A robot, characterized in that: include: Vision sensors and control components; The visual sensor is configured to obtain a current frame image during the robot's movement; The control component is configured to: when detecting that the current frame image includes a target marker, determine a first position of the target marker in the target scene; Based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative positional relationship between the target work point and the target marker, the robot is controlled to move toward the target work point with the target marker as a reference; The target marker is movably arranged in the target scene; the working point is arranged at a different position from the target marker, and different working points are formed as the target marker is located at different positions of the target scene.

2. The robot according to claim 1, characterized in that The control component, when determining the first position of the target marker in the target scene, is configured to: determining, based on the current frame image, a first observation position of the target marker in a robot coordinate system corresponding to the robot; Based on the current posture of the robot in the target scene and the first observation position of the target marker in the robot coordinate system, a first position of the target marker in the target scene is determined.

3. The robot according to claim 1 or 2, characterized in that: The control component is further configured to determine the current posture of the robot in the target scene based on the current frame image.

4. The robot according to claim 3, characterized in that The control component, when determining the current posture of the robot in the target scene based on the current frame image, is configured to: Determining, based on the current frame image, a second observation pose of at least one target reference object in the target scene in a robot coordinate system; Determining a third observation pose of the at least one target reference object in the target scene based on the reference pose of the robot in the target scene and the second observation pose of the at least one target reference object in the robot coordinate system; Based on the third observation posture and the actual posture of the at least one target reference object in the target scene, the reference posture is adjusted to obtain the current posture of the robot.

5. The robot according to claim 4, characterized in that The control component is further configured to: Performing edge detection processing on the current frame image to obtain observation contour information in the current frame image; Matching the observed contour information with contour information corresponding to a plurality of reference objects; If any observed contour information matches the contour information of any reference object successfully, the reference object is used as the target reference object included in the current frame image.

6. The robot according to claim 4, characterized in that The control component, when determining a third observation pose of the at least one target reference object in the target scene based on the reference pose of the robot in the target scene and the second observation pose of the at least one target reference object in the robot coordinate system, is configured to: Determining, based on a reference pose of the robot in the target scene, conversion relationship information between the robot coordinate system and the scene coordinate system; The conversion relationship information is used to convert the first observation position of the at least one target reference object in the robot coordinate system into a second observation posture in the target scene.

7. The robot according to claim 4, characterized in that The control component, when adjusting the reference pose based on the third observation pose and the actual pose of the at least one target reference object in the target scene to obtain the current pose of the robot, is configured to: Perform at least one iteration cycle, and in each iteration cycle: determine a pose error based on a third observed pose corresponding to the at least one target reference object and an actual pose in the target scene; and adjust a reference pose of a current iteration cycle based on the pose error to obtain an adjusted reference pose; In response to an iteration stop condition being met, determining the adjusted reference pose obtained in the last iteration cycle as the current pose of the robot; The reference posture of the current iteration cycle is determined based on the reference posture of the previous iteration cycle, or based on the initial reference posture.

8. The robot according to claim 7, characterized in that The control component is further configured to determine the initial reference pose based on an odometer installed on the robot.

9. The robot according to claim 7, characterized in that The iteration stopping condition includes at least one of the following: The number of iterations corresponding to the current iteration cycle reaches the preset number of iterations; The pose error determined in the current iteration cycle is less than the preset pose error threshold.

10. The robot according to claim 1, characterized in that The control component, when controlling the robot to travel toward the target working point based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative positional relationship between the target working point and the target marker, is configured to: Obtaining a second position of the target working point in the target scene based on the first position of the target marker in the target scene and the relative positional relationship between the target working point and the target marker; Generate a target path based on the current posture of the robot and the second position of the target workpoint in the target scene; The robot is controlled to travel toward the target working point based on the target path.

11. A robot driving control method, characterized in that: include: Get the current frame image captured by the visual sensor on the robot; When detecting that the current frame image includes a target marker, determining a first position of the target marker in the target scene; Based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative positional relationship between the target work point and the target marker, the robot is controlled to move toward the target work point with the target marker as a reference; The target marker is movably arranged in the target scene; the working point is arranged at a different position from the target marker, and different working points are formed as the target marker is located at different positions of the target scene.

12. The driving control method according to claim 11, wherein: Determining a first position of the target marker in the target scene includes: determining, based on the current frame image, a first observation position of the target marker in a robot coordinate system corresponding to the robot; Based on the current posture of the robot in the target scene and the first observation position of the target marker in the robot coordinate system, a first position of the target marker in the target scene is determined.

13. The method according to claim 11 or 12, characterized in that Also includes: Based on the current frame image, a current posture of the robot in the target scene is determined.

14. The method according to claim 13, characterized in that Determining the current posture of the robot in the target scene based on the current frame image includes: Determining, based on the current frame image, a second observation pose of at least one target reference object in the target scene in a robot coordinate system; Determining a third observation pose of the at least one target reference object in the target scene based on the reference pose of the robot in the target scene and the second observation pose of the at least one target reference object in the robot coordinate system; Based on the third observation posture and the actual posture of the at least one target reference object in the target scene, the reference posture is adjusted to obtain the current posture of the robot.

15. The method according to claim 14, characterized in that Also includes: Performing edge detection processing on the current frame image to obtain observation contour information in the current frame image; Matching the observed contour information with contour information corresponding to a plurality of reference objects; If any observed contour information matches the contour information of any reference object successfully, the reference object is used as the target reference object included in the current frame image.

16. The method according to claim 14, characterized in that The determining, based on the reference pose of the robot in the target scene and the second observation pose of the at least one target reference object in the robot coordinate system, a third observation pose of the at least one target reference object in the target scene, comprises: Determining, based on a reference pose of the robot in the target scene, conversion relationship information between the robot coordinate system and the scene coordinate system; The conversion relationship information is used to convert the first observation position of the at least one target reference object in the robot coordinate system into a second observation posture in the target scene.

17. The method according to claim 14, characterized in that The adjusting the reference pose based on the third observation pose and the actual pose of the at least one target reference object in the target scene to obtain the current pose of the robot includes: Perform at least one iteration cycle, and in each iteration cycle: determine a pose error based on a third observed pose corresponding to the at least one target reference object and an actual pose in the target scene; and adjust a reference pose of a current iteration cycle based on the pose error to obtain an adjusted reference pose; In response to an iteration stop condition being met, determining the adjusted reference pose obtained in the last iteration cycle as the current pose of the robot; The reference posture of the current iteration cycle is determined based on the reference posture of the previous iteration cycle, or based on the initial reference posture.

18. The method according to claim 17, characterized in that Also includes: The initial reference pose is determined based on an odometer installed on the robot.

19. The method according to claim 17, wherein The iteration stopping condition includes at least one of the following: The number of iterations corresponding to the current iteration cycle reaches the preset number of iterations; The pose error determined in the current iteration cycle is less than the preset pose error threshold.

20. The method according to claim 11, characterized in that The controlling the robot to move toward the target working point based on the current posture of the robot in the target scene, the first position of the target marker in the target scene, and the relative positional relationship between the target working point and the target marker includes: Obtaining a second position of the target working point in the target scene based on the first position of the target marker in the target scene and the relative positional relationship between the target working point and the target marker; Generate a target path based on the current posture of the robot and the second position of the target workpoint in the target scene; The robot is controlled to travel toward the target working point based on the target path.

Images