Robot positioning method, apparatus, scheduling device, medium, and program product

Abstract

Embodiments of the present disclosure provide a robot positioning method and device, a scheduling device, a medium and a program product. For the case of robot team splitting, the method comprises: acquiring a first image collected by a first robot at a team splitting point, and acquiring a second image collected by a second robot at the team splitting point, the first robot being a team leader of an original team, and the second robot being a new team leader after team splitting; determining a first target image matched with the first image from an environment image library according to the first image; judging whether a first pose of the second robot at the team splitting point is consistent with a second pose of the first robot at the team splitting point according to the first image and the second image; and if the first pose is consistent with the second pose, positioning the second robot according to the first target image, thereby realizing positioning of the second robot based on environment information of the first robot, reducing data processing amount of the scheduling device, and improving positioning efficiency.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of warehouse system, and particularly relates to a robot positioning method and device, a scheduling device, a medium and a program product. BACKGROUND

[0002] Warehouse is an important link of logistics process. Robots can replace manual handling of goods and play an important role in intelligent warehouse logistics.

[0003] In order to construct a map of a warehouse system or to navigate a robot, positioning of the robot is needed. A commonly used visual positioning technology is SLAM (Simultaneous Localization and Mapping). For SLAM, loop detection is an indispensable step for map construction and robot navigation.

[0004] When robots travel in a warehouse in the form of a fleet, a server or a scheduling device needs to perform robot positioning based on images collected by each robot in the fleet, which results in a large number of repeated calculation steps of the server or the scheduling device, a large server computing overhead, and high robot positioning cost and low efficiency. SUMMARY

[0005] The present disclosure provides a robot positioning method, device, scheduling device, medium and program product. For the case of splitting of a robot fleet, the scheduling device omits the step of loop detection of images collected by a second robot at a split fleet positioning point in a manner of image inheritance, reduces the computing overhead of the scheduling device, and further reduces the cost of robot positioning and improves the efficiency of robot positioning.

[0006] In a first aspect, an embodiment of the present disclosure provides a robot positioning method. The method is applied to a scheduling device, and the method comprises:

[0007] According to a first image collected by a first robot at a formation point, a first target image matching the first image is determined from an environment image library; a second image collected by a second robot at the formation point is obtained; whether a first pose of the second robot at the formation point is consistent with a second pose of the first robot at the formation point is judged according to the first image and the second image; if the first pose is consistent with the second pose, positioning of the second robot is performed according to the first target image, so as to update the environment image library or guide the second robot; wherein the first robot and the second robot are located in a first vehicle formation, the first robot is a head robot of the first vehicle formation; the first vehicle formation is split into a second vehicle formation and a third vehicle formation at the formation point, the first robot is a head robot of the second vehicle formation, the second robot is a head robot of the third vehicle formation, and the number of robots included in the second vehicle formation and the third vehicle formation is at least one.

[0008] Optionally, the first target image matching the first image is determined from the environment image library according to the first image collected by the first robot at the formation point, comprising:

[0009] According to the first image, whether there is an environment image matching the first image in the environment image library is judged based on a loop closure detection algorithm; if yes, loop closure detection is successful, and the matched environment image is determined as the first target image of the first image.

[0010] Optionally, if the first pose is not consistent with the second pose, the method further comprises:

[0011] According to the first pose and the second pose, the pose of the second robot is adjusted; at least one third image collected by the second robot during the adjustment of the pose is obtained; whether there is a third pose of the second robot corresponding to at least one third image matching the first pose is judged; if yes, positioning of the second robot is performed according to the first target image, so as to update the environment image library or guide the second robot.

[0012] Optionally, if the first pose is not consistent with the second pose, the method further comprises:

[0013] According to the second image, a second target image matching the second image is determined from the environment image library; positioning of the second robot is performed according to the second target image.

[0014] Optionally, the method further comprises:

[0015] When the formation point is determined, the sensors to be turned on by the second robot are determined according to the role of the second robot in the third vehicle team, wherein the role of the robot in the vehicle team includes a team leader, a team follower and a team member.

[0016] In a second aspect, the embodiments of the present disclosure further provide another robot positioning method, which is applied to a scheduling device, and the method comprises the following steps:

[0017] According to the first image collected by the first robot at the formation point, a first target image matching the first image is determined from an environment image library; a state parameter of the first robot when collecting the first image is obtained, and the first image and the state parameter are sent to the second robot to control the second robot to adjust the state of the second robot based on the state parameter; wherein the state parameter comprises a first pose of the first robot when collecting the first image; when receiving a pose consistent message fed back by the second robot, the positioning of the second robot is performed according to the first target image, so as to update the environment image library or perform navigation of the second robot, wherein the pose consistent message is generated when the second robot determines that the pose of the second robot at the formation point is consistent with the first pose.

[0018] Optionally, the state parameter further comprises a first state parameter, and the first state parameter is used to describe the position parameter of the sensor turned on by the first robot when collecting the first image at the formation point; the first state parameter is used to control the second robot to turn on the sensor at the corresponding position in the first state parameter, so as to perform image collection based on the turned-on sensors.

[0019] In a third aspect, the embodiments of the present disclosure further provide a robot positioning device, which is applied to a scheduling device, and the device comprises:

[0020] The image acquisition first module is configured to acquire a first image collected by a first robot at a formation point and acquire a second image collected by a second robot at the formation point; the first determination module is configured to determine a first target image matched with the first image from an environment image library according to the first image; the pose determination module is configured to determine whether a first pose of the second robot at the formation point is consistent with a second pose of the first robot at the formation point according to the first image and the second image; and the image inheritance first module is configured to, if the first pose is consistent with the second pose, perform positioning of the second robot according to the first target image, to update the environment image library or perform navigation of the second robot; wherein the first robot and the second robot are located in a first vehicle formation, the first robot is a head robot of the first vehicle formation, the first vehicle formation is split into a second vehicle formation and a third vehicle formation at the formation point, the first robot is a head robot of the second vehicle formation, the second robot is a head robot of the third vehicle formation, and the second vehicle formation and the third vehicle formation each include at least one robot.

[0021] In a fourth aspect, the embodiments of the present disclosure further provide a robot positioning device, which is applied to a scheduling device, and the device comprises:

[0022] The second determination module is configured to determine a first target image matched with a first image collected by a first robot at a formation point from an environment image library; the state parameter sending module is configured to acquire a state parameter of the first robot when collecting the first image, and send the first image and the state parameter to a second robot, to control the second robot to adjust a state of the second robot based on the state parameter; wherein the state parameter comprises a first pose of the first robot when collecting the first image; and the image inheritance second module is configured to, when receiving a pose consistency message fed back by the second robot, perform positioning of the second robot according to the first target image, to update the environment image library or perform navigation of the second robot; wherein the pose consistency message is generated by the second robot when determining that a pose of the second robot at the formation point is consistent with the first pose.

[0023] In a fifth aspect, the embodiments of the present disclosure further provide a scheduling device, which comprises a memory and at least one processor; the memory stores computer execution instructions; and the at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor performs the robot positioning method provided by any of the embodiments of the present disclosure.

[0024] In a sixth aspect, the embodiments of the present disclosure further provide a computer readable storage medium, wherein computer execution instructions are stored in the computer readable storage medium, and when a processor executes the computer execution instructions, a robot positioning method provided by any of the embodiments of the present disclosure is implemented.

[0025] In a seventh aspect, the embodiments of the present disclosure further provide a computer program product, comprising a computer program, and when the computer program is executed by a processor, a robot positioning method provided by any of the embodiments of the present disclosure is implemented.

[0026] The robot positioning method, device, scheduling device, medium and program product provided by the embodiments of the present disclosure are used for the case that a robot vehicle group walking in a group is split into multiple vehicle groups at a split point, a first robot is a head robot of the original robot vehicle group, and a second robot is a newly added head robot after splitting. By using the first image and the second image collected by the first robot and the second robot at the split point, it is determined whether the poses of the first robot and the second robot at the split point are consistent. If yes, the positioning of the second robot is performed by inheriting the first target image of the first robot, that is, by the way of inheriting the first target image of the first robot, so that the step of performing closed loop detection of the second robot at the split point is omitted, the operation cost of the scheduling device is reduced, and the efficiency of robot positioning is improved. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure together with the specification.

[0028] Figure 1 An application scenario diagram of the robot positioning method provided by the embodiments of the present disclosure;

[0029] Figure 2 A flowchart of the robot positioning method provided by one of the embodiments of the present disclosure;

[0030] Figure 3 A flowchart of the robot positioning method provided by another embodiment of the present disclosure; Figure 2 A schematic diagram of splitting a robot vehicle group in the embodiment of the present disclosure;

[0031] Figure 4 A flowchart of the robot positioning method provided by another embodiment of the present disclosure;

[0032] Figure 5 A flowchart of the robot positioning method provided by another embodiment of the present disclosure;

[0033] Figure 6 A structural schematic diagram of the robot positioning device provided by one of the embodiments of the present disclosure;

[0034] Figure 7A structural schematic diagram of a robot positioning device provided for another embodiment of the present disclosure is shown in FIG. 8.

[0035] Figure 8 A structural schematic diagram of a scheduling device provided for an embodiment of the present disclosure is shown in FIG. 9.

[0036] The specific embodiments of the present disclosure have been shown and described in the above-mentioned drawings, and will be described in more detail hereinafter. These drawings and the written description are not intended to limit the scope of the present disclosure in any way, but to illustrate the concept of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0037] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to indicate the same or similar components. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0038] The technical solutions of the present disclosure and how the technical solutions of the present disclosure solve the above-mentioned technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the present disclosure will be described below with reference to the drawings.

[0039] The application scenarios of the embodiments of the present disclosure will be explained below:

[0040] Figure 1 An application scenario diagram of a robot positioning method provided for an embodiment of the present disclosure is shown in FIG. 10. Figure 1 As shown in FIG. 10, when constructing a warehouse system map, robot positioning needs to be performed. The main process of the positioning technology based on visual SLAM is as follows: the robot 110 uploads the detection images collected by sensors such as image sensors and inertial sensors to the scheduling device 120. The scheduling device 120 determines the camera poses corresponding to each frame of detection image based on the detection images received at each time, and performs loop detection or closed loop detection on each frame of detection image based on the image library to determine whether the robot has previously arrived at the position at which the frame of image is collected, thereby positioning the robot 110. Then, based on the loop detection result and the camera poses corresponding to each frame of detection image, the images in the image library are optimized to obtain a globally consistent trajectory and map, thereby constructing or optimizing the warehouse system map.

[0041] Loop detection is a key step for robot positioning based on SLAM technology. The robot 110 determines whether it has been to the current position before by comparing the image currently collected by the robot 110 with each frame image in the pre-stored environment image or map, so as to position the robot 110 or adjust the constructed map.

[0042] In some embodiments, in order to improve the efficiency of picking and placing goods or the efficiency of constructing a map, the robot 110 can travel in the form of a team, that is, a team composed of multiple robots 110, and travel in the form of a robot team.

[0043] In the robot team, the roles of the robots 110 include a team leader, a team tail, and team robots. The team leader is the first robot in the robot team along the walking direction, the team tail is the last robot in the robot team along the walking direction, and the team robots are the robots between the team tail and the team leader.

[0044] When performing tasks in the form of a robot team, such as constructing a map, patrolling, picking and placing goods, and the like, each robot 110 in the robot team needs to detect the environment based on sensors, and then the scheduling device 120 or the server performs positioning, obstacle avoidance, and the like based on the data collected by the sensors of the robots. When the robot 110 travels in the form of a team, compared with the form of a single robot 110, the amount of data to be processed by the scheduling device 120 or the server increases exponentially.

[0045] In order to reduce the data processing task of the scheduling device 120, the present disclosure provides a robot positioning method for the case of splitting the team after the robots form a team. The positioning of the newly added team leader, that is, the second robot, is performed by inheriting the positioning result or the target image of the original team leader, that is, the first robot, and the step of performing loop detection or target image determination on the image collected by the second robot, that is, the second image, is omitted.

[0046] Figure 2 The flowchart of the robot positioning method provided by an embodiment of the present disclosure is shown in Figure 2 The robot positioning method is applicable to a warehouse system and can be executed by a scheduling device. The robot positioning method provided by the embodiment includes the following steps:

[0047] In step S201, a first target image matching a first image collected by a first robot at a split team positioning point is determined from an environment image library.

[0048] The split point is a location where the first robot is located in the first vehicle team for team splitting. The environment image library is a plurality of images corresponding to a map of the established warehouse system. The first robot is a head robot of the first vehicle team and the second vehicle team, and the head robot is a robot located at the front in a walking direction of the vehicle team to which the robot belongs.

[0049] In some embodiments, the first vehicle team is divided into two vehicle teams, the second vehicle team and the third vehicle team, at the split point, the first robot is a head robot of the second vehicle team, and the second robot is a head robot of the third vehicle team.

[0050] In some embodiments, when the number of robots in the vehicle team to which the robot belongs is one, the robot is a single-walking robot.

[0051] Specifically, the first target image of the first image can be determined from the environment image library by feature matching, that is, an image with the most matching features or the highest similarity to the first image.

[0052] Specifically, the first target image of the first image can be determined from the environment image library according to the first image and an odometer corresponding to the first image. The odometer corresponding to the first image can be determined according to data collected by an inertial sensor, a wheel speed meter or other sensors of the first robot collecting the first image.

[0053] Optionally, the first target image of the first image is determined from the environment image library according to the first image collected by the first robot at the split point, comprising:

[0054] Based on a loop closure detection algorithm, whether there is an environment image matching the first image in the environment image library is determined according to the first image. If yes, loop closure detection is successful, and the matched environment image is determined as the first target image of the first image.

[0055] The loop closure detection algorithm can be any algorithm for loop closure detection, such as BA (Bundle Adjustment), bag-of-words model, similarity calculation, deep learning, etc.

[0056] Specifically, after obtaining the first image, the scheduling device can extract features of the first image, and then determine whether there is an environment image matching the first image in the environment image library based on the loop closure detection algorithm and the features of the first image. If there is an environment image matching the first image, loop closure detection is successful, and the environment image matching the first image, i.e., the first target image, is output.

[0057] Specifically, based on the first image and the first target image, i.e., the result of the loop closure detection of the first image, the update of each frame image in the environment image library can be performed to reduce the inter-frame error of the environment image library, thereby reducing the navigation or positioning error of the robot.

[0058] In step S202, a second image collected by the second robot at the split point of the team is obtained.

[0059] The first robot and the second robot are located in a first team, and the first robot is a head robot of the first team. The first team is split into a second team and a third team at the split point of the team, the first robot is a head robot of the second team, and the second robot is a head robot of the third team. The second team and the third team each include at least one robot.

[0060] In some embodiments, the second robot can be any robot in the first team except the first robot, such as a tail robot or a middle robot.

[0061] In some embodiments, each robot mentioned in the present disclosure, including the first robot and the second robot, is the same robot, and the type, size, and other parameters of the robot are the same. The positions and quantities of the various vision sensors on the robot are the same.

[0062] Specifically, the split point of the team is a position point at which the first team including the first robot and the second robot is split. The first robot is a head robot of the first team. When the first robot and the second robot walk to the split point of the team in the form of the first team, the first team is split into two teams, i.e., the second team and the third team. The first robot is a head robot of the second team, the second robot is a head robot of the third team, and the second team and the third team have different walking directions at the split point of the team.

[0063] For example, Figure 3 For the purpose of the present disclosure Figure 2 The schematic diagram of the split of the team of the robot is shown in the embodiments, for example, Figure 3 For example, the first team includes three robots, i.e., robot 301 to robot 303. In the first team, the robot 301 is a head robot, the robot 302 is a middle robot, and the robot 303 is a tail robot. When the head robot of the first team, i.e., the robot 301, reaches the next intersection C1, the tail robot of the first team, i.e., the robot 303, needs to leave the original first team and walk to the road r1 on the left side of the intersection C1. The robot 301 and the robot 302 continue to walk straight at the intersection C1, i.e., along the road r2, Figure 3The first vehicle team is represented by the dashed box, and the intersection is a team setting point. The robot 301 is a first robot, and the robot 303 is a second robot.

[0064] In some embodiments, at least one visual sensor, such as a laser sensor, an image sensor, etc., can be arranged on each side of the robot to collect images of the external environment of the robot. For example, at least one visual sensor can be arranged on each side of the robot chassis, and the visual sensor can also be arranged on the temporary storage shelf or the goods taking and placing device of the robot. The visual sensor can be divided into a front-view sensor, a rear-view sensor, and a side-view sensor. The front-view sensor is used to collect images of the environment in front of the robot in the walking direction, the rear-view sensor is used to collect images of the environment behind the robot in the walking direction, and the side-view sensor is used to collect images of the environment on the left or right side of the robot in the walking direction.

[0065] In some embodiments, when the robot is in the form of a robot team or walks alone, the robot can turn on the visual sensors arranged thereon, such as the front-view sensor, the rear-view sensor, and the side-view sensor described above.

[0066] Specifically, when the robot is in the form of a robot team or the role of the robot in the robot team is updated, the robot can determine the sensors required to be turned on according to the role of the robot in the robot team. The role of the robot in the team includes a team leader, a team member, and a team follower, or can only include a team leader and a team member.

[0067] In some embodiments, the sensors turned on by the team leader in the robot team can be the front-view sensor and the side-view sensor, the sensors turned on by the team member can be the side-view sensor, and the sensors turned on by the team follower can be the rear-view sensor and the side-view sensor.

[0068] In some embodiments, the sensors turned on by the team leader in the robot team can be the front-view sensor and the side-view sensor, the sensors turned on by the team member and the team follower can be the rear-view sensor and the side-view sensor.

[0069] Specifically, the robot in the form of a team or a robot team can turn on the sensors according to a set period to collect images of the external environment of the robot. The dispatching device performs feature extraction, image matching, odometer calculation, etc. on the images of the external environment uploaded by each robot to perform positioning of the robot or reconstruction of the map.

[0070] In the first team, since the first robot is in front of the second robot along the walking direction of the first team, the first robot reaches the teaming point earlier than the second robot. When the first robot reaches the teaming point, a first image is collected at the teaming point based on the sensor opened by the first robot, and the first image is uploaded to the scheduling device; the scheduling device performs positioning of the first robot based on the first image; and then when the second robot reaches the teaming point, a second image is collected at the teaming point based on the sensor opened by the second robot, and the second image is uploaded to the scheduling device.

[0071] Specifically, the position corresponding to the image uploaded by the robot (the first robot or the second robot) can be determined by the inertial sensor, the gyroscope, the wheel speed meter and the like arranged on the robot, so as to obtain the image collected by the robot at the teaming point, i.e., the first image or the second image.

[0072] In step S203, whether the first pose of the second robot at the teaming point is consistent with the second pose of the first robot at the teaming point is judged according to the first image and the second image.

[0073] The first pose is the pose of the first robot when the first image is collected at the teaming point, and the second pose is the pose of the second robot when the second image is collected at the teaming point.

[0074] Specifically, whether the first pose of the second robot at the teaming point is consistent with the second pose of the first robot at the teaming point can be judged by extracting the image features of the first image and the second image and based on the image feature matching mode. If the image features of the first image and the second image are matched or the similarity of the images is greater than a preset value, it is determined that the first pose and the second pose are consistent.

[0075] Specifically, whether the first pose and the second pose are consistent can be judged based on the image features of the same key region in the first image and the second image.

[0076] In step S204, if the first pose and the second pose are consistent, positioning of the second robot is performed according to the first target image to update the environment image library or to perform navigation of the second robot.

[0077] Specifically, if the first pose and the second pose are consistent, it can be determined that the orientations of the first robot and the second robot when reaching the teaming point are consistent, and then the positioning result of the first robot or the result of loop detection, i.e., the first target image, can be directly used for positioning of the second robot, so as to omit the process of the scheduling device searching for the second target image matching the second image in the environment image library, and reduce the operation cost of the scheduling device.

[0078] The robot positioning method provided by the embodiments of the present disclosure is used for the case that a robot team walking in a team mode is split into multiple teams at a team split point. A first robot is a team leader of the original robot team, and a second robot is a newly added team leader after the split. The first robot and the second robot collect a first image and a second image at the team split point. It is determined whether the poses of the first robot and the second robot at the team split point are consistent. If yes, the positioning of the second robot is performed by inheriting the first target image of the first robot, so that the step of performing closed-loop detection of the second robot at the team split point is omitted, the computational overhead of the scheduling device is reduced, and the efficiency of robot positioning is improved.

[0079] In some embodiments, when the second robot reaches the team split point, the walking speed is reduced to split the team, such as changing direction. If the first pose and the second pose are inconsistent, the second robot cannot perform positioning based on the positioning result of the first robot, that is, the first target image.

[0080] Optionally, if the first pose and the second pose are inconsistent, the method further includes:

[0081] According to the second image, a second target image matching the second image is determined from the environment image library, and positioning of the second robot is performed according to the second target image.

[0082] Specifically, if the first pose and the second pose are inconsistent, it indicates that the pose of the second robot when reaching the team split point is different from the pose of the first robot when reaching the team split point. Then, a second target image matching the second image is determined from the environment image library according to the second image collected by the second robot at the team split point. The specific method is similar to step S201, and only the first image is replaced by the second image. Further, positioning of the second robot is performed according to the second target image to perform navigation of the second robot, and each frame of image in the environment image library is updated based on the second image and the second target image or the closed-loop detection result of the second image, so as to reduce the inter-frame error of the environment image library and thus reduce the navigation or positioning error of the robot.

[0083] In some embodiments, when the second robot walks to the team split point, the second robot and each robot located behind the second robot in the walking direction can pause walking until a walking control instruction sent by the scheduling device is received.

[0084] Specifically, when the first pose is inconsistent with the second pose, the pose of the second robot at the formation positioning point can also be adjusted, and one or more frames of third images collected by the second robot at the formation positioning point after the pose adjustment are obtained. If the third pose of the second robot corresponding to at least one frame of third images is consistent with the first pose, positioning of the second robot is performed based on the first target image, and a walking control instruction of the second robot is generated based on the positioning result, so that the second robot walks in a set direction to reach a destination or a next path node of the second robot.

[0085] Figure 4 A flowchart of a robot positioning method provided for another embodiment of the present disclosure is shown in FIG. 4. The robot positioning method provided by the present embodiment is based on the robot positioning method provided by the embodiment shown in FIG. 3. Figure 2 Based on the embodiment shown in FIG. 3, after step S203, the case where the first pose is inconsistent with the second pose is added, and the step of determining the sensors turned on by the second robot before obtaining the second image is added, as shown in FIG. 4. Figure 4 The robot positioning method provided by the present embodiment can include the following steps:

[0086] Step S401: Obtain a first image collected by a first robot at a formation positioning point.

[0087] Specifically, the first image can be obtained based on data collected by the sensors turned on by the first robot when walking to the formation positioning point, such as front-view sensors and various side-view sensors.

[0088] Step S402: Determine a first target image matched with the first image from an environment image library according to the first image.

[0089] Step S403: When the formation positioning point is determined, determine the sensors turned on by the second robot according to the role of the second robot in the third vehicle formation.

[0090] The role of the robot in the vehicle formation includes a head robot, a tail robot, and a middle robot.

[0091] Since the role of the second robot in the vehicle formation changes from a tail robot or a middle robot in the first vehicle formation to a head robot in the third vehicle formation at the formation positioning point, the sensors turned on by the second robot need to be determined again. Specifically, the sensors turned on by the second robot, such as front-view sensors and various side-view sensors, can be determined according to the role of the second robot in the third vehicle formation, i.e., the head robot.

[0092] Specifically, when the role of each robot in the vehicle formation changes, the sensors turned on by the robot can be determined based on the new role of the robot.

[0093] For example, the sensors turned on by the head robot in the robot team can be a front-view sensor and each side-view sensor, the sensors turned on by the robots in the team can be each side-view sensor, and the sensors turned on by the tail robot can be a rear-view sensor and each side-view sensor. Alternatively, the sensors turned on by the head robot in the robot team can be a front-view sensor and each side-view sensor, and the sensors turned on by the robots in the team and the tail robot can be each side-view sensor.

[0094] In step S404, a second image of the second robot at the sub-team designated point is acquired based on the sensors turned on by the second robot.

[0095] Specifically, when the second robot walks to the sub-team designated point, the second image of the second robot at the sub-team designated point can be acquired based on the sensors turned on by the second robot, such as a front-view sensor and each side-view sensor.

[0096] In step S405, whether the first pose of the second robot at the sub-team designated point and the second pose of the first robot at the sub-team designated point are consistent is determined according to the first image and the second image.

[0097] Specifically, since the roles of the first robot and the second robot at the sub-team designated point are the same, both of which are head robots, whether the poses of the first robot and the second robot at the sub-team designated point, i.e., the first pose and the second pose, are consistent can be determined by comparing the features of the first image and the second image.

[0098] Specifically, the images of each sensor of the first robot and the second robot at the sub-team designated point, i.e., the first image and the second image, can be determined in turn according to the corresponding sensors of the images, i.e., I 1i and I 2i are compared in turn, and whether the first pose and the second pose are consistent is determined according to the comparison result.

[0099] wherein i = 1, 2, …, n, n is the total number of the sensors turned on by the head robot, I 1i is the first image acquired by the i-th sensor turned on by the first robot, and I 2i is the second image acquired by the i-th sensor turned on by the second robot.

[0100] Specifically, if I 1i and I 2i , i = 1, 2, …, n, are matched, it is determined that the first pose and the second pose are consistent. Alternatively, if the first images and the second images acquired by at least m sensors are matched, it is determined that the first pose and the second pose are consistent. Wherein m is a positive integer less than n.

[0101] Specifically, the first pose of the first robot at the formation point can be determined according to each first image corresponding to the first robot, and the second pose of the second robot at the formation point can be determined based on each second image corresponding to the second robot. Then, it is determined whether the first pose and the second pose are consistent, for example, whether the pose difference between the first pose and the second pose is less than a preset difference. If yes, it is determined that the first pose and the second pose are consistent. The pose difference can be represented by two parameters, i.e., the difference of the orientation angle and the difference of the distance to the formation point.

[0102] In step S406, if the first pose and the second pose are consistent, the positioning of the second robot is performed according to the first target image to update the environment image library or to guide the navigation of the second robot.

[0103] Further, if the pose difference between the first pose and the second pose is too large, for example, the difference of the distance to the formation point is greater than a first difference, or the difference of the orientation angle is greater than a second difference, a second target image matching the second image is determined from the environment image library according to the second image, and the positioning of the second robot is performed according to the second target image.

[0104] In step S407, if the first pose and the second pose are inconsistent, the pose of the second robot is adjusted according to the first pose and the second pose.

[0105] Specifically, when the first pose and the second pose are inconsistent, the pose adjustment instruction of the second robot can be generated based on the pose difference between the first pose and the second pose, and the position and the orientation of the second robot are adjusted based on the pose adjustment instruction to change the pose of the second robot at the formation point.

[0106] Specifically, the pose adjustment instruction of the second robot can also be generated according to the difference of the image coordinates of the corresponding points of the first image and the second image, and the pose of the second robot is adjusted based on the pose adjustment instruction.

[0107] Further, when the second pose is inconsistent with the first pose, the type and the priority of the task performed by each robot in the second robot or the third vehicle group are obtained, and when the type and the priority of the task performed by each robot in the second robot or the third vehicle group meet a preset condition, the pose adjustment instruction of the second robot is generated according to the first pose and the second pose. If the type and the priority of the task performed by each robot in the second robot or the third vehicle group do not meet the preset condition, a second target image matching the second image is determined from the environment image library according to the second image, and the positioning of the second robot is performed according to the second target image.

[0108] For example, the preset condition can be that the type of the task is a setting type, and the task priority is lower than a preset priority.

[0109] In step S408, at least one third image collected by the second robot is acquired during the pose adjustment of the second robot.

[0110] Specifically, the specific manner of acquiring the third image is similar to that of acquiring the second image, and only the time is different, which will not be described here.

[0111] In step S409, it is determined whether there is a third pose of the second robot corresponding to at least one third image that is consistent with the first pose.

[0112] Specifically, feature extraction can be performed on the third image, and based on the features corresponding to the third image, the third pose of the second robot corresponding to each third image is determined, and then it is determined whether there is a pose consistent with the first pose in each third pose.

[0113] In step S410, if there is, the positioning of the second robot is performed according to the first target image to update the environment image library or to navigate the second robot.

[0114] Specifically, if there is at least one third pose corresponding to the third image that is consistent with the first pose, the positioning of the second robot can be performed based on the first target image of the first robot, that is, the second robot inherits the result of the loop detection or positioning of the first robot, so that it is not necessary to find a target image matching the second image from the environment image library including a large number of images, the computing resources of the scheduling device are saved, and the positioning efficiency of the second robot in the formation point positioning is improved.

[0115] Further, if there is no third pose corresponding to the third image that is consistent with the first pose, a second target image matching the second image or the third image is determined from the environment image library according to the second image or the third image; and the positioning of the second robot is performed according to the second target image.

[0116] In the embodiment, for the case that the robot team for team walking is split into multiple teams at a split point, based on the role of the second robot in the new team, i.e., the third team obtained after the first team is split, the sensors started by the second robot are automatically controlled, and then based on the second image collected by the started sensors at the split point and the first image collected by the original team leader robot, i.e., the first robot, at the split point, it is determined whether the poses of the second robot and the first robot at the split point are consistent. If consistent, the second robot inherits the matching result or loop detection result of the first robot, and performs positioning of the second robot based on the first target image of the first robot. If not consistent, the pose of the second robot at the split point is adjusted to obtain a new image collected by the second robot at the split point, i.e., a third image, so as to determine whether the pose of the second robot corresponding to the third image is consistent with the pose of the first robot corresponding to the first image. If consistent, the second robot inherits the matching result or loop detection result of the first robot, and performs positioning of the second robot based on the first target image of the first robot. The image matching result of the robot with the same pose and role at the split point is inherited, the data processing amount of the scheduling device is reduced, and the efficiency of robot positioning is improved.

[0117] Figure 5 A flowchart of a robot positioning method provided for another embodiment of the present disclosure is provided. The robot positioning method provided by the embodiment is executed by a scheduling device, as shown in Figure 5 The robot positioning method provided by the embodiment includes the following steps:

[0118] Step S501, determining a first target image matched with the first image from an environment image library according to a first image collected by a first robot at a split point.

[0119] Step S502, obtaining a state parameter of the first robot when collecting the first image, and sending the first image and the state parameter to a second robot to control the second robot to adjust the state of the second robot based on the state parameter.

[0120] The state parameter includes a first pose of the first robot when collecting the first image.

[0121] Specifically, the first robot can send the state parameter and the first image when the first image is collected to the scheduling device after collecting the first image at the formation point. The scheduling device sends the state parameter and the first image to the second robot, so that the second robot adjusts the pose of the second robot at the formation point based on the state parameter, collects the second image, and determines whether the collected second image matches the first image, such as similarity greater than a preset value. If yes, it is determined that the pose of the second robot at the formation point is consistent with the first pose of the first robot, and a pose consistent message is generated.

[0122] Specifically, the second robot can determine whether the second image collected by the second robot at the formation point matches the first image; if it matches, it is determined that the current pose of the second robot at the formation point is consistent with the first pose; if it does not match, the pose of the second robot is fine-tuned, or the pose of the second robot is adjusted based on the pose difference between the current pose and the first pose, and a new image is collected during the adjustment of the pose of the second robot. Determine whether there is an image matching the first image in the new image, if yes, it is determined that the pose of the second robot is consistent with the first pose.

[0123] Optionally, the state parameter further comprises a first state parameter, the first state parameter is used to describe the position parameter of the sensor opened by the first robot when collecting the first image at the formation point; the first state parameter is used to control the second robot to open the sensor at the corresponding position in the first state parameter, so as to collect the image based on each opened sensor.

[0124] Specifically, when the second robot walks to the formation point, the sensor of the second robot can be opened based on the first state parameter in the state parameter, so that the second robot is consistent with the sensor opened by the first robot at the formation point, such as front view sensor and each side view sensor; and then the second image is collected based on each opened sensor.

[0125] Step S503, when receiving the pose consistent message fed back by the second robot, positioning the second robot according to the first target image to update the environment image library or navigate the second robot.

[0126] Among them, the pose consistent message is generated when the second robot determines that the pose of the second robot at the formation point is consistent with the first pose.

[0127] Specifically, when the second robot determines that the image collected by the second robot at the split formation point matches the first image or the similarity is greater than a preset value through image feature matching, it is determined that the pose of the second robot at the split formation point is consistent with the first pose, and a pose consistency message is generated and sent to the scheduling device. After receiving the pose consistency message fed back by the second robot, the scheduling device directly performs positioning of the second robot based on the first target image, thereby omitting the process of searching for an image matching the second image from the environment image library, i.e., omitting the process of closed-loop detection of the second image, reducing the calculation amount of the scheduling device and improving the positioning efficiency of the second robot at the split formation point.

[0128] In this embodiment, for the case that the robot vehicle group walking in formation is split into multiple vehicle groups at the split formation point, the first robot is the head robot of the original robot vehicle group, and the second robot is the head robot newly added after splitting. Based on the state parameters of the first robot when collecting the first image at the split formation point, the second robot is controlled to collect the second image at the split formation point with the same state parameters, and then the second robot judges whether the image features of the first image and the second image match, i.e., whether the poses of the first robot and the second robot at the split formation point are consistent. If consistent, a pose consistency message is fed back to the scheduling device, and the scheduling device directly performs positioning of the second robot based on the first target image, thereby omitting the steps of closed-loop detection or image matching of the second robot at the split formation point, reducing the operation overhead of the scheduling device and improving the efficiency of robot positioning.

[0129] Figure 6 The structural schematic diagram of the robot positioning device provided by one embodiment of the present disclosure is shown in FIG. 1. Figure 6 As shown in FIG. 1, the robot positioning device is applied to a scheduling device, and the device includes a first determination module 610, an image acquisition module 620, a pose judgment module 630, and an image inheritance first module 640.

[0130] The first determining module 610 is configured to determine a first target image matched with the first image from an environment image library according to the first image collected by the first robot at the formation point; the image obtaining module 620 is configured to obtain a second image collected by a second robot at the formation point; the pose determining module 630 is configured to determine whether a first pose of the second robot at the formation point is consistent with a second pose of the first robot at the formation point according to the first image and the second image; the image inheritance first module 640 is configured to, if the first pose is consistent with the second pose, perform positioning of the second robot according to the first target image, so as to update the environment image library or perform navigation of the second robot; wherein the first robot and the second robot are located in a first vehicle formation, the first robot is a head robot of the first vehicle formation; the first vehicle formation is split into a second vehicle formation and a third vehicle formation at the formation point, the first robot is a head robot of the second vehicle formation, the second robot is a head robot of the third vehicle formation, and the number of robots included in the second vehicle formation and the third vehicle formation is at least one.

[0131] Optionally, the first determining module 610 is specifically configured to:

[0132] According to a closed-loop detection algorithm, it is determined whether there is an environment image matched with the first image in the environment image library according to the first image; if there is an environment image matched with the first image in the environment image library, the closed-loop detection is successful, and the matched environment image is determined as a first target image of the first image.

[0133] Optionally, the apparatus further comprises:

[0134] The pose adjusting module is configured to, if the first pose is not consistent with the second pose, adjust a pose of the second robot according to the first pose and the second pose; the third image obtaining module is configured to obtain at least one third image collected by the second robot during the adjustment of the pose of the second robot; the pose consistency determining module is configured to determine whether there is at least one third pose of the second robot corresponding to the at least one third image, which is consistent with the first pose; and the image inheritance third module is configured to, if there is at least one third pose of the second robot corresponding to the at least one third image, which is consistent with the first pose, perform positioning of the second robot according to the first target image, so as to update the environment image library or perform navigation of the second robot.

[0135] Optionally, the apparatus further comprises:

[0136] The second target image determination module is used to determine a second target image that matches the second image from the environmental image library if the first pose and the second pose are inconsistent; the second robot localization module is used to localize the second robot based on the second target image.

[0137] Optionally, the device further includes:

[0138] The sensor activation module is used to determine the sensors activated by the second robot based on the role of the second robot in the third convoy when the convoy is fixed at its location. The robot's role in the convoy includes the head robot, the tail robot, and the middle robot.

[0139] The robot positioning device provided in this disclosure embodiment can execute this disclosure. Figure 2 or Figure 4 The robot localization method provided in the corresponding embodiments has the corresponding functional modules and beneficial effects of the execution method.

[0140] Figure 7 This is a schematic diagram of the structure of a robot positioning device provided in another embodiment of the present disclosure, as shown below. Figure 7 As shown, the robot positioning device is applied to a scheduling device. The device includes: a second determination module 710, a status parameter sending module 720, and an image inheritance second module 730.

[0141] The second determining module 710 is used to determine a first target image matching the first image from an environmental image library based on the first image collected by the first robot at a designated point in the team. The state parameter sending module 720 is used to obtain the state parameters of the first robot collecting the first image and send the first image and the state parameters to the second robot to control the second robot to adjust its state based on the state parameters. The state parameters include the first pose of the first robot when collecting the first image. The image inheritance second module 730 is used to locate the second robot based on the first target image when it receives a pose consistency message from the second robot, so as to update the environmental image library or perform navigation for the second robot. The pose consistency message is generated by the second robot when it determines that its pose at the designated point in the team is consistent with the first pose.

[0142] Optionally, the second determining module 710 is specifically used for:

[0143] Based on a loop closure detection algorithm, it is determined, according to the first image, whether there is an environment image matching the first image in the environment image library; if there is an environment image matching the first image in the environment image library, loop closure detection is successful, and the matched environment image is determined as a first target image of the first image.

[0144] Optionally, the state parameter further includes a first state parameter, and the first state parameter is used to describe a position parameter of a sensor started by the first robot when the first robot collects the first image at the formation designated point; the first state parameter is used to control the second robot to start the sensor at a corresponding position in the first state parameter, so as to collect images based on the started sensors.

[0145] The robot positioning device provided in the embodiments of the present disclosure can perform the robot positioning method provided in the embodiments of the present disclosure Figure 5 The robot positioning method provided in the corresponding embodiments has the corresponding function modules and beneficial effects of the execution method.

[0146] Figure 8 The structure schematic diagram of the scheduling device provided in an embodiment of the present disclosure is shown in the figure, which includes a memory 810, a processor 820 and a computer program. Figure 8

[0147] The computer program is stored in the memory 810 and is configured to be executed by the processor 820 to implement the robot positioning method provided in any one of the embodiments of the present disclosure Figure 2 、 Figure 4 and Figure 5 .

[0148] The memory 810 and the processor 820 are connected through a bus 830.

[0149] The related description can be understood by referring to the related description and effects corresponding to the steps of Figure 2 、 Figure 4 and Figure 5 , which will not be described in detail here.

[0150] The present disclosure provides a warehouse system, which comprises: Figure 8 The scheduling device and the robot provided in the corresponding embodiments.

[0151] In some embodiments, the warehouse system further comprises a storage rack for storing goods or containers.

[0152] In some embodiments, the warehouse system further comprises an operation table and transfer devices such as unloading machines, elevators and transportation lines. The robot is used to carry the goods or containers to the operation table or the transfer devices connected with the operation table, so as to complete the orders corresponding to the operation table.​

[0153] One embodiment of the present disclosure provides a computer readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement any one of the embodiments of the present disclosure Figure 2 、 Figure 4 and Figure 5 the robot positioning method provided in any one of the embodiments corresponding to the robot positioning method provided in any one of the embodiments.

[0154] The computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0155] The present disclosure also provides a program product, which includes an executable computer program stored in a readable storage medium. At least one processor of a scheduling device or a warehousing system can read the computer program from the readable storage medium, and the at least one processor executes the computer program to enable a robot positioning device to implement the robot positioning method provided in the various embodiments.

[0156] In several embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other manners. For example, the above-described device embodiments are merely illustrative, and the division of the modules is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling or direct coupling or communication connection between modules can be indirect coupling or communication connection through some interface, device or module, and can be electrical, mechanical or in other forms.

[0157] The modules illustrated as separate components can or can not be physically separate, and the components illustrated as modules can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment.

[0158] In addition, each functional module in each embodiment of the present disclosure can be integrated in one processing unit, or each module can be physically present separately, or two or more modules can be integrated in one unit. The unit formed by the above modules can be realized in the form of hardware, or in the form of hardware plus software function unit.

[0159] The integrated modules implemented in the form of software function modules can be stored in a computer readable storage medium. The software function modules are stored in a storage medium and include a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute part of the steps of the methods described in various embodiments of the present disclosure.

[0160] It should be understood that the processor described above can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in combination with the present disclosure can be directly embodied as hardware processor execution, or a combination of hardware and software modules in the processor.

[0161] The memory can include a high-speed RAM memory, and can also include a non-volatile storage NVM, such as at least one disk memory, and can also be a U disk, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0162] The bus can be an industry standard architecture (ISA) bus, a peripheral component (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, the bus in the drawings of the present disclosure does not limit to only one bus or one type of bus.

[0163] The storage medium described above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0164] An example storage medium is coupled to the processor such that the processor can read information from, and can write information to, the storage medium. Of course, the storage medium can be a part of the processor. Consistent with the teachings provided herein, the processor and the storage medium can be located in a single ASIC or several ASICs. Alternatively, the processor and the storage medium can be located in different devices or components of a computing device or electronic system.

[0165] Those skilled in the art can understand that all or part of the steps of the foregoing method embodiments can be completed by relevant hardware instructed by programs. The foregoing programs can be stored in a computer readable storage medium. When the programs are executed, the steps of the foregoing method embodiments are executed; and the foregoing storage medium includes various media capable of storing program codes, such as ROM, RAM, magnetic disks, or optical disks.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than limit the same; although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not cause the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A robot localization method, characterized in that, The method is applied to a scheduling device, and the method includes: Based on the first image collected by the first robot at the designated location of the team, a first target image matching the first image is determined from the environmental image database; Acquire a second image captured by the second robot at the designated location of the squad; Based on the first image and the second image, determine whether the first pose of the second robot at the squad fixed point is consistent with the second pose of the first robot at the squad fixed point; If the first pose is consistent with the second pose, the second robot is localized based on the first target image to update the environmental image library or to navigate the second robot. The first robot and the second robot are located in the first convoy, and the first robot is the lead robot of the first convoy. At the designated location of the convoy, the first convoy splits into the second convoy and the third convoy. The lead robot of the second convoy is the first robot, and the lead robot of the third convoy is the second robot. The second convoy and the third convoy each include at least one robot.

2. The method according to claim 1, characterized in that, The step of determining a first target image matching the first image from an environmental image database based on the first image captured by the first robot at a fixed point in the team includes: Based on the closed-loop detection algorithm, it is determined whether there is an environmental image in the environmental image library that matches the first image, according to the first image; If so, the loop closure detection is successful, and the matched environmental image is determined as the first target image of the first image.

3. The method according to claim 1, characterized in that, If the first pose and the second pose are inconsistent, the method further includes: Adjust the pose of the second robot based on the first pose and the second pose; During the pose adjustment of the second robot, at least one frame of the third image acquired by the second robot is obtained; Determine whether there exists at least one frame of the third image where the third pose of the second robot is consistent with the first pose; If it exists, the second robot is located based on the first target image to update the environmental image library or to navigate the second robot.

4. The method according to claim 1, characterized in that, If the first pose and the second pose are inconsistent, the method further includes: Based on the second image, a second target image matching the second image is determined from the environmental image library; The second robot is located based on the second target image.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: When the platoon is positioned, the sensors activated by the second robot are determined according to the role of the second robot in the third platoon. The roles of the robot in the platoon include the head robot, the tail robot, and the middle robot.

6. A robot localization method, characterized in that, The method is applied to a scheduling device, and the method includes: Based on the first image collected by the first robot at the designated location of the team, a first target image matching the first image is determined from the environmental image database; The first robot acquires the state parameters of the first image it captures, and sends the first image and the state parameters to the second robot to control the second robot to adjust its state based on the state parameters; wherein, the state parameters include the first pose of the first robot when it acquires the first image; When the pose consistency message is received from the second robot, the second robot is located based on the first target image to update the environmental image library or to navigate the second robot. The pose consistency message is generated when the second robot determines that the pose of the second robot at the squad fixed point is consistent with the first pose.

7. The method according to claim 6, characterized in that, The state parameters also include a first state parameter, which describes the position parameters of the sensors activated by the first robot when the team collects the first image at a fixed point; the first state parameter is used to control the second robot to activate the sensors at the corresponding positions in the first state parameter, so as to collect images based on the activated sensors.

8. A robot positioning device, characterized in that, The device is used in a scheduling device, and the device includes: The first determining module is used to determine a first target image that matches the first image from an environmental image library based on the first image collected by the first robot at a fixed point in the team. The image acquisition module is used to acquire a second image collected by the second robot at the designated location of the squad. The pose determination module is used to determine, based on the first image and the second image, whether the first pose of the second robot at the squad fixed point is consistent with the second pose of the first robot at the squad fixed point; The image inheritance first module is used to locate the second robot based on the first target image if the first pose is consistent with the second pose, so as to update the environmental image library or to navigate the second robot. The first robot and the second robot are located in the first convoy, and the first robot is the lead robot of the first convoy. At the designated location of the convoy, the first convoy splits into the second convoy and the third convoy. The lead robot of the second convoy is the first robot, and the lead robot of the third convoy is the second robot. The second convoy and the third convoy each include at least one robot.

9. A robot positioning device, characterized in that, The device is used in a scheduling device, and the device includes: The second determining module is used to determine a first target image that matches the first image from an environmental image library based on the first image collected by the first robot at a fixed point in the team. A status parameter sending module is used to acquire status parameters of the first robot acquiring the first image, and send the first image and the status parameters to the second robot to control the second robot to adjust the state of the second robot based on the status parameters; wherein, the status parameters include the first pose of the first robot when acquiring the first image; The image inheritance second module is used to locate the second robot based on the first target image when a pose consistency message is received from the second robot, so as to update the environmental image library or to navigate the second robot. The pose consistency message is generated by the second robot when it determines that the pose of the second robot at the designated point of the squad is consistent with the first pose.

10. A scheduling device, characterized in that, include: Memory and at least one processor; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the robot localization method as described in any one of claims 1-7.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by the processor, implement the robot localization method as described in any one of claims 1-7.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the robot localization method as described in any one of claims 1-7.

Images