Path planning method and device, processing equipment, chip and storage medium

Abstract

The application discloses a path planning method and device, a processing device, a chip and a computer readable storage medium. The method comprises the following steps: obtaining a first local map and a second local map, wherein the first local map is a map obtained by a first robot in a first region, the second local map is a map obtained by a second robot in a second region, the first region and the second region are both part of a to-be-tested region, and the to-be-tested region is a whole region of path planning; obtaining a relative pose relationship between the first robot and the second robot; fusing the first local map and the second local map according to the relative pose relationship to obtain a first global map; obtaining a boundary point of the first global map, and constructing a first boundary point set according to the boundary point of the first global map; and planning a path of the first robot and / or the second robot according to the first boundary point set.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of environmental perception, and in particular to a path planning method and device, a processing device, a chip and a storage medium. BACKGROUND

[0002] Exploration of an unknown environment is a prerequisite for path planning. The exploration of an unknown environment can be implemented by a single robot or by multiple robots. Among them, the exploration of an unknown environment by multiple robots has more advantages. At present, the exploration of an unknown environment by multiple robots is mostly implemented under the condition that the initial poses of the multiple robots are known. When the initial poses of the multiple robots are unknown, how to implement the exploration process of the unknown environment by the multiple robots still needs to be improved. SUMMARY

[0003] Embodiments of the present application provide a path planning method, device, processing device, chip and computer readable storage medium.

[0004] Embodiments of the present application provide a path planning method, which comprises:

[0005] obtaining a first local map and a second local map; wherein the first local map is a map obtained by a first robot in a first region, and the second local map is a map obtained by a second robot in a second region; the first region and the second region are both part of a to-be-tested region, and the to-be-tested region is the entire region of path planning;

[0006] obtaining a relative pose relationship between the first robot and the second robot;

[0007] fusing the first local map and the second local map according to the relative pose relationship to obtain a first global map;

[0008] obtaining a boundary point of the first global map, and constructing a first boundary point set according to the boundary point of the first global map;

[0009] planning a path of the first robot and / or the second robot according to the first boundary point set.

[0010] Embodiments of the present application provide a path planning device, which comprises:

[0011] a first obtaining unit configured to obtain a first local map and a second local map; wherein the first local map is a map obtained by a first robot in a first region, and the second local map is a map obtained by a second robot in a second region; the first region and the second region are both part of a to-be-tested region, and the to-be-tested region is the entire region of path planning;

[0012] The first acquisition unit is further configured to obtain a relative pose relationship between the first robot and the second robot.

[0013] The first processing unit is configured to fuse the first local map and the second local map according to the relative pose relationship to obtain a first global map.

[0014] The first acquisition unit is further configured to obtain boundary points of the first global map, and construct a first boundary point set according to the boundary points of the first global map.

[0015] The first processing unit is further configured to plan a path of the first robot and / or the second robot according to the first boundary point set.

[0016] The processing device provided by the embodiments of the present application comprises a processor and a memory, the memory is configured to store a computer program, and the processor is configured to call and run the computer program stored in the memory to execute the path planning method.

[0017] The chip provided by the embodiments of the present application comprises a processor, which is configured to call and run a computer program from a memory, so that the device installed with the chip executes the path planning method.

[0018] The computer readable storage medium provided by the embodiments of the present application is configured to store a computer program, and the computer program makes a computer execute the path planning method.

[0019] The technical solution described in this application involves obtaining a first local map and a second local map. The first local map is a map obtained by a first robot within a first region, and the second local map is a map obtained by a second robot within a second region. Both the first and second regions are parts of a region to be measured, which is the entire region for path planning. The relative pose relationship between the first robot and the second robot is obtained. The first and second local maps are fused based on this relative pose relationship to obtain a first global map. The boundary points of the first global map are obtained, and a first set of boundary points is constructed based on these boundary points. The paths of the first robot and / or the second robot are planned based on the first set of boundary points. Thus, by using the map information collected by each robot within the first robot set as the basis for estimating the relative pose relationship of each robot, the relative pose transformation relationship of multiple robots is obtained even when the initial pose relationship of each robot is unknown, thereby improving the accuracy of the relative pose relationship estimation. Furthermore, by using the maps collected by each robot in the first robot ensemble and the relative pose relationships between the robots in the first robot ensemble, the local map is fused according to the relative pose relationships between the robots in the first robot ensemble to obtain the first global map. The boundary points of the first global map are obtained, and by filtering the boundary points of the first global map, a first boundary point set is constructed. The paths of each robot in the first robot ensemble are planned according to the first boundary point set, which simplifies the target points and improves the exploration efficiency. Attached Figure Description

[0020] The accompanying drawings, which are provided to further illustrate this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application.

[0021] Figure 1 This is a flowchart illustrating the path planning method provided in the embodiments of this application. Figure 1 ;

[0022] Figure 2(a) is a flowchart illustrating the method for obtaining the relative pose relationship between the first robot and the second robot provided in an embodiment of this application.

[0023] Figure 2(b) is a flowchart illustrating the method for obtaining image tags provided in an embodiment of this application;

[0024] Figure 2(c) is a flowchart illustrating the method for obtaining a set of target images that match the second image according to an embodiment of this application;

[0025] Figure 2(d) is a flowchart illustrating the method for determining the relative pose relationship between the first robot and the second robot based on the second image and the target image set provided in this application embodiment.

[0026] FIG. 2(e) is a flow diagram of a method for constructing a first boundary point set according to boundary points of a first global map according to an embodiment of the present application;

[0027] Figure 3 FIG. 2(e) is a flow diagram of a method for constructing a first boundary point set according to boundary points of a first global map according to an embodiment of the present application;

[0028] Figure 4 FIG. 2(e) is a flow diagram of a method for constructing a first boundary point set according to boundary points of a first global map according to an embodiment of the present application;

[0029] Figure 5 FIG. 2(e) is a flow diagram of a method for constructing a first boundary point set according to boundary points of a first global map according to an embodiment of the present application;

[0030] Figure 6 FIG. 2(e) is a flow diagram of a method for constructing a first boundary point set according to boundary points of a first global map according to an embodiment of the present application; DETAILED DESCRIPTION

[0031] In order to enable a person skilled in the art to more fully understand the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings, which are only used for reference and are not intended to limit the embodiments of the present application.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terminology used in the present application is for the purpose of describing the embodiments of the present application only and is not intended to limit the present application.

[0033] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0034] It should also be noted that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific order of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0035] The term "and / or" in the embodiments of the present application is only used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, object A and / or object B, which can represent the existence of object A alone, the existence of object A and object B, and the existence of object B alone.

[0036] In addition, in the embodiments of the present application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0037] Currently, the exploration of unknown environment is roughly divided into two kinds for single robot or multi-robot. One method uses the grid map constructed by laser radar, uses the free area (unoccupied grid) in the detected area and the junction of the undetected area (called frontier) as the target point for the next detection, and constantly detects the frontier point to detect the undetected area, and finally iterates the exploration task of the entire unknown environment. Another method is an information entropy-based method, which considers the uncertainty of the robot's own sensor and motion control, measures the path cost and map entropy increase, and determines the next control method of the robot. This method can be applied to laser radar sensor mapping and visual active exploration process using cameras. However, this method is more complex, and compared with it, the active exploration algorithm based on frontier point has lower complexity and is easier to apply.

[0038] In indoor environment, the position accuracy cannot meet the needs of robot navigation, such as the accuracy of global positioning system (GPS), so the robot needs to complete environment perception and establish map information through methods such as simultaneous localization and mapping (SLAM). However, the general SLAM method uses passive motion, that is, the human controls the motion of the robot to complete mapping. Compared with it, active exploration gives the robot autonomy, frees it from human operation, and can improve exploration efficiency through multiple robots. The current multi-robot exploration method based on frontier points is applicable to the case where the initial pose is known, that is, the multi-robot is mutually observable at initialization, that is, the relative pose can be estimated. During the process of multi-robot autonomous exploration of the environment, a large number of frontier points are clustered, and then the clusters of each frontier point are assigned to each robot for the next exploration according to the geometric distance.

[0039] In one scheme, the environment data collected by the sensor is acquired, the SLAM algorithm is used to identify the area to be visited on the currently estimated SLAM initial environment map; for the identified visited area, path planning is performed based on the active exploration method, and the exploration path is selected from the planned path according to the highest utility principle, and the path exploration operation is performed; according to the path exploration result, the corresponding environment map of autonomous exploration is established.

[0040] In another solution, user 1 and user 2 respectively take pictures of the indoor environment in front of them; user 1 constitutes his own database with the images, and user 2 shares the images with user 1; the database identifies the semantic information contained in each image in user 1's image database by using a region-based fully convolutional neural network (R-FCN), and converts the semantic information into a corresponding semantic sequence to constitute a semantic database; user 1 receives the image of user 2, converts the image into a semantic sequence by using the R-FCN, matches the semantic sequence with the semantic database, and if the same is not searched, continues to walk along the current direction, and if the same semantic sequence is searched, selects the most representative semantic target to establish a position connection; at this time, a coordinate system is established with the target as the center, and interactive positioning between users in an unknown environment is realized.

[0041] The foregoing solutions are all realized on the premise that the initial pose is known, and the problem of active exploration between multiple robots in an unknown environment has not been solved, and the problem of mutual positioning in the case of unknown initial pose has not been solved, and the exploration method based on boundary points does not consider the influence of adjacent boundary points in the process of dynamic exploration of the front boundary, that is, the exploration of boundary point A may cover the exploration range of boundary point B, resulting in a decrease in the utility of the boundary points for environmental exploration.

[0042] In order to solve the problem of active exploration and mutual positioning between multiple robots in the case of unknown initial pose, and further improve the exploration efficiency, the following technical solutions of the embodiments of the present application are provided.

[0043] Figure 1 is a flowchart of a path planning method provided by the embodiments of the present application Figure 1 As shown in Figure 1 , the method comprises the following steps:

[0044] Step 101: obtaining a first local map and a second local map.

[0045] The first local map is a local map obtained by a first robot in a first region, and the second local map is a local map obtained by a second robot in a second region, the first region and the second region are both part of a to-be-measured region, and the to-be-measured region is the entire region of path planning. The first region and the second region are respectively the initial exploration regions of the first robot and the second robot, and as the first robot and the second robot continue to explore, the regions explored by them will overlap to a certain extent.

[0046] Here, the first robot and the second robot are any two robots in the first robot set, and the first robot set includes at least two robots. The first robot set can be understood as a set of robots performing an exploration process. The exploration process here refers to an exploration process of an unknown area.

[0047] In some embodiments, the first robot set only includes the first robot and the second robot. In other embodiments, the first robot set includes one or more robots in addition to the first robot and the second robot.

[0048] In some embodiments, before step 101, the following step is performed: constructing a relative pose association graph between the robots in the first robot set, which describes the relative pose relationship between the robots. Here, before the relative pose relationship between the robots in the first robot set is obtained, the constructed relative pose association graph does not describe the relative pose relationship between the robots. After the relative pose relationship between any two robots is obtained, the relative pose association graph is updated according to the obtained relative pose relationship between the two robots, and the updated relative pose association graph describes the relative pose relationship between the two robots.

[0049] In addition, before the relative pose relationship between the robots in the first robot set is obtained, that is, before the relative pose association graph does not describe the relative pose relationship between the robots in the first robot set, the first robot set can be divided into multiple sets, where different robots in the first robot set are divided into different sets, and different sets maintain different local maps, which are part of a global map. The global map here refers to a map corresponding to the area to be explored by all robots in the first robot set.

[0050] In some embodiments, the robots in the first robot set use a visual SLAM module and a single-line laser radar to obtain a local map, use a single-line laser radar to establish a grid map to guide robot exploration, and use a visual SLAM module combined with semantic information to establish a more abundant map to realize mutual positioning of multiple robots. The local map obtained by the robots in the first robot set includes the grid map established by the single-line laser radar and the map established by the visual SLAM module.

[0051] In some embodiments, the first robot obtains the first local map through a visual SLAM module and a single-line laser radar, and the second robot obtains the second local map through the visual SLAM module and the single-line laser radar, wherein the first local map is a first local map obtained by the first robot in a first region, the second local map is a second local map obtained by the second robot in a second region, the first region and the second region are both part of a to-be-measured region, and the to-be-measured region is an entire region to be explored by multiple robots. The first robot and the second robot can also obtain the first local map and the second local map through other manners, wherein the first local map and the second local map can also include maps of other forms, which are not limited in the present application.

[0052] It should be noted that although the technical solutions of the embodiments of the present application are described by obtaining the local maps of the first robot and the second robot, any robot in the first robot set is similar, so as to obtain the local maps of each robot in the first robot set.

[0053] Step 102: obtaining the relative pose relationship between the first robot and the second robot.

[0054] It should be noted that although the technical solutions of the embodiments of the present application are described by obtaining the relative pose relationship between the first robot and the second robot, any robot in the first robot set is similar, so as to obtain the relative pose relationship between each robot in the first robot set.

[0055] In some embodiments, referring to FIG. 2(a), the relative pose relationship between the first robot and the second robot can be obtained through the following process:

[0056] Step 1021: obtaining a first image collected by the first robot.

[0057] In some embodiments, the first robot obtains the first image through visual SLAM. However, the first robot can also obtain the first image through other manners, which are not limited in the present application.

[0058] In some embodiments, after obtaining the first image, the first robot further obtains a label of the first image, wherein the label represents a feature of the first image, and the label of the first image is determined according to semantic information of the first image.

[0059] It should be noted that although the technical solutions of the embodiments of the present application are described by obtaining the label collected by the first robot, any robot in the first robot set is similar, so as to obtain the label of the image collected by each robot in the first robot set.

[0060] In some embodiments, referring to FIG. 2(b), the label of the image can be obtained by the following process:

[0061] Step 1021a: obtaining semantic information of the first image, and determining the label of the first image according to the semantic information of the first image.

[0062] Step 1021b: obtaining semantic information of the second image, and determining the label of the second image according to the semantic information of the second image.

[0063] The label of the first image and the label of the second image are used to determine the second image matched with the first image.

[0064] In some embodiments, the first robot obtains the first image, obtains the semantic information of the first image through the semantic module, and determines the label of the first image according to the semantic information of the first image; the second robot obtains the second image, obtains the semantic information of the second image through the semantic module, and determines the label of the second image according to the semantic information of the second image; wherein the label of the first image and the label of the second image are used to determine the second image matched with the first image. For example, the first image contains a pot of flowers and two chairs, the category of the semantic information is flower and chair, and the corresponding number is 1 and 2, so the category and number of the semantic information are taken as the label of the first image, and the representation form of the label can be set according to the actual situation, which is not limited in the present application.

[0065] It should be noted that although the technical solution of the embodiments of the present application is described by obtaining the labels of the first image and the second image, it is not limited thereto, and obtaining the labels of the images collected by any robot in the first robot set is similar, so as to obtain the labels of the images of each robot in the first robot set.

[0066] In some embodiments, after obtaining the first image collected by the first robot and obtaining the label of the first image, the following steps are further included:

[0067] Storing the label of the first image, and retrieving the first image by using the label of the first image.

[0068] The label of the collected first image is stored in a database, which can be a database of the first robot that acquires the first image, a database of the first robot set, or a database of the set to which the first robot belongs, and the application does not make specific limitations. The first image collected by the first robot and the label corresponding to the first image are stored in the corresponding database in the form of a key-value pair, or stored according to the actual situation, and the application does not make specific limitations. The database includes robot number, collected image, label corresponding to the image, collection time, and position at the time of collecting the image, and can store the required related information according to the actual situation, and the application does not make specific limitations. At the same time, the first image can be retrieved from the database by using the label of the first image.

[0069] It should be noted that although the technical solutions of the embodiments of the application are described based on the label of the first image, the label of any image collected by the robot in the first robot set is obtained, and the label of the image collected by the robot in the first robot set is stored. The label of the image collected by the robot in the first robot set can also be retrieved from the database according to the label.

[0070] Step 1022: determining a second image matched with the first image, the second image being collected by the second robot.

[0071] The label of the first image is matched with the label in the database of the image collected by the second robot, and according to the matching result, the second image matched with the first image is determined, and the second image is collected by the second robot. That is, the label of the first image is the same as the label of the second image.

[0072] It should be noted that although the technical solutions of the embodiments of the application are described based on the matching of the label of the first image and the label of the second image, the matching process of the label of the image collected by any other robot in the first robot set is similar, so that the matching result of the label of the image collected by each robot in the first robot set and the label of the image collected by the remaining robot can be obtained.

[0073] Step 1023: determining a plurality of images matched with the second image, the plurality of images being collected by the first robot, and the plurality of images forming a target image set.

[0074] In some embodiments, in order to ensure that the relative pose relationship between the first robot and the second robot can be accurately acquired, more images collected by the first robot matched with the second image need to be acquired, therefore, a target image set matched with the second image is acquired, and the target image set is collected by the first robot.

[0075] In some embodiments, referring to FIG. 2(c), the method of obtaining the target image set matched with the second image can be achieved by the following process:

[0076] Step 1023a: Obtain an initial image set; the initial image set is an image set obtained by the first robot at the position where the first image is captured.

[0077] The initial image set includes the first image.

[0078] If there is a label of the second image matched with the label of the first image, the initial image set obtained by the first robot at the position where the first image is captured includes the first image. In some embodiments, the images are captured in a multi-view and multi-perspective manner in the vicinity of the position where the first image is captured, and the obtained initial image set is multi-perspective and multi-view. The purpose of multi-perspective and multi-view is to capture enough images to improve the accuracy of relative pose estimation. In some embodiments, under the support of laser radar data, the first robot is controlled to move and rotate randomly within a certain range to realize image capture in a larger range without collision with the environment. The certain range can be determined according to actual conditions, which is not limited in the present application. In some embodiments, a square range with a side length of 0.3 meters centered on the current position can be selected. In addition, the first robot moves and rotates randomly within the range, and the specific moving route and rotating angle can be set according to actual conditions, which is not limited in the present application.

[0079] Step 1023b: Compare the label of the second image with the labels of the images in the initial image set, and determine an intermediate image set according to the comparison result.

[0080] The labels of the images in the initial image set are matched with the label of the second image by using the foregoing matching method, the images with matching results are retained, and the images without matching are deleted.

[0081] In some embodiments, the images corresponding to the labels consistent with the label of the second image in the initial image set are retained, and the images corresponding to the labels inconsistent with the label of the second image are deleted to obtain the intermediate image set, which is used as the basis for confirming the relative pose between robots.

[0082] Step 1023c: Obtain the matching result of the second image and the images in the intermediate image set, and determine a target image set according to the matching result.

[0083] In some embodiments, the information of the image is represented by the feature points and the descriptors, and the target image set is obtained by matching the information. Specifically, the feature points and the descriptors of the second image are matched with the feature points and the descriptors of each image in the intermediate image set to obtain the target image set. The feature points and the descriptors used can be set according to actual conditions, which are not limited in the present application. In some embodiments, the ORB (Oriented FAST and Rotated BRIEF) feature points and the BRIEF (Binary Robust Independent Elementary Features) descriptors are selected, and the ORB feature points and the BRIEF descriptors of the second image are matched with the ORB feature points and the BRIEF descriptors of the images in the intermediate image set. The proportion of the number of matched descriptors to the descriptors in the second image is calculated, and the proportion value is used as a measurement index of the similarity. If the proportion value exceeds a first threshold value, the images in the intermediate image set that meet the condition are used to construct the target image set. The first threshold value can be set according to actual conditions, which is not limited in the present application. In some embodiments, the first threshold value can be set to 0.7, and the images in the intermediate image set with a similarity greater than 0.7 are selected to construct the target image set.

[0084] It should be noted that the technical solutions of the embodiments of the present application are described by collecting the target image set, but are not limited thereto. The process of collecting and matching the image set matched with the image by the remaining robots in the first robot set is similar, so that the image set of each robot in the first robot set can be obtained.

[0085] Step 1024: determining the relative pose relationship between the first robot and the second robot according to the second image and the target image set.

[0086] In some embodiments, referring to FIG. 2(d), the relative pose relationship between the first robot and the second robot can be determined according to the second image and the target image set by the following process:

[0087] Step 1024a: obtaining the first list and the second list.

[0088] The first list includes the second image, the feature information of the second image, and the position information of the second robot collecting the second image, and the second list includes the target image set, the feature information of the target image set, and the position information of the first robot collecting the target image set.

[0089] Step 1024b: obtaining the relative pose transformation according to the matching result of the first list and the second list.

[0090] Step 1024c: determining the relative pose relationship between the first robot and the second robot according to the relative pose transformation.

[0091] In some embodiments, the second image and the descriptors in the target image set that match each other are established into a set, the second image, the descriptors in the second image, the corresponding map points and the position of the second robot collecting the second image are stored in a first list, the target image set, the descriptors in the target image set, the corresponding map points and the position of the first robot collecting the target image set are stored in a second list, and the information in the two lists is matched. In some embodiments, a random sample consensus (RANSAC) algorithm can be selected for matching, and the iteration is continuously performed. Other methods can also be selected for matching according to actual conditions, which are not limited in the present application. The relative pose transformation of the first robot collecting the target image set and the second robot collecting the second image is obtained through the matching result, and the relative pose relationship between the first robot and the second robot at this moment is estimated by using the relative pose of the first robot at this moment and the relative pose of the second robot at this moment.

[0092] Step 103: fusing the first local map and the second local map according to the relative pose relationship to obtain a first global map.

[0093] The first local map and the second local map are fused according to the relative pose transformation relationship between the first robot and the second robot to obtain a first global map, wherein the first global map is also a local map of the to-be-detected region. Until the entire multi-robot exploration process is completed and the relative pose relationship between all robots is determined, the local map collected by each robot is fused according to the relative pose relationship to obtain a global map of the entire to-be-detected region.

[0094] In some embodiments, the first local map and the second local map can be fused according to the relative pose relationship to obtain the first global map, and after the first global map is obtained, the following steps are further included:

[0095] The relative pose relationship is updated according to the relative pose relationship, and the first robot and the second robot are merged into a first set.

[0096] The relative pose relationship is updated according to the relative pose relationship, and the first robot and the second robot are merged into a first set.

[0097] The relative pose relationship between the first robot and the second robot is unknown before the autonomous exploration process starts, and the first robot and the second robot belong to different sets. When the relative pose relationship between the first robot and the second robot is determined, the sets to which the first robot and the second robot belong are merged to obtain a first set. At this time, the first global map is the map of the first robot and the second robot in the first set. As described above, if each set or each robot has a corresponding database, the databases also need to be merged. The data stored in the database is as described above, and will not be repeated here. As the autonomous exploration process of the multiple robots continues, when the relative pose relationship of the robots in the two sets is determined, the two sets continue to merge until all the robots are merged into one set. The robots in the subsequent set can be one or more, which is determined according to the actual situation, and the present application does not make specific limitations

[0098] Step 104: Obtain the boundary points of the first global map, and construct a first boundary point set according to the boundary points of the first global map.

[0099] At the beginning of each planning period, all boundary points are found according to the information of the global map in the current set, and the unreachable boundary points are removed to obtain the boundary points of the global map. The above-mentioned global map refers to the map in the set. The robots in the set can be one or more, which is determined according to the actual situation, and the present application does not make specific limitations. The planning period is the beginning of the exploration period, and the specific duration can be set according to the actual situation. In some embodiments, the exploration period can be set to 10 seconds. The exploration period is the time for the robot to complete the exploration task, and the specific duration is set according to the actual situation, and the present application does not make specific limitations. The boundary point is the detected grid in the grid map that is adjacent to the undetected grid and is not occupied. Unreachable means that the path planning cannot reach, and any path planning algorithm can be used to determine whether it is reachable, and the present application does not make specific limitations.

[0100] In some embodiments, at the beginning of the planning period, the boundary points are found according to the information of the first global map, the unreachable boundary points are removed, and the boundary points of the first global map are obtained and constructed into a set S F In some embodiments, each grid of the grid map is represented by a number. The unknown occupancy of the grid is -1, the idle is 0, and the occupancy is 1. The boundary point is the grid adjacent to the grid with -1, and the grid itself is 0. Whether it is reachable or not can be confirmed by any path planning algorithm according to the current grid map. If a path from the current pose point to the target point can be planned, it is reachable. The specific data representation of the grid map can be set according to the actual situation, and the present application does not make specific limitations.

[0101] In some embodiments, referring to FIG. 2(e), the first set of boundary points can be constructed according to the boundary points of the first global map by the following process:

[0102] Step 1041: dividing the boundary points of the first global map into a plurality of sets;

[0103] In some embodiments, S F K-means clustering is performed, and the clustering is into N sets, where N is the number of robots in the first set. S F may also be grouped according to actual conditions, which are not specifically limited in the present application.

[0104] In some embodiments, the robots in the first set are matched with the N sets. The centroids of the respective clusters are calculated, and the positions of the respective sets after clustering are replaced by the positions of the centroids. The distances of the respective robots in the first set to the centroids of the respective clusters are calculated. Through the matching of the robots and the clusters, the sum of the distances of all the robots to the centroids is minimized.

[0105] Step 1042: obtaining boundary point pairs in the plurality of sets.

[0106] In the boundary point pairs, the distance between the two boundary points is less than a second threshold value, and the points on the line connecting the two boundary points are all unoccupied.

[0107] In some embodiments, the grids in the respective clusters are screened, and the distance between two grids is calculated exhaustively. If the distance between the centers of the two grids is less than the second threshold value and the grids directly connected by the line between the two grids are all unoccupied, the coordinates of the two grids are stored, representing that the two grids do not need to be accessed entirely. Accessing one of the grids can simultaneously cover the detection of the other grid, reducing the remaining exploration range of the radar. The second threshold value can be set according to actual conditions, which are not specifically limited in the present application. In some embodiments, assuming that the radius of a single-line laser radar is R, the second threshold value can be set as Then the remaining exploration length of the radar is at least

[0108] Step 1043: screening the other boundary point of the boundary point whose occurrence frequency in the boundary point pairs is greater than or equal to a third threshold value, to obtain a first part of boundary points.

[0109] In some embodiments, the number of all stored boundary points in the cluster is counted, and boundary points with a frequency greater than or equal to a third threshold value are sequentially retained, and the third threshold value can be set according to actual conditions, which is not specifically limited in the present application. These boundary points will be the target of subsequent robot access. By accessing these boundary points, the most information can be obtained in the same time. The boundary points corresponding to the boundary point pair with a frequency greater than the third threshold value are deleted, that is, when accessing the boundary point with a high frequency, the corresponding boundary point can be covered. According to the frequency, the boundary points in the corresponding boundary point pair are sequentially deleted until all grid pairs in the cluster are simplified, and the remaining boundary points in the boundary point pair are the first part of boundary points.

[0110] Step 1044: constructing a first boundary point set according to the first part of boundary points and the second part of boundary points.

[0111] Among them, the second part of boundary points is the boundary points in the multiple sets except the boundary point pair.

[0112] In the cluster, the boundary points except the boundary point pair are the second part of boundary points, the first part of boundary points and the second part of boundary points are constructed into a first boundary point set, which is used for subsequent path planning of the robot set.

[0113] Step 105: planning the path of the first robot and / or the second robot according to the first boundary point set.

[0114] According to the boundary points in the robot set, the exploration of the robots in the set is planned, and the planning method can be selected according to actual conditions, which is not specifically limited in the present application. In addition, the robots in the set can be one or more, which is determined according to actual conditions. In some embodiments, the first boundary point set is the boundary points in the first global map, and the boundary points in the first boundary point set are assigned to the first robot and / or the second robot, so that the first robot and / or the second robot perform the exploration process.

[0115] The technical scheme of the embodiment of the application first acquires a first local map and a second local map, wherein the first local map is a map obtained by a first robot in a first region, the second local map is a map obtained by a second robot in a second region, the first region and the second region are both part of a to-be-detected region, and the to-be-detected region is the whole region of path planning; secondly, a relative pose relationship between the first robot and the second robot is obtained according to information of the first local map and the second local map, the relative pose relationship between the multiple robots is realized in the case that initial poses of the multiple robots are unknown, mutual positioning of the multiple robots is realized, the first local map and the second local map are fused according to the relative pose relationship between the first robot and the second robot, a first global map is obtained, meanwhile, a set in which the first robot and the second robot are located is merged, that is, maps of all robots in the set are fused into one map, then, a boundary point of the first global map is obtained, the boundary point of the first global map is screened, a first boundary point set is constructed, in this way, simplification of the boundary point is realized, and a path of the first robot and / or the second robot is planned according to the first boundary point set. In this way, mutual positioning of the multiple robots can be realized in the case that initial poses of the multiple robots are unknown, and the boundary point of the obtained map is simplified, the to-be-explored target point is simplified on the basis of not losing the obtained exploration environment information, and exploration efficiency is improved.

[0116] Based on the foregoing embodiment, Figure 3 A flowchart of a second schematic diagram of a whole framework of a path planning method provided by the embodiment of the application is shown in Figure 3 The method comprises the following steps:

[0117] Step 301: A third map is acquired by using a visual SLAM module.

[0118] In some embodiments, the visual SLAM uses a centralized collaborative monocular SLAM (CCM-SLAM) algorithm to obtain the third map, and realizes multi-robot visual collaborative mapping.

[0119] Step 302: An image semantic information is recognized by a semantic recognition module.

[0120] In some embodiments, the semantic recognition module uses a YOLO (You Only Look Once) network to acquire the semantic information of the image.

[0121] Step 303: Mutual positioning of multiple robots is realized.

[0122] The mutual positioning of the multiple robots is realized by using the visual SLAM module and the semantic recognition mode, and the method is as described above, which will not be repeated here.

[0123] Step 304: acquiring a fourth map by using the single-line laser radar.

[0124] In some embodiments, the single-line laser radar acquires the fourth map by using the cartographer framework.

[0125] Step 305: acquiring boundary points of the fourth map.

[0126] The method of acquiring the boundary points of the fourth map is as described above, which is not repeated here.

[0127] Step 306: implementing multi-robot task allocation.

[0128] According to the multi-robot mutual positioning and the boundary points acquired in step 305, the task allocation of the multi-robot is completed.

[0129] In the technical scheme of the embodiments of the present application, the third map is acquired by using the visual SLAM module, the image semantic information is recognized by using the semantic recognition module, the multi-robot mutual positioning is realized by using the visual SLAM module and the semantic recognition module, the fourth map is acquired by using the single-line laser radar, the boundary points of the fourth map are acquired, and the multi-robot task allocation is realized according to the multi-robot mutual positioning and the acquired boundary points of the fourth map. In this way, the method combining the single-line laser radar mapping and the visual mapping is used, the grid map is established by using the single-line laser radar to guide the robot exploration, the richer map is established by using the visual mapping combined with the semantic information to realize the mutual positioning of the multi-robot, the two modules are combined to realize the autonomous exploration of the multi-robot, and the exploration efficiency can be effectively improved.

[0130] The preferred embodiments of the present application are described in detail above with reference to the drawings, but the present application is not limited to the specific details in the above-described embodiments. Within the technical concept of the present application, various simple modifications can be made to the technical scheme of the present application, and these simple modifications all belong to the protection scope of the present application. For example, in the above-described embodiments, various specific technical features are described, which can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, various possible combination manners are not described again in the present application. For example, various different embodiments of the present application can also be combined arbitrarily, as long as it does not deviate from the idea of the present application, and it should also be considered as disclosed in the present application. For example, under the premise of no conflict, each embodiment described in the present application and / or the technical features in each embodiment can be combined with any prior art, and the technical scheme obtained after the combination should also fall within the protection scope of the present application.

[0131] It should be understood that in various method embodiments of the present application, the size of the serial number of the above-described processes does not mean the order of execution, and the execution order of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0132] based on the same inventive concept as the preceding embodiments, Figure 4 is a structural composition schematic diagram of the path planning device provided by the embodiments of the present application, as Figure 4 indicated, the path planning device comprises:

[0133] The first acquisition unit 401 is configured to acquire a first local map and a second local map; the first local map is a map obtained by a first robot in a first region, and the second local map is a map obtained by a second robot in a second region; the first region and the second region are both part of a to-be-tested region, and the to-be-tested region is the entire region of path planning.

[0134] The first acquisition unit 401 is further configured to obtain a relative pose relationship between the first robot and the second robot.

[0135] The first processing unit 402 is configured to fuse the first local map and the second local map according to the relative pose relationship to obtain a first global map.

[0136] The first acquisition unit 401 is further configured to obtain a boundary point of the first global map, and construct a first boundary point set according to the boundary point of the first global map.

[0137] The first processing unit 402 is further configured to plan a path of the first robot and / or the second robot according to the first boundary point set.

[0138] In some embodiments, the first processing unit 402 is specifically further configured to update a relative pose association graph according to the relative pose relationship; the relative pose association graph is a relative pose association graph of robots in a first robot set, and the first robot set includes the first robot and the second robot.

[0139] In some embodiments, the first acquisition unit 401 is specifically further configured to acquire a first image, and the first image is collected by the first robot.

[0140] In some embodiments, the first processing unit 402 is specifically further configured to determine a second image matched with the first image, and the second image is collected by the second robot; determine a plurality of images matched with the second image, and the plurality of images are collected by the first robot, and the plurality of images form a target image set; and determine the relative pose relationship between the first robot and the second robot according to the second image and the target image set.

[0141] In some embodiments, the first acquisition unit 401 is specifically further configured to acquire semantic information of the first image, determine a label of the first image according to the semantic information of the first image; and acquire semantic information of the second image, and determine a label of the second image according to the semantic information of the second image.

[0142] In some embodiments, the first obtaining unit 401 is further configured to: obtain an initial image set; images in the initial image set are images captured by the first robot at positions where the first image is captured, and the initial image set includes the first image; compare the label of the second image with labels of images in the initial image set, and determine an intermediate image set according to a comparison result; obtain a matching result of the second image and images in the intermediate image set, and determine the target image set according to the matching result.

[0143] In some embodiments, the first processing unit 402 is further configured to: obtain a first list and a second list, the first list including the second image, feature information of the second image, and position information where the second image is captured by the second robot, and the second list including the target image set, feature information of the target image set, and position information where the target image set is captured by the first robot; obtain a relative pose transformation according to a matching result of the first list and the second list; and determine a relative pose relationship between the first robot and the second robot according to the relative pose transformation.

[0144] In some embodiments, the first processing unit 402 is further configured to: divide boundary points of the first global map into a plurality of sets; obtain a boundary point pair in the plurality of sets, the distance between two boundary points in the boundary point pair being less than a second threshold, and points on a line connecting the two boundary points being unoccupied; screen another boundary point of the boundary point pair, which has a frequency of occurrence greater than or equal to a third threshold, to obtain a first part of boundary points; and construct a first boundary point set according to the first part of boundary points and a second part of boundary points, the second part of boundary points being boundary points in the plurality of sets other than the boundary point pair.

[0145] Those skilled in the art should understand that, Figure 4 The implementation functions of each unit in the path planning apparatus shown can be understood with reference to the related descriptions of the foregoing methods. Figure 4 The functions of each unit in the path planning apparatus shown can be implemented by a program running on a processor, or by a specific logic circuit.

[0146] Figure 5 FIG. 5 is a schematic structural diagram of a processing device 500 provided by an embodiment of the present application. Figure 5 The processing device 500 shown includes a processor 501, which can call and run a computer program from a memory to implement the method in the embodiments of the present application.

[0147] Optionally, as shown, Figure 5 The processing device 500 shown can also include a memory 502. The processor 501 can call and run a computer program from the memory 502 to implement the method in the embodiments of the present application.

[0148] The memory 502 can be a separate device independent of the processor 501, or can be integrated in the processor 501.

[0149] Optionally, as shown in the figure, the processing device 500 can further include a transceiver 503, and the processor 501 can control the transceiver 503 to communicate with other devices, specifically, can send information or data to other devices, or receive information or data sent by other devices. Figure 5

[0150] The transceiver 503 can include a transmitter and a receiver. The transceiver 503 can further include an antenna, and the number of antennas can be one or more.

[0151] Figure 6 The chip is a schematic structural diagram of the chip of the embodiment of the present application. Figure 6 The chip 600 shown in the figure includes a processor 601, which can call and run a computer program from the memory to implement the method in the embodiment of the present application.

[0152] Optionally, as shown in the figure, the chip 600 can further include a memory 602. The processor 601 can call and run a computer program from the memory 602 to implement the method in the embodiment of the present application. Figure 6

[0153] The memory 602 can be a separate device independent of the processor 601, or can be integrated in the processor 601.

[0154] Optionally, the chip 600 can further include an input interface 603. The processor 601 can control the input interface 603 to communicate with other devices or chips, specifically, can obtain information or data sent by other devices or chips.

[0155] Optionally, the chip 600 can further include an output interface 604. The processor 601 can control the output interface 604 to communicate with other devices or chips, specifically, can output information or data to other devices or chips.

[0156] Optionally, the chip can be applied to the processing device in the embodiment of the present application, and the chip can implement the corresponding processes realized by the processing device in each method of the embodiment of the present application. For the sake of brevity, it will not be repeated here.

[0157] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system chip, a chip system, or a system on chip, etc.

[0158] ​​It should be understood that the processor of the embodiments of the present application can be an integrated circuit chip with a processing capability of signals. In the implementation process, each step of the method embodiments described above can be completed by the integrated logic circuit of hardware in the processor or the instructions in the form of software. The processor described above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware coding processor for execution, or a combination of hardware and software modules in the coding processor for execution. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the storage, and the processor reads the information in the storage, and combines the hardware to complete the steps of the above method.

[0159] It is to be understood that the memory in the embodiments of the present application can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a read-only memory (Read-Only Memory, ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM) or a flash memory. The volatile memory can be a random access memory (Random Access Memory, RAM) used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synchlink DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM). It should be noted that the memory of the system and method described herein is intended to include, but not limited to, these and any other suitable types of memory.

[0160] It should be understood that the above-mentioned memory is exemplary but not limiting, for example, the memory in the embodiments of the present application can also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM) and the like. That is, the memory in the embodiments of the present application is intended to include, but not limited to, these and any other suitable types of memory.

[0161] The embodiment of the present application further provides a computer readable storage medium for storing the computer program.

[0162] Optionally, the computer readable storage medium can be applied to the processing device in the embodiment of the present application, and the computer program makes the computer execute the corresponding process realized by the processing device in each method of the embodiment of the present application. For the sake of brevity, details are not repeated here.

[0163] The embodiment of the present application further provides a computer program product comprising computer program instructions.

[0164] Optionally, the computer program product can be applied to the processing device in the embodiment of the present application, and the computer program instructions make the computer execute the corresponding process realized by the processing device in each method of the embodiment of the present application. For the sake of brevity, details are not repeated here.

[0165] The embodiment of the present application further provides a computer program.

[0166] Optionally, the computer program can be applied to the processing device in the embodiment of the present application, and when the computer program runs on the computer, makes the computer execute the corresponding process realized by the processing device in each method of the embodiment of the present application. For the sake of brevity, details are not repeated here.

[0167] Those skilled in the art can understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software mode depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0168] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the above-mentioned system, device and unit can refer to the corresponding process in the foregoing method embodiments, and details are not repeated here.

[0169] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be realized by other ways. For example, the above-mentioned device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division mode, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0170] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0171] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0172] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the part of the present application which essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0173] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be limited by the protection scope of the claims.

Claims

1. A path planning method, characterized in that, The method includes: Obtain a first partial map and a second partial map; the first partial map is a map obtained by the first robot in a first region, and the second partial map is a map obtained by the second robot in a second region; both the first region and the second region are part of the region to be tested, and the region to be tested is the entire region of the path planning; Obtain the relative pose relationship between the first robot and the second robot; The first local map and the second local map are fused according to the relative pose relationship to obtain the first global map; Obtain the boundary points of the first global map, and construct a first set of boundary points based on the boundary points of the first global map; Plan the path of the first robot and / or the second robot based on the first set of boundary points; The step of obtaining the relative pose relationship between the first robot and the second robot includes: acquiring a first image, which is acquired by the first robot; determining a second image that matches the first image, which is acquired by the second robot; determining multiple images that match the second image, which are acquired by the first robot, and the multiple images form a target image set; and determining the relative pose relationship between the first robot and the second robot based on the second image and the target image set. The step of constructing a first boundary point set based on the boundary points of the first global map includes: dividing the boundary points of the first global map into multiple sets; obtaining boundary point pairs in the multiple sets, wherein the distance between two boundary points in the boundary point pair is less than a second threshold and the points on the line connecting the two boundary points are not occupied; filtering out another boundary point in the boundary point pair whose occurrence frequency is greater than or equal to a third threshold to obtain a first part of boundary points; constructing a first boundary point set based on the first part of boundary points and the second part of boundary points; wherein the second part of boundary points are boundary points in the multiple sets other than the boundary point pairs; the step of filtering out another boundary point in the boundary point pair whose occurrence frequency is greater than or equal to the third threshold includes: deleting another boundary point in the boundary point pair whose occurrence frequency is greater than or equal to the third threshold.

2. The method according to claim 1, characterized in that, After fusing the first local map and the second local map according to the relative pose relationship to obtain the first global map, the method includes: The relative pose association graph is updated based on the relative pose relationship; the relative pose association graph is the relative pose association graph of robots in the first robot set, and the first robot set includes the first robot and the second robot.

3. The method according to claim 2, characterized in that, The method further includes: Obtain the semantic information of the first image, and determine the label of the first image based on the semantic information of the first image; Obtain the semantic information of the second image, and determine the label of the second image based on the semantic information of the second image.

4. The method according to claim 3, characterized in that, The determination of multiple images that match the second image, wherein the multiple images are acquired by the first robot, and the multiple images form a target image set, including: Obtain an initial image set; the images in the initial image set are images acquired by the first robot at the location where the first image was acquired, and the initial image set includes the first image; Compare the label of the second image with the labels of each image in the initial image set, and determine the intermediate image set based on the comparison results; Obtain the matching results between the second image and the images in the intermediate image set, and determine the target image set based on the matching results.

5. The method according to claim 2, characterized in that, Determining the relative pose relationship between the first robot and the second robot based on the second image and the target image set includes: Obtain a first list and a second list. The first list includes the second image, the feature information of the second image, and the location information of the second robot acquiring the second image. The second list includes the target image set, the feature information of the target image set, and the location information of the first robot acquiring the target image set. Based on the matching results of the first list and the second list, the relative pose transformation is obtained; The relative pose relationship between the first robot and the second robot is determined based on the relative pose transformation.

6. A path planning device, characterized in that, The device includes: The first acquisition unit is used to acquire a first local map and a second local map; the first local map is a map obtained by the first robot in a first region, and the second local map is a map obtained by the second robot in a second region; both the first region and the second region are part of the region to be tested, and the region to be tested is the entire region of the path planning; The first acquisition unit is also used to acquire the relative pose relationship between the first robot and the second robot; The first processing unit is used to fuse the first local map and the second local map according to the relative pose relationship to obtain a first global map; The first acquisition unit is further configured to acquire the boundary points of the first global map and construct a first boundary point set based on the boundary points of the first global map; The first processing unit is further configured to plan the path of the first robot and / or the second robot based on the first set of boundary points; The first acquisition unit is further configured to acquire a first image, which is acquired by the first robot; the first processing unit is further configured to determine a second image that matches the first image, which is acquired by the second robot; determine multiple images that match the second image, which are acquired by the first robot, and form a target image set; and determine the relative pose relationship between the first robot and the second robot based on the second image and the target image set. The first processing unit is further configured to divide the boundary points of the first global map into multiple sets; obtain boundary point pairs in the multiple sets, wherein the distance between two boundary points in the boundary point pair is less than a second threshold and the points on the line connecting the two boundary points are not occupied; filter out another boundary point in the boundary point pair whose occurrence frequency is greater than or equal to a third threshold to obtain a first part of boundary points; construct a first set of boundary points based on the first part of boundary points and the second part of boundary points; wherein the second part of boundary points are boundary points in the multiple sets other than the boundary point pairs; the first processing unit is further configured to delete another boundary point in the boundary point pair whose occurrence frequency is greater than or equal to the third threshold.

7. A processing device, characterized in that, include: A processor and a memory for storing a computer program, the processor for calling and running the computer program stored in the memory to perform the method as described in any one of claims 1 to 5.

8. A chip, characterized in that, include: A processor for retrieving and running a computer program from memory, causing a device on which the chip is mounted to perform the method as described in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1 to 5.

Images