Image matching method, map construction method, driving control method, equipment and medium

Abstract

The invention relates to the field of equipment control, and provides an image matching method, a map construction method, a driving control method, automatic walking equipment and a computer readable storage medium. The method comprises the following steps: receiving a plurality of to-be-matched images and an index map acquired by collecting an edge area by automatic walking equipment; sorting the plurality of to-be-matched images to obtain a sorting result; classifying the plurality of to-be-matched images according to the sorting result to obtain a first image set and a second image set; projecting the three-dimensional feature points of the index map to a space coordinate system corresponding to the to-be-matched images in the first image set, and taking the to-be-matched images conforming to the first projection result in the first image set as first target images until the total number of the first target images reaches a target number threshold value; and establishing a matching image set matched with the edge region according to the first target image. The method and the device aim at accurately determining the image matched with the marginal area so as to improve the speed and the quality of map construction.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, to an image matching method, a map construction method, a driving control method, an automatic walking device, and a computer-readable storage medium. Background Art

[0002] In order to reduce the labor intensity and cost of lawn maintenance, automatic walking devices for cutting and trimming lawns, such as lawn mowers, have emerged. At present, when some lawn mowers are driving and operating on a lawn, they need to collect image data to construct a map of the surrounding environment of the automatic walking device.

[0003] However, the existing composition schemes have a large amount of image data required for composition, and it is difficult to determine the images that match the edge of the lawn site from a large number of collected images, resulting in poor speed and quality of map construction. Summary of the Invention

[0004] The main purpose of the present application is to provide an image matching method, a map construction method, a driving control method, an automatic walking device, and a computer-readable storage medium, aiming to accurately determine the images that match the edge area to improve the speed and quality of map construction.

[0005] In a first aspect, the present application provides an image matching method, and the image matching method includes the following steps: Receiving a plurality of images to be matched collected by an automatic walking device for an edge area in a target working area, and obtaining an index map corresponding to the edge area; Sorting the plurality of images to be matched in descending order according to the distance information between the images to be matched and the index map to obtain a sorting result; Classifying the plurality of images to be matched according to the sorting result to obtain a first image set and a second image set, wherein the distance information corresponding to the images to be matched in the first image set is greater than the distance information corresponding to the images to be matched in the second image set; Sequentially projecting the three-dimensional feature points of the index map onto the spatial coordinate system corresponding to the images to be matched in the first image set according to the sorting result, and using the images to be matched in the first image set that meet the first projection result as first target images until the total number of the first target images reaches a target number threshold; Establishing a matching image set that matches the edge area according to the first target images.

[0006] In a second aspect, the present application provides a map construction method, which is applied to an automatic walking device, and the map construction method includes the following steps: Retrieve a set of matching images corresponding to the edge region in the target working area, where the set of matching images is generated by the image matching method described above; Establish an edge map corresponding to the edge region according to the set of matching images.

[0007] In a third aspect, the present application provides a driving control method applied to an automatic walking device, and the driving control method includes the following steps: Retrieve an edge map corresponding to the edge region in the target working area, where the edge map is generated by the map construction method described above; Determine the driving pose of the automatic walking device according to the edge map and the image collected by the automatic walking device, and perform a driving operation in the target working area according to the driving pose.

[0008] In a fourth aspect, the present application further provides an automatic walking device, and the automatic walking device includes: A main body; A driving module for driving the main body to travel; A working module disposed on the main body and used for mowing the position where the automatic walking device is located; A collection module at least used for collecting images of the environment where the automatic walking device is located and collecting the pose of the automatic walking device; A controller connected to the driving module and the collection module, and used for executing at least one of the image matching method, the map construction method, and the driving control method described above.

[0009] In a fifth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it realizes at least one of the image matching method, the map construction method, and the driving control method described above.

[0010] The present application provides an image matching method, a map construction method, a driving control method, an automatic walking device, and a computer-readable storage medium. The present application receives multiple to-be-matched images acquired by an automatic walking device through collecting an edge area in a target working area, and obtains an index map corresponding to the edge area; sorts the multiple to-be-matched images according to the distance information between the to-be-matched images and the index map from high to low to obtain a sorting result; classifies the multiple to-be-matched images according to the sorting result to obtain a first image set and a second image set; projects three-dimensional feature points of the index map onto a spatial coordinate system corresponding to the to-be-matched images in the first image set in sequence according to the sorting result, and uses the to-be-matched images in the first image set that conform to the first projection result as first target images until the total number of the first target images reaches a target number threshold; establishes a matching image set that matches the edge area according to the first target images. Thus, images that can be quickly and accurately matched with the edge area can be obtained from a large number of collected images. The computational complexity of image matching is relatively low, the time consumption is short, and the stability is high, which is beneficial to accurately constructing an edge map, thereby reducing the data volume of map construction, reducing the complexity of map construction, and improving the speed and quality of map construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1 It is a schematic flowchart of an image matching method provided by an embodiment of the present application; Figure 2 It is a schematic diagram of the movement trajectory of an automatic walking device provided by an embodiment of the present application; Figure 3 It is a schematic diagram of the movement trajectory of an automatic walking device provided by another embodiment of the present application; Figure 4 It is a schematic diagram of a to-be-matched image obtained by an automatic walking device during rotational movement provided by an embodiment of the present application; Figure 5 It is a schematic diagram of a to-be-matched image obtained by an automatic walking device during forward movement provided by another embodiment of the present application; Figure 6 It is a schematic diagram of a to-be-matched image obtained by an automatic walking device during crab movement provided by another embodiment of the present application; Figure 7 It is a schematic diagram of the depth distance between the acquisition position of the to-be-matched image and the acquisition position of the index map provided by an embodiment of the present application; Figure 8 A flowchart of a map construction method provided by an embodiment of the present application; Figure 9 A flowchart of a driving control method provided by an embodiment of the present application; Figure 10 A schematic structural diagram of an automatic walking device provided by an embodiment of the present application; Figure 11 A schematic block diagram of the structure of an automatic walking device provided by an embodiment of the present application. Detailed implementation manners

[0013] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0014] The flowcharts shown in the accompanying drawings are only illustrative, and do not necessarily include all the contents and operations / steps, nor do they necessarily need to be executed in the described order. For example, some operations / steps can be decomposed, combined, or partially merged, so the actual execution order may change according to the actual situation.

[0015] The embodiments of the present application provide an image matching method, a map construction method, a driving control method, an automatic walking device, and a computer-readable storage medium.

[0016] Next, some embodiments of the present application will be described in detail in conjunction with the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0017] Please refer to Figure 1 , Figure 1Schematic flowchart of an image matching method provided by an embodiment of the present application. The image matching method can be applied to an automatic walking device, a terminal, or a server to quickly and accurately match a set of matching images with an edge area from a large number of collected images, so as to facilitate subsequent construction of an edge map corresponding to the edge area. Among them, the automatic walking device can be a robot such as a lawn mowing robot or a floor sweeping robot; the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto; the server can be an independent server, a server cluster, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery network (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0018] As Figure 1 shown, the image matching method includes steps S101 to S105.

[0019] Step S101, receive multiple images to be matched collected by the automatic walking device for the edge area in the target working area, and obtain an index map corresponding to the edge area.

[0020] Among them, the target working area can be the area where the automatic walking device works. Generally, the automatic walking device will move within the target working area. The edge area is the relevant area located at the edge position in the target working area, such as the area near the boundary of the target working area. The images to be matched are the images collected by the automatic walking device for the edge area at different collection positions in the target working area. The index map corresponding to the edge area is the image collected by the automatic walking device for the edge area at a specific collection position in the target working area, and the map constructed using these images. The specific position generally refers to the collection position corresponding to the standard image of the edge area.

[0021] Exemplarily, different from the related art in which the automatic walking device obtains images without difference at various positions in the entire target working area and uses all the obtained images to construct a map, in the map construction method provided by the technical solution of the present application, only the edge images obtained by the automatic walking device at the edge position of the target working area are used to construct the edge map. Since the edge position is the position where the automatic walking device is prone to deviation during movement, the edge map constructed by the edge images has a small data volume while being able to effectively correct the deviation of the automatic walking device. Therefore, in order to accurately construct the edge map, it is necessary to quickly and accurately match the images with the edge area (such as the edge of the lawn site) from a large number of collected images.

[0022] Exemplarily, the target working area has at least two different edge positions, and corresponding edge areas are formed on one side close to each edge position. Multiple to-be-matched images collected when the automatic walking device works in the target working area are obtained.

[0023] Taking the automatic walking device as a lawn mower as an example, during the operation of the lawn mower on the lawn, the lawn has a certain operation boundary. Therefore, there are operation boundaries in multiple directions forming the target working area. Therefore, during the operation of the lawn mower, multiple to-be-matched images can be obtained by collecting the edge areas of the lawn.

[0024] As Figure 2 shown, exemplarily, the automatic walking device can reciprocate along a bow-shaped path in the target working area, so that the movement trajectory can cover the target working area as much as possible. In the embodiment of the present application, the position where the automatic walking device turns around can be used as the edge position, and the area formed by the positions before and after the automatic walking device turns around is used as the edge area.

[0025] As Figure 3 shown, the actual movement trajectory of the automatic walking device may be offset. For example, due to the cumulative error generated during the movement of the automatic walking device, it is usually difficult to ensure that the movement trajectories are completely parallel, which easily leads to different angles of different paths and causes some positions in the target working area to be not covered. Taking the lawn mowing robot as an example, Figure 3 the black area in

[0026] is the missed area of the lawn mowing operation. Figure 3 To avoid the above situation, in the embodiment of the present application, an edge map is constructed for the edge area in front of the automatic walking device turning around in the target working area, that is, the position where the automatic walking device turns around is used as the edge position, and the area formed by the positions before and after the automatic walking device turns around is used as the edge area. Before each turn, the movement trajectory of the automatic walking device is corrected to avoid continuous generation and accumulation of errors during the movement of the automatic walking device as

[0027] As Figures 4 - 6 shown, the triangles in the figure represent image data, and the circles in the figure represent three-dimensional feature points in the target working area.

[0028] As Figure 4 shown, during the rotational movement, the viewing angle of the automatic walking device changes greatly, resulting in significant changes in the projection positions of the feature points in different image data. If the rotation angle is relatively large, the feature points may even exceed the viewing range of the image. Therefore, the co-visibility of the image data 1, image data 2, and image data 3 obtained by the automatic walking device during the rotational movement is poor.

[0029] As Figure 5 shown, the co - visibility ability of image data 1, image data 2, and image data 3 obtained by the automatic walking device during forward movement is better than that during rotational movement. However, during forward movement, the automatic walking device moves linearly along a certain direction, and the parallax change is mainly reflected in the depth direction. As the automatic walking device moves forward, the position change of feature points with a smaller shooting angle (such as Figure 5 feature point 2 in Figure 5 is small in the image, while feature points with a smaller shooting angle (such as

[0030] feature point 1 and feature point 3 in Figure 6 may quickly move out of the field of view. In this case, the number of co - visible feature points between key frames may be small, and the co - visibility ability of image data 1, image data 2, and image data 3 is poor.

[0031] As

[0032] shown, the co - visibility ability of image data 1, image data 2, and image data 3 obtained by the automatic walking device during crab - like movement is strong. During crab - like movement, the automatic walking device mainly moves horizontally, and the parallax change is mainly reflected in the horizontal direction. This movement method makes the horizontal position change of three - dimensional feature points large in the image, but it is easier to keep them within the field of view. Therefore, the number of co - visible feature points between key frames during crab - like movement is usually large, which is conducive to establishing a richer co - visibility relationship, and the co - visibility ability of image data 1, image data 2, and image data 3 is strong.

[0033] Among them, the distance information is the Euclidean distance between the acquisition position of the to - be - matched image and the acquisition position of the index map. Since the acquisition position of the index map is fixed and the acquisition positions of the to - be - matched images are different, the to - be - matched images can be sorted from large to small according to the Euclidean distance between the acquisition position of each to - be - matched image and the acquisition position of the index map, so as to obtain the corresponding sorting result.

[0034] Since the coordinate systems corresponding to the image to be matched and the index map are different, it is impossible to directly calculate the Euclidean distance between the acquisition position of the image to be matched and the acquisition position of the index map. Therefore, before calculating the Euclidean distance, the coordinate system corresponding to the image to be matched can be first converted into the coordinate system corresponding to the index map, or the coordinate system corresponding to the index map can be converted into the coordinate system corresponding to the image to be matched, that is, the Euclidean distance is calculated after converting to the same coordinate system.

[0035] Exemplarily, a coordinate system can be constructed with reference to the index map, and then the device pose corresponding to each image to be matched is mapped to the coordinate system corresponding to the index map, so that the device poses corresponding to each image to be matched and the device pose corresponding to the index map are in the same coordinate system. Thus, the Euclidean distance between the acquisition position of the image to be matched and the acquisition position of the index map can be calculated through the device pose corresponding to each image to be matched and the device pose corresponding to the index map.

[0036] Finally, after calculating the Euclidean distance between the acquisition position of each image to be matched and the acquisition position of the index map, the Euclidean distances corresponding to each image to be matched are sorted from largest to smallest, so as to obtain the corresponding sorting result.

[0037] Step S103: Classify multiple images to be matched according to the sorting result to obtain a first image set and a second image set, where the distance information corresponding to the images to be matched in the first image set is greater than the distance information corresponding to the images to be matched in the second image set.

[0038] Since the larger the Euclidean distance between the acquisition position of the image to be matched and the acquisition position of the index map, the better the image triangulation quality. Therefore, the quality of the image to be matched can be determined by the Euclidean distance. Since the distance information corresponding to the images to be matched in the first image set is greater than the distance information corresponding to the images to be matched in the second image set, the quality of the images to be matched in the first image set is better than the quality of the images to be matched in the second image set.

[0039] Exemplarily, the first n of the sorting results can be selected to form the first image set, and the remaining images to be matched can be selected to form the second image set, where n can be any integer value.

[0040] Exemplarily, it can also be determined whether the Euclidean distance corresponding to each image to be matched is greater than a preset distance threshold. The images to be matched that are greater than or equal to the preset distance threshold are selected to form the first image set, and the images to be matched that are less than the preset distance threshold are selected to form the second image set.

[0041] Step S104: Project the three-dimensional feature points of the index map onto the spatial coordinate system corresponding to the images to be matched in the first image set in sequence according to the sorting result, and use the images to be matched in the first image set that conform to the first projection result as the first target images until the total number of the first target images reaches the target number threshold.

[0042] Among them, the three-dimensional feature points can be any map points in the index map. The target number threshold can be any value and is not specifically limited here. The first target image is the image to be matched in the first image set that conforms to the first projection result, and the corresponding projection result of the first projection result is: the number of successfully matched three-dimensional feature points in the image to be matched in the first image set exceeds the corresponding feature point number threshold.

[0043] In some embodiments, according to the projection coordinates of the three-dimensional feature points projected onto the spatial coordinate system, determine the number of feature points of the three-dimensional feature points located within the image to be matched; when the number of feature points exceeds the feature point number threshold, determine the image to be matched as the first target image. In this way, the first target images can be accurately screened out from the images to be matched.

[0044] In the related art, generally, the pixel matching method is used to determine the first target image, that is, each pixel point of the image to be matched is traversed for matching. However, the data volume of this matching method is large, resulting in a long time consumption and low efficiency in determining the first target image. In the embodiments of the present application, there is no need to use the pixel matching method to determine the first target image, and the first target image can be directly determined by using the projection quantity of the three-dimensional feature points. It can not only accurately screen out the first target images from the images to be matched, but also has a low computational complexity for image matching, short time consumption, and high stability, ensuring the real-time nature of map construction and ensuring that the automatic walking device can operate normally without stopping and waiting.

[0045] Exemplarily, based on the device pose and camera internal parameters corresponding to the index map, project the three-dimensional feature points onto the spatial coordinate system corresponding to the images to be matched in the first image set, so as to obtain the projection coordinates of the three-dimensional feature points in the spatial coordinate system; according to the projection coordinates and the image coordinates of the image to be matched, it can be determined whether each three-dimensional feature point is located within the image to be matched, and then determine the number of feature points of the three-dimensional feature points located within the image to be matched; when the number of feature points exceeds the feature point number threshold, determine the image to be matched as the first target image, and so on, each image to be matched in the first image set can be traversed, so as to determine all the first target images from the first image set.

[0046] Exemplarily, according to the sorting result of the images to be matched in the first image set, based on the device pose and camera internal parameters corresponding to the index map, the three-dimensional feature points of the index map can be sequentially projected into the spatial coordinate system of the corresponding images to be matched in the first image set, so that the images to be matched with good quality in the first image set can be traversed preferentially.

[0047] In some embodiments, a spatial coordinate system is established according to the acquisition position of the image to be matched; based on the real-world coordinates of the three-dimensional feature points and the spatial coordinate system, the projection coordinates corresponding to the three-dimensional feature points in the spatial coordinate system are determined.

[0048] Since the coordinate systems corresponding to the image to be matched and the three-dimensional feature points are different, it is impossible to directly determine whether the three-dimensional feature points are within the image to be matched. Therefore, before determining whether the three-dimensional feature points are within the image to be matched, the three-dimensional feature points can be projected into the coordinate system corresponding to the image to be matched (i.e., the spatial coordinate system).

[0049] Exemplarily, a spatial coordinate system can be established according to the acquisition position of each image to be matched; based on the coordinate system conversion relationship, the real-world coordinates of the three-dimensional feature points are converted into the spatial coordinate system, so as to determine the projection coordinates corresponding to the three-dimensional feature points in the spatial coordinate system.

[0050] In some embodiments, the coordinate range corresponding to the image to be matched is determined in the spatial coordinate system according to the image contour of the image to be matched; the number of three-dimensional feature points whose corresponding projection coordinates are within the coordinate range is counted as the number of feature points.

[0051] Exemplarily, since the image to be matched generally has a certain size, therefore, the coordinate range corresponding to the image to be matched in the spatial coordinate system can be determined according to the image contour of the image to be matched. For example, it can be a rectangular range, a circular range, etc. These ranges all occupy at least multiple coordinates. Then, the number of three-dimensional feature points whose corresponding projection coordinates are within the coordinate range is counted, and this number of three-dimensional feature points is used as the number of feature points.

[0052] It should be noted that if the projection coordinates corresponding to the three-dimensional feature points are on the boundary of the coordinate range, the projection coordinates can be considered to be within the coordinate range, or the projection coordinates can be considered to be outside the coordinate range. No specific limitation is made here.

[0053] Step S105, establish a set of matching images that match the edge area according to the first target image.

[0054] If the total number of the first target images reaches the target number threshold, a set of matching images that match the edge area can be established according to the first target image, so as to establish an edge map corresponding to the edge area according to the set of matching images.

[0055] After traversing all the images to be matched in the first image set, the images to be matched in the first image set that conform to the first projection result are obtained as the first target images, and the number of the first target images is counted. If the total number of the first target images reaches the target number threshold, it means that enough images with the best quality are found for accurate subsequent map construction. If the total number of the first target images does not reach the target number threshold, it means that more and better-quality images need to be found for subsequent map construction.

[0056] In some embodiments, if the number of the first target images is less than the target number threshold, the second image set is classified according to the depth distance to obtain a third image set and a fourth image set. Among them, the depth distance of the images to be matched in the third image set is within a preset depth distance range, and the depth distance of the images to be matched in the fourth image set is outside the preset depth distance range; the three-dimensional feature points of the index map are projected onto the spatial coordinate system corresponding to the images to be matched in the third image set in the order of the sorting sequence, and the images to be matched in the third image set that conform to the second projection result are used as the second target images until the total number of the first target images and the second target images reaches the target number threshold; a matching image set is established according to the first target images and the second target images. Thus, when the number of the first target images is insufficient, the images to be matched can be further screened to find better-quality images for subsequent map construction.

[0057] Among them, the second target image is the image to be matched in the third image set that conforms to the second projection result, and the projection result corresponding to the second projection result is that the number of successfully matched three-dimensional feature points in the image to be matched in the third image set exceeds the corresponding feature point number threshold.

[0058] As Figure 7 shown, for the acquisition position of the index map, within the range of x meters before and after in the depth direction, it is its best depth matching area, where the value of x can be any value, which can be determined according to the actual situation, or the best depth matching area can be divided according to the acquisition positions of the images to be matched in the first image set. In Figure 7 it, the first image set includes Image Data 1, Image Data 2, Image Data 3, Image Data 4, and Image Data 5, and these image data are located within the best depth matching area, and the second image set includes the remaining image data, and these image data are located outside the best depth matching area.

[0059] As Figure 7As shown, exemplarily, before classifying the second image set, first calculate the depth distance of the images to be matched in the second image set. The depth distance of the image to be matched can be calculated by the position of the image to be matched and the boundary of the optimal depth matching region, and the straight-line distance between the image to be matched and the boundary of the optimal depth matching region is the depth distance of the image to be matched.

[0060] Exemplarily, first determine whether the depth distance of each image to be matched in the second image set is within a preset depth distance range. If there are images to be matched whose depth distances are within the preset depth distance range, then screen out these images to be matched to form a third image set. If there are images to be matched whose depth distances are outside the preset depth distance range, then screen out these images to be matched to form a fourth image set.

[0061] Among them, the preset depth distance range can be any depth distance range, and no specific limitation is made here.

[0062] Exemplarily, according to the sorting order of the images to be matched in the second image set, project the three-dimensional feature points of the index map into the spatial coordinate system of the corresponding images to be matched in the second image set in sequence, so as to be able to preferentially traverse the images to be matched with good quality in the second image set.

[0063] Exemplarily, project the three-dimensional feature points of the index map into the spatial coordinate system corresponding to the images to be matched in the third image set. According to the projection coordinates of the three-dimensional feature points projected into the spatial coordinate system, determine the number of feature points of the three-dimensional feature points located within the image to be matched; when the number of feature points exceeds the feature point number threshold, determine the image to be matched as the second target image; until the total number of the first target images and the second target images reaches the target number threshold; establish a matching image set according to the first target images and the second target images.

[0064] If the total number of the first target images and the second target images still does not reach the target number threshold, it means that more images with relatively good quality need to be further found for subsequent map construction.

[0065] In some embodiments, if the total number of the first target images and the second target images is less than the target number threshold; project the three-dimensional feature points of the index map into the spatial coordinate system corresponding to the images to be matched in the fourth image set in sequence according to the sorting order, and use the images to be matched in the fourth image set that meet the third projection result as the third target images until the total number of the first target images, the second target images and the third target images reaches the target number threshold; establish a matching image set according to the first target images, the second target images and the third target images.

[0066] Among them, the third target image is a to-be-matched image in the fourth image set that conforms to the third projection result, and the projection result corresponding to the third projection result is that the number of successfully matched three-dimensional feature points in the to-be-matched images in the fourth image set exceeds the corresponding feature point quantity threshold.

[0067] It should be noted that the corresponding feature point quantity thresholds in the first projection result, the second projection result, and the third projection result may be the same or different.

[0068] It should be noted that the step of projecting the three-dimensional feature points of the index map into the corresponding spatial coordinate system of the to-be-matched images in the fourth image set in sequence according to the sorting order, and using the to-be-matched images in the fourth image set that conform to the third projection result as the third target images is similar to the step of projecting the three-dimensional feature points of the index map into the corresponding spatial coordinate system of the to-be-matched images in the first image set in sequence according to the sorting result, and using the to-be-matched images in the first image set that conform to the first projection result as the first target images, and the step of projecting the three-dimensional feature points of the index map into the corresponding spatial coordinate system of the to-be-matched images in the third image set in sequence according to the sorting order, and using the to-be-matched images in the third image set that conform to the second projection result as the second target images. Reference can be made to the relevant embodiments above, and details will not be repeated here.

[0069] Please refer to Figure 8 , Figure 8 FIG. is a schematic flowchart of a map construction method provided by an embodiment of the present application. This map construction method is applied to an automatic walking device and includes steps S201 to S202.

[0070] S201. Retrieve a set of matching images corresponding to the edge area in the target working area; S202. Establish an edge map corresponding to the edge area according to the set of matching images.

[0071] The set of matching images obtained by the image matching method provided by the embodiment of the present application is beneficial to accurately constructing the edge map, reducing the data volume of map construction, reducing the complexity of map construction, and improving the speed and quality of map construction.

[0072] Please refer to Figure 9 , Figure 9 FIG. is a schematic flowchart of a driving control method provided by an embodiment of the present application. This driving control method is applied to an automatic walking device and includes steps S301 to S302.

[0073] S301. Retrieve the edge map corresponding to the edge area in the target working area; S302. Determine the driving pose of the automatic walking device according to the edge map and the image collected by the automatic walking device, and perform driving operations in the target working area according to the driving pose.

[0074] The edge map constructed by the map construction method provided in the embodiments of the present application is conducive to accurately determining the driving pose of the automatic walking device, enabling the automatic walking device to accurately perform driving operations in the target working area.

[0075] As Figure 10 shown, the embodiments of the present application further provide an automatic walking device 100, which includes a body 10, a driving module 20, a working module 30, a collection module (not shown in the figure), and a controller (not shown in the figure). The driving module 20 is used to drive the body 10 to travel; the working module 30 is arranged on the body 10 and is used to perform preset operations on the position where the automatic walking device 100 is located; the collection module is at least used to collect images of the environment where the automatic walking device 100 is located and collect the pose of the automatic walking device 100; the controller is connected to the driving module 20 and the collection module.

[0076] Exemplarily, the automatic walking device 100 provided in the embodiments of the present application can be used to perform mowing operations during movement. Of course, it is not limited thereto. The automatic walking device 100 provided in the embodiments of the present application can also perform operations such as cleaning, snow sweeping, and leaf blowing, which are not limited herein.

[0077] Taking the automatic walking device 100 as a lawn mower as an example, the working module 30 is a cutting mechanism, and the cutting mechanism is arranged at the bottom of the body and is used to cut the object to be cut. Among them, the object to be cut includes but is not limited to the grass on the lawn, in the garden, and on the path. That is, the self-propelled robot can cut the grass on the lawn to ensure the beauty of the lawn. The driving module 20 is arranged on the body 10 and is used to drive the body 10 to move forward, so that the body 10 can drive the cutting mechanism to cut the grass on the lawn along a preset trajectory, thereby greatly reducing manual operation, saving time and effort, and truly liberating people from the labor of lawn maintenance.

[0078] Please refer to Figure 11 , Figure 11 which is a schematic block diagram of the structure of an automatic walking device 100 provided in the embodiments of the present application. In Figure 11 it, the automatic walking device 100 includes a processor 200 and a memory 300. Among them, the processor 200 and the memory 300 are connected through a bus, and this bus can be any applicable bus such as an I2C (Inter-integrated Circuit) bus.

[0079] Among them, the memory 300 may include a storage medium and an internal memory. The storage medium can store an operating system and a computer program. The computer program includes program instructions that, when executed, can cause the processor to execute at least one of the image matching method, the map building method, and the driving control method described in any one of the embodiments.

[0080] The processor 200 is used to provide computing and control capabilities to support the operation of the entire autonomous mobile device 100.

[0081] Among them, the processor 200 may be a central processing unit (CPU), and the processor may also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and other types of processors. The general-purpose processor may be a microprocessor, or the general-purpose processor may also be any conventional processor, etc.

[0082] Among them, the processor 200 is used to run the computer program stored in the memory 300 and implement the following steps when executing the computer program: Receive multiple to-be-matched images acquired by the autonomous mobile device by collecting the edge area in the target working area, and obtain an index map corresponding to the edge area; Sort the multiple to-be-matched images according to the distance information between the to-be-matched images and the index map from high to low to obtain a sorting result; Classify the multiple to-be-matched images according to the sorting result to obtain a first image set and a second image set, where the distance information corresponding to the to-be-matched images in the first image set is greater than the distance information corresponding to the to-be-matched images in the second image set; Project the three-dimensional feature points of the index map onto the spatial coordinate system corresponding to the to-be-matched images in the first image set in sequence according to the sorting result, and use the to-be-matched images in the first image set that meet the first projection result as first target images until the total number of the first target images reaches a target number threshold; Establish a matching image set that matches the edge area according to the first target images.

[0083] In some embodiments, after the processor 200 implements using the to-be-matched image that meets the target projection result in the first image set as the first target image, it is used to implement: If the number of the first target images is less than the target number threshold, classify the second image set according to the depth distance to obtain a third image set and a fourth image set, where the depth distance of the to-be-matched images in the third image set is within a preset depth distance range, and the depth distance of the to-be-matched images in the fourth image set is outside the preset depth distance range; project the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the to-be-matched images in the third image set in sequence according to the sorting order, and use the to-be-matched images in the third image set that meet the second projection result as the second target images until the total number of the first target images and the second target images reaches the target number threshold; establish the matching image set according to the first target images and the second target images.

[0084] In some embodiments, after the processor 200 implements using the to-be-matched images that meet the second projection result in the third image set as the second target images, it is used to implement: If the total number of the first target images and the second target images is less than the target number threshold; project the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the to-be-matched images in the fourth image set in sequence according to the sorting order, and use the to-be-matched images in the fourth image set that meet the third projection result as the third target images until the total number of the first target images, the second target images, and the third target images reaches the target number threshold; establish the matching image set according to the first target images, the second target images, and the third target images.

[0085] In some embodiments, when the processor 200 implements using the to-be-matched images that meet the first projection result in the first image set as the first target images, it is used to implement: Determine the number of feature points of the three-dimensional feature points located within the to-be-matched image according to the projection coordinates of the three-dimensional feature points projected to the spatial coordinate system; when the number of feature points exceeds the feature point number threshold, determine the to-be-matched image as the first target image.

[0086] In some embodiments, when the processor 200 implements projecting the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the to-be-matched images in the first image set, it is used to implement: Establish the spatial coordinate system according to the acquisition position of the image to be matched; determine the projection coordinates corresponding to the three-dimensional feature points in the spatial coordinate system based on the actual coordinates of the three-dimensional feature points and the spatial coordinate system.

[0087] In some embodiments, when the processor 200 realizes determining the number of feature points of the three-dimensional feature points located within the image to be matched, it is used to realize: Determine the coordinate range corresponding to the image to be matched in the spatial coordinate system according to the image contour of the image to be matched; count the number of three-dimensional feature points whose corresponding projection coordinates are within the coordinate range as the number of feature points.

[0088] The processor 200 is further used to realize the following steps when executing the computer program: Retrieve a set of matching images corresponding to the edge area in the target working area; Establish an edge map corresponding to the edge area according to the set of matching images.

[0089] The processor 200 is further used to realize the following steps when executing the computer program: Retrieve an edge map corresponding to the edge area in the target working area; Determine the driving pose of the automatic walking device according to the edge map and the image collected by the automatic walking device, and perform driving operations in the target working area according to the driving pose.

[0090] In an embodiment of the present application, a computer-readable storage medium is further provided. The computer-readable storage medium stores a computer program, and the computer program includes program instructions. The processor executes the program instructions to implement at least one of the image matching method, map construction method, and driving control method provided in the embodiments of the present application. For example, when the computer program is loaded by the processor, the following steps can be executed: Receive multiple to-be-matched images acquired by an automatic walking device through collection of an edge area in a target working area, and obtain an index map corresponding to the edge area; sort the multiple to-be-matched images according to the distance information between the to-be-matched images and the index map in descending order to obtain a sorting result; classify the multiple to-be-matched images according to the sorting result to obtain a first image set and a second image set, wherein the distance information corresponding to the to-be-matched images in the first image set is greater than the distance information corresponding to the to-be-matched images in the second image set; project the three-dimensional feature points of the index map onto the spatial coordinate system corresponding to the to-be-matched images in the first image set in sequence according to the sorting result, and use the to-be-matched images in the first image set that conform to the first projection result as first target images until the total number of the first target images reaches a target number threshold; establish a matching image set that matches the edge area according to the first target images.

[0091] When the computer program is loaded by a processor, the following steps may further be executed: Retrieve a matching image set corresponding to an edge area in a target working area; establish an edge map corresponding to the edge area according to the matching image set.

[0092] When the computer program is loaded by a processor, the following steps may further be executed: Retrieve an edge map corresponding to an edge area in a target working area; determine the driving pose of the automatic walking device according to the edge map and the image acquired by the automatic walking device, and perform a driving operation in the target working area according to the driving pose.

[0093] For the specific implementation of each of the above operations, reference may be made to the previous embodiments, which will not be elaborated here.

[0094] It should be understood that the terms used in the specification of the present application are merely for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of the present application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0095] It should also be understood that the term "and / or" used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations. It should be noted that, in this text, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or system comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or system comprising that element.

[0096] The serial numbers of the embodiments of the present application above are only for description and do not represent the superiority or inferiority of the embodiments. The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image matching method, characterized in that The method includes: Receiving multiple to-be-matched images acquired by an automatic walking device through collecting an edge area in a target working area, and obtaining an index map corresponding to the edge area; Sorting the multiple to-be-matched images according to the order of the distance information between the to-be-matched images and the index map from high to low to obtain a sorting result; Classifying the multiple to-be-matched images according to the sorting result to obtain a first image set and a second image set, wherein the distance information corresponding to the to-be-matched images in the first image set is greater than the distance information corresponding to the to-be-matched images in the second image set; Sequentially projecting three-dimensional feature points of the index map onto a spatial coordinate system corresponding to the to-be-matched images in the first image set according to the sorting result, and taking the to-be-matched images in the first image set that meet the first projection result as first target images until the total number of the first target images reaches a target number threshold; Establishing a matching image set that matches the edge area according to the first target images.

2. The method according to claim 1, wherein After taking the to-be-matched images in the first image set that meet the target projection result as the first target images, the method further includes: If the number of the first target images is less than the target number threshold, classifying the second image set according to the depth distance to obtain a third image set and a fourth image set, wherein the depth distance of the to-be-matched images in the third image set is within a preset depth distance range, and the depth distance of the to-be-matched images in the fourth image set is outside the preset depth distance range; Sequentially projecting three-dimensional feature points of the index map onto a spatial coordinate system corresponding to the to-be-matched images in the third image set according to the sorting order, and taking the to-be-matched images in the third image set that meet the second projection result as second target images until the total number of the first target images and the second target images reaches the target number threshold; Establishing the matching image set according to the first target images and the second target images.

3. The method according to claim 2, wherein After taking the to-be-matched images in the third image set that meet the second projection result as the second target images, the method further includes: If the total number of the first target images and the second target images is less than the target number threshold; Sequentially projecting three-dimensional feature points of the index map onto a spatial coordinate system corresponding to the to-be-matched images in the fourth image set according to the sorting order, and taking the to-be-matched images in the fourth image set that meet the third projection result as third target images until the total number of the first target images, the second target images, and the third target images reaches the target number threshold; Establishing the matching image set according to the first target images, the second target images, and the third target images.

4. The method according to claim 1, characterized in that Taking the to-be-matched images in the first image set that meet the first projection result as the first target images includes: Determine the number of feature points of the three-dimensional feature points located within the to-be-matched image according to the projection coordinates of the three-dimensional feature points projected onto the spatial coordinate system; When the number of feature points exceeds the feature point number threshold, determine that the to-be-matched image is the first target image.

5. The method according to claim 4, wherein The projecting the three-dimensional feature points of the index map onto the spatial coordinate system corresponding to the to-be-matched image in the first image set includes: Establish the spatial coordinate system according to the acquisition position of the to-be-matched image; Based on the real-world coordinates of the three-dimensional feature points and the spatial coordinate system, determine the projection coordinates corresponding to the three-dimensional feature points in the spatial coordinate system.

6. The method according to claim 5, characterized in that, The determining the number of feature points of the three-dimensional feature points located within the to-be-matched image includes: Determine the coordinate range corresponding to the to-be-matched image in the spatial coordinate system according to the image contour of the to-be-matched image; Count the number of three-dimensional feature points whose corresponding projection coordinates are within the coordinate range as the number of feature points.

7. A map construction method, applied to an automatic walking device, characterized in that, The method further includes: Retrieve a set of matching images corresponding to the edge area in the target working area, the set of matching images being generated by the image matching method according to any one of claims 1-6; Establish an edge map corresponding to the edge area according to the set of matching images.

8. A driving control method applied to an automatic walking device, characterized in that, The method includes: Retrieve an edge map corresponding to the edge area in the target working area, the edge map being generated by the map construction method according to claim 7; Determine the driving pose of the automatic walking device according to the edge map and the image collected by the automatic walking device, and perform driving operations in the target working area according to the driving pose.

9. An automatic walking device, characterized in that, The device includes: A body; A driving module for driving the body to move; A working module disposed on the body and used for performing a preset operation on the position where the automatic walking device is located; An acquisition module for performing image acquisition; A controller connected to the driving module and the acquisition module, and used for executing at least one of the image matching method according to any one of claims 1-6, the map construction method according to claim 7, and the driving control method according to claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements at least one of the image matching method according to any one of claims 1-6, the map construction method according to claim 7, and the driving control method according to claim 8.

Images