Image matching method, map construction method, driving control method, device and medium

Abstract

The present application relates to the field of equipment control, and provides an image matching method, a map construction method, a driving control method, an automatic walking device, and a computer-readable storage medium. The method includes: receiving a plurality of images to be matched and an index map acquired by the automatic walking device from the edge area; sorting the plurality of images to be matched to obtain a sorting result; classifying the plurality of images to be matched according to the sorting result to obtain a first image set and a second image set; projecting the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the images to be matched in the first image set, and using the images to be matched in the first image set that meet the first projection result as the first target image, until the total number of first target images reaches the target number threshold; and establishing a matching image set that matches the edge area based on the first target image. The present application aims to accurately determine the images that match the edge area to improve the speed and quality of map construction.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image matching method, a map construction method, a driving control method, an automatic driving device, and a computer-readable storage medium. Background Art

[0002] To reduce the labor intensity and cost of lawn maintenance, autonomous vehicles, such as lawn mowers, have been developed for cutting and mowing lawns. Currently, some lawn mowers require image acquisition data to construct a map of the surrounding environment of the autonomous vehicle while it is operating on the lawn.

[0003] However, existing composition schemes require a large amount of image data for composition, and it is difficult to determine images that match the edges of the lawn field from a large number of collected images, resulting in poor speed and quality of map construction. Summary of the Invention

[0004] The main purpose of this application is to provide an image matching method, a map construction method, a driving control method, an automatic driving device and a computer-readable storage medium, which aim to accurately determine images that match edge areas to improve the speed and quality of map construction.

[0005] In a first aspect, the present application provides an image matching method, the image matching method comprising the following steps:

[0006] Receiving a plurality of images to be matched obtained by the autonomous vehicle from capturing edge areas in a target working area, and obtaining an index map corresponding to the edge areas;

[0007] Sorting the plurality of images to be matched according to the distance information between the images to be matched and the index map from high to low to obtain a sorting result;

[0008] Classifying the plurality of images to be matched according to the sorting result to obtain a first image set and a second image set, wherein distance information corresponding to the images to be matched in the first image set is greater than distance information corresponding to the images to be matched in the second image set;

[0009] Projecting the three-dimensional feature points of the index map to the spatial coordinate systems corresponding to the images to be matched in the first image set in sequence according to the sorting result, and using the images to be matched in the first image set that meet the first projection result as first target images, until the total number of the first target images reaches a target number threshold;

[0010] A matching image set matching the edge area is established according to the first target image.

[0011] In a second aspect, the present application provides a map construction method, which is applied to an autonomous vehicle. The map construction method comprises the following steps:

[0012] Retrieving a matching image set corresponding to an edge area in the target working area, wherein the matching image set is generated by the image matching method described above;

[0013] An edge map corresponding to the edge area is established according to the matching image set.

[0014] In a third aspect, the present application provides a driving control method for an autonomous vehicle, the driving control method comprising the following steps:

[0015] Retrieving an edge map corresponding to an edge area in the target working area, wherein the edge map is generated by the map construction method described above;

[0016] The driving posture of the autonomous driving device is determined according to the edge map and the image collected by the autonomous driving device, and a driving operation is performed in a target working area according to the driving posture.

[0017] In a fourth aspect, the present application further provides an automatic walking device, the automatic walking device comprising:

[0018] ontology;

[0019] A driving module, used for driving the main body to travel;

[0020] A working module is provided on the main body and is used to perform a grass cutting operation at the location where the automatic walking device is located;

[0021] An acquisition module is at least used to acquire images of the environment in which the autonomous vehicle is located and to acquire the position and posture of the autonomous vehicle;

[0022] A controller is connected to the driving module and the acquisition module, and is used to execute at least one of the above-mentioned image matching method, the above-mentioned map construction method, and the above-mentioned driving control method.

[0023] In a fifth aspect, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements at least one of the image matching method described above, the map construction method described above, and the driving control method described above.

[0024] The present application provides an image matching method, a map construction method, a driving control method, an automatic driving device, and a computer-readable storage medium. The present application receives a plurality of images to be matched acquired by the automatic driving device from an edge area of ​​a target working area, and acquires an index map corresponding to the edge area; sorts the plurality of images to be matched according to the distance information between the images to be matched and the index map from high to low to obtain a sorting result; classifies the plurality of images to be matched according to the sorting result to obtain a first image set and a second image set; projects the three-dimensional feature points of the index map into the spatial coordinate system corresponding to the images to be matched in the first image set according to the sorting result, and uses the images to be matched in the first image set that meet the first projection result as the first target image until the total number of first target images reaches a target number threshold; and establishes a matching image set that matches the edge area according to the first target image. In this way, images that match the edge area can be quickly and accurately obtained from a large number of acquired images. The image matching has low computational complexity, short time consumption, and high stability, which is conducive to accurately constructing the edge map, thereby reducing the amount of data for map construction, reducing the complexity of map construction, and improving the speed and quality of map construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0026] Figure 1 A flowchart of an image matching method provided in one embodiment of the present application;

[0027] Figure 2 A schematic diagram of the motion trajectory of an autonomous walking device provided in one embodiment of the present application;

[0028] Figure 3 A schematic diagram of the motion trajectory of an autonomous walking device provided in another embodiment of the present application;

[0029] Figure 4 Schematic diagram of an image to be matched obtained by an autonomous vehicle during rotational motion according to an embodiment of the present application;

[0030] Figure 5 is a schematic diagram of an image to be matched obtained by an autonomous vehicle during forward motion provided by another embodiment of the present application;

[0031] Figure 6 1 is a schematic diagram of an image to be matched obtained by an autonomous walking device during crab walking motion provided by another embodiment of the present application;

[0032] Figure 7 This is a schematic diagram of the depth distance between the acquisition position of the image to be matched and the acquisition position of the index map provided in an embodiment of the present application;

[0033] Figure 8 A flowchart of a map construction method provided in one embodiment of the present application;

[0034] Figure 9 A schematic flow chart of a driving control method provided in one embodiment of the present application;

[0035] Figure 10 A schematic structural diagram of an automatic walking device provided in one embodiment of the present application;

[0036] Figure 11 This is a schematic block diagram of the structure of an automatic walking device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0038] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0039] Embodiments of the present application provide an image matching method, a map construction method, a driving control method, an automatic driving device, and a computer-readable storage medium.

[0040] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0041] Please refer to Figure 1 , Figure 1A flow chart of an image matching method provided for an embodiment of the present application. The image matching method can be applied to an automatic walking device, a terminal or a server to realize a set of matching images that are quickly and accurately matched with the edge area from a large number of collected images, thereby facilitating the subsequent construction of an edge map corresponding to the edge area. Among them, the automatic walking device can be a robot such as a lawn mower robot or a sweeping robot; the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this; the server can be an independent server, a server cluster, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms.

[0042] like Figure 1 As shown, the image matching method includes steps S101 to S105.

[0043] Step S101: receiving a plurality of images to be matched obtained by an autonomous vehicle from capturing edge areas in a target working area, and obtaining an index map corresponding to the edge areas.

[0044] Among them, the target working area can be the area where the automatic walking device works. Generally, the automatic walking device will move within the target working area. The edge area is a related area located at the edge position of the target working area, such as the area near the boundary of the target working area. The image to be matched is the image of the edge area collected by the automatic walking device at different collection positions in the target working area, and the index map corresponding to the edge area is the image of the edge area collected by the automatic walking device at a specific collection position in the target working area, and the map constructed using these images. The specific position generally refers to the collection position corresponding to the standard image of the edge area.

[0045] For example, unlike the related art in which the automatic walking device indiscriminately acquires images throughout the entire target working area and uses all acquired images to construct a map, in the map construction method provided by the technical solution of the present application, only the edge images acquired by the automatic walking device at the edge of the target working area are used to construct an edge map. Since the edge position is a position where the automatic walking device is prone to deviation during movement, the edge map obtained by constructing the edge image has a smaller amount of data while being able to effectively correct the deviation of the automatic walking device. Therefore, in order to accurately construct an edge map, it is necessary to quickly and accurately obtain images that match the edge area (such as the edge of a lawn field) from a large number of collected images.

[0046] Exemplarily, the target working area has at least two different edge positions, and a corresponding edge area is formed on a side close to each edge position, and multiple images to be matched are acquired when the autonomous vehicle is working in the target working area.

[0047] Taking a lawn mower as an example, when the mower is operating on a lawn, the lawn has a certain operating boundary. Therefore, there are operating boundaries in multiple directions to form the target working area. Therefore, during the operation of the mower, the edge area of ​​the lawn can be collected to obtain multiple images to be matched.

[0048] like Figure 2 As shown, for example, the autonomous vehicle can reciprocate along a bow-shaped path in the target working area, so that the motion trajectory covers the target working area as much as possible. In an embodiment of the present application, the position where the autonomous vehicle turns around can be used as the edge position, and the area formed by the positions before and after the autonomous vehicle turns around can be used as the edge area.

[0049] like Figure 3 As shown in the figure, the actual motion trajectory of the autonomous walking device may be offset. For example, due to the accumulated errors generated during the movement of the autonomous walking device, it is usually difficult to ensure that the motion trajectory is completely parallel, which may easily lead to different angles of different paths, resulting in some locations in the target working area not being covered. Taking the lawn mowing robot as an example, Figure 3 The black area in the middle is the area missed by the mowing operation.

[0050] In order to avoid the above situation, the embodiment of the present application constructs an edge map for the edge area before the automatic walking device turns around in the target working area, that is, the position where the automatic walking device turns around is regarded as the edge position, and the area formed by the positions before and after the automatic walking device turns around is regarded as the edge area. Figure 3 The autonomous walking device shown continuously generates and accumulates errors during movement, resulting in increasingly larger errors.

[0051] like Figure 4-Figure 6 As shown, the triangles in the figure represent image data, and the circles in the figure represent three-dimensional feature points in the target working area.

[0052] like Figure 4 As shown, the autonomous vehicle's viewing angle changes significantly during rotation, causing the projected positions of feature points in different image data to vary significantly. For large rotation angles, feature points may even fall outside the image's field of view. Therefore, the shared viewing capability of Image Data 1, Image Data 2, and Image Data 3 acquired by the autonomous vehicle during rotation is poor.

[0053] like Figure 5 As shown in the figure, the common viewing ability of the image data 1, image data 2, and image data 3 obtained by the autonomous walking device in the forward motion is better than that in the rotation motion. However, in the forward motion, the autonomous walking device moves in a straight line along a certain direction, and the parallax change is mainly reflected in the depth direction. As the autonomous walking device moves forward, the feature points with a smaller shooting angle (such as Figure 5 The position of the feature point 2 in the image changes little, while the feature points with smaller shooting angles (such as Figure 5 Feature points 1 and 3 in the image data may quickly move out of view. In this case, the number of common feature points between key frames may be small, and the common viewing ability of image data 1, image data 2, and image data 3 is poor.

[0054] like Figure 6 As shown, the image data 1, image data 2, and image data 3 acquired by the autonomous vehicle during crab-like motion have strong common view capabilities. During crab-like motion, the autonomous vehicle primarily moves horizontally, and parallax changes primarily occur in the horizontal direction. This motion pattern causes large variations in the horizontal position of 3D feature points within the image, but makes it easier to maintain the 3D feature points within the field of view. Therefore, the number of common view feature points between key frames during crab-like motion is typically large, facilitating the establishment of richer common view relationships, resulting in strong common view capabilities for image data 1, image data 2, and image data 3.

[0055] Therefore, the autonomous vehicle can reciprocate along a bow-shaped path within the target work area. This is equivalent to capturing multiple matching images captured during a crab-like motion, which exhibits a good common view relationship. This allows for effective adjustment of the autonomous vehicle's trajectory and reduces cumulative error. It should be noted that common view refers to the ability of two or more images in a map to observe the same map point. These images with a common view relationship can complement each other's information, optimizing map accuracy.

[0056] Step S102 : Sort the multiple images to be matched according to the distance information between the images to be matched and the index map from high to low to obtain a sorting result.

[0057] The distance information is the Euclidean distance between the acquisition location of the image to be matched and the acquisition location of the index map. Since the acquisition location of the index map is fixed, while the acquisition location of the image to be matched is different, the images to be matched can be sorted from largest to smallest based on the Euclidean distance between the acquisition location of each image to be matched and the acquisition location of the index map, thereby obtaining the corresponding sorting result.

[0058] Since the coordinate systems corresponding to the image to be matched and the index map are different, the Euclidean distance between the acquisition position of the image to be matched and the acquisition position of the index map cannot be directly calculated. Therefore, before calculating the Euclidean distance, the coordinate system corresponding to the image to be matched can be converted into the coordinate system corresponding to the index map, or the coordinate system corresponding to the index map can be converted into the coordinate system corresponding to the image to be matched, that is, the Euclidean distance can be calculated after converting them to the same coordinate system.

[0059] For example, a coordinate system can be constructed with the index map as a reference, and then the device posture corresponding to each image to be matched can be mapped to the coordinate system corresponding to the index map. In this way, the device posture corresponding to each image to be matched and the device posture corresponding to the index map can be located in the same coordinate system, and the Euclidean distance between the acquisition position of the image to be matched and the acquisition position of the index map can be calculated through the device posture corresponding to each image to be matched and the device posture corresponding to the index map.

[0060] Finally, after calculating the Euclidean distance between the acquisition position of each image to be matched and the acquisition position of the index map, the Euclidean distances corresponding to each image to be matched are sorted from large to small to obtain the corresponding sorting result.

[0061] Step S103: classify the multiple images to be matched according to the sorting result to obtain a first image set and a second image set, wherein the distance information corresponding to the images to be matched in the first image set is greater than the distance information corresponding to the images to be matched in the second image set.

[0062] Since the greater the Euclidean distance between the acquisition location of the image to be matched and the acquisition location of the index map, the better the image triangulation quality, the quality of the image to be matched can be determined by the Euclidean distance. Since the distance information corresponding to the image to be matched in the first image set is greater than the distance information corresponding to the image to be matched in the second image set, the quality of the image to be matched in the first image set is better than that of the image to be matched in the second image set.

[0063] Exemplarily, the first n images of the sorting results may be screened out to form a first image set, and the remaining images to be matched may be screened out to form a second image set, where n may be any integer value.

[0064] Exemplarily, it is also possible to determine whether the Euclidean distance corresponding to each image to be matched is greater than a preset distance threshold, and to filter out images to be matched that are greater than or equal to the preset distance threshold to form a first image set, and to filter out images to be matched that are less than the preset distance threshold to form a second image set.

[0065] Step S104: Project the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the images to be matched in the first image set in sequence according to the sorting results, and use the images to be matched in the first image set that meet the first projection results as the first target images until the total number of first target images reaches the target number threshold.

[0066] The three-dimensional feature point can be any map point in the index map. The target number threshold can be any value and is not specifically limited here. The first target image is the image to be matched in the first image set that meets the first projection result. The projection result corresponding to the first projection result is: the number of successfully matched three-dimensional feature points in the image to be matched in the first image set exceeds the corresponding feature point number threshold.

[0067] In some embodiments, the number of three-dimensional feature points within the image to be matched is determined based on the projected coordinates of the three-dimensional feature points projected onto the spatial coordinate system. When the number of feature points exceeds a threshold, the image to be matched is determined to be the first target image. This allows the first target image to be accurately selected from the images to be matched.

[0068] In related technologies, pixel matching is generally used to determine the first target image, that is, traversing every pixel point of the image to be matched to perform matching. However, this matching method requires a large amount of data, resulting in a long time consumption and low efficiency in determining the first target image. In the embodiment of the present application, there is no need to determine the first target image through pixel matching. The first target image can be directly determined by the number of projections of three-dimensional feature points. Not only can the first target image be accurately screened from the image to be matched, but the image matching also has low computational complexity, short time consumption and high stability, ensuring the real-time nature of map construction and ensuring that the autonomous walking device can operate normally without stopping and waiting.

[0069] Exemplarily, the three-dimensional feature points can be projected into the spatial coordinate system corresponding to the image to be matched in the first image set based on the device posture and camera internal parameters corresponding to the index map, thereby obtaining the projection coordinates of the three-dimensional feature points in the spatial coordinate system; according to the projection coordinates and the image coordinates of the image to be matched, it can be determined whether each three-dimensional feature point is located within the image to be matched, and then the number of feature points of the three-dimensional feature points located within the image to be matched can be determined; when the number of feature points exceeds the feature point number threshold, the image to be matched is determined to be the first target image, and so on, each image to be matched in the first image set can be traversed, thereby determining all first target images from the first image set.

[0070] For example, according to the sorting results of the images to be matched in the first image set, based on the device posture and camera intrinsic parameters corresponding to the index map, the three-dimensional feature points of the index map can be projected into the spatial coordinate system of the corresponding images to be matched in the first image set in sequence, so that the images to be matched with good quality in the first image set can be traversed preferentially.

[0071] In some embodiments, a spatial coordinate system is established according to the acquisition position of the image to be matched; and based on the real coordinates of the three-dimensional feature point and the spatial coordinate system, the projection coordinates corresponding to the three-dimensional feature point in the spatial coordinate system are determined.

[0072] Since the coordinate systems corresponding to the image to be matched and the three-dimensional feature points are different, it is impossible to directly determine whether the three-dimensional feature points are located within the image to be matched. Therefore, before determining whether the three-dimensional feature points are located within the image to be matched, the three-dimensional feature points can be projected into the coordinate system corresponding to the image to be matched (i.e., the spatial coordinate system).

[0073] For example, a spatial coordinate system can be established according to the acquisition position of each image to be matched; based on the coordinate system conversion relationship, the real coordinates of the three-dimensional feature points are converted to the spatial coordinate system, thereby determining the projection coordinates corresponding to the three-dimensional feature points in the spatial coordinate system.

[0074] In some embodiments, a coordinate range corresponding to the image to be matched is determined in a spatial coordinate system according to an image contour of the image to be matched; and the number of three-dimensional feature points whose corresponding projection coordinates are within the coordinate range is counted as the number of feature points.

[0075] For example, since the image to be matched generally has a certain size, the coordinate range corresponding to the image to be matched in the spatial coordinate system can be determined based on the image contour of the image to be matched, such as a rectangular range, a circular range, etc. These ranges occupy at least multiple coordinates, and then the number of three-dimensional feature points whose projection coordinates corresponding to the three-dimensional feature points are within the coordinate range is counted, and this number of three-dimensional feature points is used as the number of feature points.

[0076] It should be noted that if the projection coordinates corresponding to the three-dimensional feature point are located on the boundary of the coordinate range, the projection coordinates can be considered to be within the coordinate range, or the projection coordinates can be considered to be outside the coordinate range, which is not specifically limited here.

[0077] Step S105 : establishing a matching image set that matches the edge area according to the first target image.

[0078] If the total number of the first target images reaches the target number threshold, a matching image set matching the edge area may be established based on the first target images, so as to establish an edge map corresponding to the edge area based on the matching image set.

[0079] After traversing all the images to be matched in the first image set, all the images to be matched in the first image set that meet the first projection result are obtained as first target images, and the number of first target images is counted. If the total number of first target images reaches the target number threshold, it means that a sufficient number of images with the best quality have been found for accurate subsequent map construction. If the total number of first target images does not reach the target number threshold, it means that more images with better quality need to be further found for subsequent map construction.

[0080] In some embodiments, if the number of first target images is less than a target number threshold, the second image set is classified according to depth distance to obtain a third image set and a fourth image set, wherein the depth distance of the images to be matched in the third image set is within a preset depth distance range, and the depth distance of the images to be matched in the fourth image set is outside the preset depth distance range; the three-dimensional feature points of the index map are sequentially projected onto the spatial coordinate system corresponding to the images to be matched in the third image set according to the sorting order, and the images to be matched in the third image set that meet the second projection result are used as the second target images until the total number of first target images and second target images reaches the target number threshold; and a matching image set is established based on the first target images and the second target images. In this way, when the number of first target images is insufficient, the images to be matched can be further screened to find images of better quality for subsequent map construction.

[0081] Among them, the second target image is the image to be matched in the third image set that meets the second projection result, and the projection result corresponding to the second projection result is: the number of successfully matched three-dimensional feature points in the image to be matched in the third image set exceeds the corresponding feature point number threshold.

[0082] like Figure 7 As shown, for the acquisition position of the index map, the range of x meters in the depth direction is its best depth matching area, where the value of x can be any value and can be determined according to the actual situation. The best depth matching area can also be divided according to the acquisition position of the image to be matched in the first image set. Figure 7 In the figure, the first image set includes image data 1, image data 2, image data 3, image data 4, and image data 5, which are located within the optimal depth matching area, and the second image set includes the remaining image data, which are located outside the optimal depth matching area.

[0083] like Figure 7As shown, for example, before classifying the second image set, the depth distance of the image to be matched in the second image set is calculated first. The straight-line distance between the image to be matched and the boundary of the best depth matching area can be calculated through the position of the image to be matched and the boundary of the best depth matching area. This straight-line distance is the depth distance of the image to be matched.

[0084] Exemplarily, first determine whether the depth distance of each image to be matched in the second image set is within the preset depth distance range. If there are images to be matched whose depth distance is within the preset depth distance range, then filter out these images to be matched to form a third image set. If there are images to be matched whose depth distance is outside the preset depth distance range, then filter out these images to be matched to form a fourth image set.

[0085] The preset depth distance range may be any depth distance range and is not specifically limited here.

[0086] For example, the three-dimensional feature points of the index map can be projected into the spatial coordinate system of the corresponding images to be matched in the second image set according to the sorting order of the images to be matched in the second image set, so that the images to be matched with good quality in the second image set can be traversed preferentially.

[0087] Exemplarily, the three-dimensional feature points of the index map are projected to the spatial coordinate system corresponding to the image to be matched in the third image set, and the number of feature points of the three-dimensional feature points located within the image to be matched is determined based on the projection coordinates of the three-dimensional feature points projected to the spatial coordinate system; when the number of feature points exceeds a feature point number threshold, the image to be matched is determined to be the second target image; until the total number of the first target image and the second target image reaches the target number threshold; a matching image set is established based on the first target image and the second target image.

[0088] If the total number of the first target images and the second target images still does not reach the target number threshold, it means that more images with relatively good quality need to be found for subsequent map construction.

[0089] In some embodiments, if the total number of first target images and second target images is less than the target number threshold; the three-dimensional feature points of the index map are projected to the spatial coordinate system corresponding to the image to be matched in the fourth image set according to the sorting order, and the image to be matched in the fourth image set that meets the third projection result is used as the third target image until the total number of the first target image, the second target image and the third target image reaches the target number threshold; a matching image set is established based on the first target image, the second target image and the third target image.

[0090] Among them, the third target image is the image to be matched in the fourth image set that meets the third projection result, and the projection result corresponding to the third projection result is: the number of successfully matched three-dimensional feature points in the image to be matched in the fourth image set exceeds the corresponding feature point number threshold.

[0091] It should be noted that the threshold values ​​of the number of feature points corresponding to the first projection result, the second projection result, and the third projection result may be the same or different.

[0092] It should be noted that the steps of projecting the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the image to be matched in the fourth image set according to the sorting order, and using the image to be matched in the fourth image set that meets the third projection result as the third target image are similar to the steps of projecting the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the image to be matched in the first image set according to the sorting result, and using the image to be matched in the first image set that meets the first projection result as the first target image, and the steps of projecting the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the image to be matched in the third image set according to the sorting order, and using the image to be matched in the third image set that meets the second projection result as the second target image. Please refer to the relevant embodiments above and will not be repeated here.

[0093] Please refer to Figure 8 , Figure 8 A flowchart of a map construction method provided in an embodiment of the present application is provided. The map construction method is applied to an autonomous vehicle and includes steps S201 to S202.

[0094] S201, retrieving a set of matching images corresponding to edge areas in a target working area;

[0095] S202: Establish an edge map corresponding to the edge area according to the matching image set.

[0096] The set of matching images obtained by the image matching method provided in the embodiment of the present application is conducive to accurately constructing an edge map, reducing the amount of data for map construction, reducing the complexity of map construction, and improving the speed and quality of map construction.

[0097] Please refer to Figure 9 , Figure 9 The present invention provides a flow chart of a driving control method according to an embodiment of the present invention. The driving control method is applied to an autonomous vehicle and includes steps S301 to S302.

[0098] S301, retrieve an edge map corresponding to an edge area in a target working area;

[0099] S302: Determine the driving posture of the autonomous vehicle according to the edge map and the image collected by the autonomous vehicle, and perform driving operations in the target working area according to the driving posture.

[0100] The edge map constructed by the map construction method provided in the embodiment of the present application is conducive to accurately determining the driving posture of the automatic walking device, so that the automatic walking device can accurately perform driving operations in the target working area.

[0101] like Figure 10 As shown, the embodiment of the present application further provides an autonomous vehicle 100, which includes a main body 10, a driving module 20, a working module 30, a collection module (not shown), and a controller (not shown). The driving module 20 is used to drive the main body 10; the working module 30 is disposed on the main body 10 and is used to perform preset operations on the location of the autonomous vehicle 100; the collection module is used to at least capture images of the environment in which the autonomous vehicle 100 is located and to collect the position and posture of the autonomous vehicle 100; and the controller is connected to the driving module 20 and the collection module.

[0102] For example, the automatic walking device 100 provided in the embodiment of the present application can be used to perform lawn mowing operations during movement, but of course it is not limited to this. The automatic walking device 100 provided in the embodiment of the present application can also perform cleaning, snow sweeping, leaf blowing and other operations, which are not limited here.

[0103] Taking the autonomous robot 100 as a lawn mower, for example, the working module 30 is the cutting mechanism, located at the bottom of the main body and used to cut the material to be cut. This includes, but is not limited to, lawns, gardens, and grass on paths. The autonomous robot can cut grass on lawns to ensure its aesthetic appeal. The driving module 20, located on the main body 10, is used to drive the main body 10 forward, enabling the cutting mechanism to cut the grass along a predetermined trajectory. This significantly reduces manual labor, saves time and effort, and truly frees people from the tedious tasks of lawn maintenance.

[0104] See also Figure 11 , Figure 11 This is a schematic block diagram of the structure of an automatic walking device 100 provided in an embodiment of the present application. Figure 11 In the embodiment, the autonomous driving device 100 includes a processor 200 and a memory 300 , wherein the processor 200 and the memory 300 are connected via a bus, which may be any applicable bus such as an I2C (Inter-integrated Circuit) bus.

[0105] The memory 300 may include a storage medium and an internal memory. The storage medium may store an operating system and a computer program. The computer program may include program instructions that, when executed, cause the processor to perform at least one of the image matching method, map construction method, and driving control method described in any embodiment.

[0106] The processor 200 is used to provide computing and control capabilities to support the operation of the entire autonomous vehicle 100 .

[0107] The processor 200 may be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or any conventional processor.

[0108] The processor 200 is configured to run the computer program stored in the memory 300 and implement the following steps when executing the computer program:

[0109] Receiving a plurality of images to be matched obtained by the autonomous vehicle from capturing edge areas in a target working area, and obtaining an index map corresponding to the edge areas;

[0110] Sorting the plurality of images to be matched according to the distance information between the images to be matched and the index map from high to low to obtain a sorting result;

[0111] Classifying the plurality of images to be matched according to the sorting result to obtain a first image set and a second image set, wherein distance information corresponding to the images to be matched in the first image set is greater than distance information corresponding to the images to be matched in the second image set;

[0112] Projecting the three-dimensional feature points of the index map to the spatial coordinate systems corresponding to the images to be matched in the first image set in sequence according to the sorting result, and using the images to be matched in the first image set that meet the first projection result as first target images, until the total number of the first target images reaches a target number threshold;

[0113] A matching image set matching the edge area is established according to the first target image.

[0114] In some embodiments, after taking the image to be matched in the first image set that meets the target projection result as the first target image, the processor 200 is configured to implement:

[0115] If the number of the first target images is less than the target number threshold, the second image set is classified according to the depth distance to obtain a third image set and a fourth image set, wherein the depth distance of the image to be matched in the third image set is within a preset depth distance range, and the depth distance of the image to be matched in the fourth image set is outside the preset depth distance range; the three-dimensional feature points of the index map are projected to the spatial coordinate system corresponding to the image to be matched in the third image set in sequence according to the sorting order, and the image to be matched in the third image set that meets the second projection result is used as the second target image until the total number of the first target images and the second target images reaches the target number threshold; the matching image set is established based on the first target image and the second target image.

[0116] In some embodiments, after taking the image to be matched in the third image set that meets the second projection result as the second target image, the processor 200 is configured to implement:

[0117] If the total number of the first target images and the second target images is less than the target number threshold; project the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the image to be matched in the fourth image set in sequence according to the sorting order, and use the image to be matched in the fourth image set that meets the third projection result as the third target image, until the total number of the first target images, the second target images and the third target images reaches the target number threshold; establish the matching image set based on the first target image, the second target image and the third target image.

[0118] In some embodiments, when the processor 200 uses the image to be matched in the first image set that meets the first projection result as the first target image, it is configured to implement:

[0119] According to the projection coordinates of the three-dimensional feature points projected onto the spatial coordinate system, the number of feature points of the three-dimensional feature points located within the image to be matched is determined; when the number of feature points exceeds a feature point number threshold, the image to be matched is determined to be the first target image.

[0120] In some embodiments, when projecting the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the image to be matched in the first image set, the processor 200 is configured to implement:

[0121] The spatial coordinate system is established according to the acquisition position of the image to be matched; and the projection coordinates corresponding to the three-dimensional feature points in the spatial coordinate system are determined based on the real coordinates of the three-dimensional feature points and the spatial coordinate system.

[0122] In some embodiments, when determining the number of feature points of the three-dimensional feature points located within the image to be matched, the processor 200 is configured to implement:

[0123] The coordinate range corresponding to the image to be matched is determined in the spatial coordinate system according to the image contour of the image to be matched; and the number of three-dimensional feature points whose corresponding projection coordinates are within the coordinate range is counted as the number of feature points.

[0124] When executing the computer program, the processor 200 is further configured to implement the following steps:

[0125] Retrieving a set of matching images corresponding to edge areas in the target working area;

[0126] An edge map corresponding to the edge area is established according to the matching image set.

[0127] When executing the computer program, the processor 200 is further configured to implement the following steps:

[0128] Retrieving an edge map corresponding to an edge area in the target working area;

[0129] The driving posture of the autonomous driving device is determined according to the edge map and the image collected by the autonomous driving device, and a driving operation is performed in a target working area according to the driving posture.

[0130] The present application also provides a computer-readable storage medium that stores a computer program. The computer program includes program instructions. A processor executes the program instructions to implement at least one of the image matching methods, map construction methods, and driving control methods provided in the present application. For example, the computer program, when loaded by the processor, may execute the following steps:

[0131] Receive multiple images to be matched that are captured by the automatic walking device from the edge area of ​​the target working area, and obtain an index map corresponding to the edge area; sort the multiple images to be matched in descending order according to the distance information between the images to be matched and the index map to obtain a sorting result; classify the multiple images to be matched according to the sorting result to obtain a first image set and a second image set, wherein the distance information corresponding to the images to be matched in the first image set is greater than the distance information corresponding to the images to be matched in the second image set; project the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the images to be matched in the first image set in sequence according to the sorting result, and use the images to be matched in the first image set that meet the first projection result as first target images until the total number of the first target images reaches a target number threshold; establish a matching image set that matches the edge area according to the first target image.

[0132] The computer program is loaded by the processor and can further perform the following steps:

[0133] A matching image set corresponding to an edge area in a target working area is retrieved; and an edge map corresponding to the edge area is established according to the matching image set.

[0134] The computer program is loaded by the processor and can further perform the following steps:

[0135] Retrieve an edge map corresponding to an edge area in a target working area; determine a driving posture of the automatic walking device based on the edge map and an image collected by the automatic walking device, and perform a driving operation in the target working area based on the driving posture.

[0136] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0137] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0138] It should also be understood that the term "and / or" used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or system. In the absence of further limitations, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0139] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments. The above description is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An image matching method, characterized in that: The method comprises: Receiving a plurality of images to be matched obtained by the autonomous vehicle from capturing edge areas in a target working area, and obtaining an index map corresponding to the edge areas; Sorting the plurality of images to be matched according to the distance information between the images to be matched and the index map from high to low to obtain a sorting result; Classifying the plurality of images to be matched according to the sorting result to obtain a first image set and a second image set, wherein distance information corresponding to the images to be matched in the first image set is greater than distance information corresponding to the images to be matched in the second image set; Projecting the three-dimensional feature points of the index map to the spatial coordinate systems corresponding to the images to be matched in the first image set in sequence according to the sorting result, and using the images to be matched in the first image set that meet the first projection result as first target images, until the total number of the first target images reaches a target number threshold; A matching image set matching the edge area is established according to the first target image.

2. The method according to claim 1, wherein After taking the image to be matched in the first image set that meets the target projection result as the first target image, the method further includes: If the number of the first target images is less than the target number threshold, classifying the second image set according to the depth distance to obtain a third image set and a fourth image set, wherein the depth distance of the to-be-matched images in the third image set is within a preset depth distance range, and the depth distance of the to-be-matched images in the fourth image set is outside the preset depth distance range; Projecting the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the to-be-matched images in the third image set in sequence according to the sorting order, and using the to-be-matched images in the third image set that meet the second projection result as second target images, until the total number of the first target images and the second target images reaches the target number threshold; The matching image set is established according to the first target image and the second target image.

3. The method according to claim 2, wherein After taking the image to be matched in the third image set that meets the second projection result as the second target image, the method further includes: If the sum of the number of the first target image and the second target image is less than the target number threshold; sequentially projecting the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the image to be matched in the fourth image set according to the sorting order, and using the image to be matched in the fourth image set that meets the third projection result as the third target image, until the total number of the first target image, the second target image, and the third target image reaches the target number threshold; The matching image set is established according to the first target image, the second target image, and the third target image.

4. The method according to claim 1, wherein The taking the image to be matched in the first image set that meets the first projection result as the first target image includes: determining the number of feature points of the three-dimensional feature points located within the image to be matched according to the projection coordinates of the three-dimensional feature points projected onto the spatial coordinate system; When the number of feature points exceeds a feature point number threshold, the image to be matched is determined to be the first target image.

5. The method according to claim 4, wherein The projecting of the three-dimensional feature points of the index map to the spatial coordinate system corresponding to the image to be matched in the first image set includes: Establishing the spatial coordinate system according to the acquisition position of the image to be matched; Based on the real coordinates of the three-dimensional feature point and the spatial coordinate system, a projection coordinate corresponding to the three-dimensional feature point in the spatial coordinate system is determined.

6. The method according to claim 5, wherein The determining the number of feature points of the three-dimensional feature points located within the image to be matched includes: determining a coordinate range corresponding to the image to be matched in the spatial coordinate system according to the image contour of the image to be matched; The number of three-dimensional feature points whose corresponding projection coordinates are within the coordinate range is counted as the number of feature points.

7. A map construction method, applied to an autonomous vehicle, characterized in that: The method further comprises: Retrieving a matching image set corresponding to an edge area in a target working area, wherein the matching image set is generated by the image matching method according to any one of claims 1 to 6; An edge map corresponding to the edge area is established according to the matching image set.

8. A driving control method, applied to an automatic walking device, characterized in that: The method comprises: Retrieving an edge map corresponding to an edge area in the target working area, wherein the edge map is generated by the map construction method according to claim 7; The driving posture of the autonomous driving device is determined according to the edge map and the image collected by the autonomous driving device, and a driving operation is performed in a target working area according to the driving posture.

9. An automatic walking device, characterized in that: The device comprises: ontology; A driving module, used for driving the main body to move; A working module is provided on the main body and is used to perform a preset operation on the position where the automatic walking device is located; Acquisition module, used for image acquisition; A controller, connected to the driving module and the acquisition module, is used to execute at least one of the image matching method according to any one of claims 1 to 6, the map construction method according to claim 7, and the driving control method according to claim 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements at least one of the image matching method according to any one of claims 1 to 6, the map construction method according to claim 7, and the driving control method according to claim 8.

Images