A vehicle-based loop detection method and device, and vehicle-mounted terminal

Abstract

The embodiment of the present invention discloses a vehicle-based loop detection method and device, and a vehicle-mounted terminal. The method includes: obtaining first image frames in multiple directions around the vehicle collected by an image acquisition device at a first location; extracting key points in each first image frame according to the pixel value distribution of the pixel points in the image frame, and determining the feature information of each key point in the first image frame; matching each first image frame with multiple image frames corresponding to multiple locations in an image database according to the feature information corresponding to each image frame; when a first matching result that meets a preset location matching condition is obtained, determining a location that points to the same location as the first location from the second location indicated by the first matching result. Applying the solution provided by the embodiment of the present invention can improve the accuracy of loop detection.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a vehicle-based loop detection method and device, and a vehicle-mounted terminal. Background Art

[0002] Loop detection refers to matching the captured images with historical images when a movable object equipped with a camera moves to a certain location. When the match is successful, it is considered that the movable object has moved to the location corresponding to the historical image that successfully matched. For example, the movable object can be a vehicle, robot, etc. Loop detection can associate the location corresponding to the current image with the location corresponding to the historical image.

[0003] However, when the vehicle moves to a certain position again, the angle of the image taken by the camera on the vehicle will change, and when matching the image with the historical image, it may not be matched successfully. In this way, the same position as the current position cannot be detected, and the accuracy of loop detection is not high enough. Summary of the invention

[0004] The present invention provides a vehicle-based loop detection method and device, and a vehicle-mounted terminal to improve the accuracy of loop detection. The specific technical solution is as follows.

[0005] In a first aspect, an embodiment of the present invention discloses a vehicle-based loop detection method, comprising:

[0006] Acquire first image frames in multiple directions around the vehicle acquired by an image acquisition device at a first location;

[0007] Extracting key points in each first image frame according to pixel value distribution of pixel points in the image frame, and determining feature information of each key point in the first image frame;

[0008] According to the feature information corresponding to each image frame, each first image frame is matched with a plurality of image frames corresponding to a plurality of locations in an image database; when there are at least a preset number of groups of image frames that successfully match each first image frame with a plurality of image frames corresponding to a second location in the image database, a first matching result that satisfies a preset location matching condition is determined; wherein the location indicated by the first matching result is the second location; wherein the image database is used to store feature information of image frames at a plurality of first-category locations, and each image frame at a first-category location includes image frames in a plurality of directions around the vehicle at the location;

[0009] A location pointing to the same location as the first location is determined from the second locations.

[0010] Optionally, the image database is further used to store feature information of image frames at multiple second-category locations, each image frame at the second-category location includes an image frame in one direction around the vehicle at the location; after determining the feature information of each key point in the first image frame, the method further includes:

[0011] According to the feature information corresponding to each image frame, each first image frame is matched with an image frame at each location in the image database; when at least one first image frame among the first image frames successfully matches the image frame in the image database, a second matching result satisfying a preset location matching condition is determined; wherein the third location indicated by the second matching result is: the location corresponding to the successfully matched image frame in the image database;

[0012] A location pointing to the same location as the first location is determined from the third locations.

[0013] Optionally, after acquiring each first image frame captured by the image acquisition device at the first location, the method further includes:

[0014] splicing the first image frames to obtain a first panoramic image frame at the first location;

[0015] Extracting key points in the first panoramic image frame according to pixel value distribution of pixel points in the image frame, and determining feature information of each key point in the first panoramic image frame;

[0016] According to the feature information corresponding to each panoramic image frame, the first panoramic image frame is matched with the panoramic image frames corresponding to multiple locations in the image database; when the first panoramic image frame is successfully matched with the panoramic image frames in the image database, a third matching result satisfying a preset location matching condition is determined, and the fourth location indicated by the third matching result is: the location corresponding to the successfully matched panoramic image frame in the image database; wherein the image database is used to store the feature information of the panoramic image frames at multiple locations;

[0017] A location pointing to the same location as the first location is determined from the fourth locations.

[0018] Optionally, when there are multiple second locations, the step of determining a location from the second locations that points to the same location as the first location includes:

[0019] Acquire a fifth location indicated by a matching result determined from the image database at a previous location before the first location;

[0020] Determine whether there is a target image frame that is continuous with the image frame corresponding to the fifth location among the plurality of image frames corresponding to the second location in the image database;

[0021] If it exists, the second location corresponding to the target image frame is determined as a location pointing to the same location as the first location.

[0022] Optionally, the step of matching each first image frame with a plurality of image frames corresponding to a plurality of locations in the image database according to feature information corresponding to each image frame comprises:

[0023] For each first image frame and each image frame in the image database, each feature information of the first image frame is compared with each feature information of the image frame to obtain the degree of overlap of the feature information between the first image frame and the image frame; when the degree of overlap of the feature information is greater than a preset overlap threshold, it is determined that the first image frame matches the image frame successfully.

[0024] Optionally, when it is determined that no matching result satisfying the preset location matching condition can be obtained, the method further includes:

[0025] The feature information of each first image frame at the first location is stored in the image database.

[0026] Optionally, the step of determining feature information of each key point in the first image frame includes:

[0027] For each key point in the first image frame, determine a descriptor corresponding to the key point according to the key point and pixel points at preset positions around the key point;

[0028] The word corresponding to the descriptor of each key point in the first image frame is determined from a preset descriptor dictionary, and the words corresponding to all the key points in the first image frame are used as feature information of the first image frame.

[0029] In a second aspect, an embodiment of the present invention discloses a vehicle-based loop detection device, comprising:

[0030] An acquisition module is configured to acquire first image frames in multiple directions around the vehicle acquired by an image acquisition device at a first location;

[0031] A first extraction module is configured to extract key points in each first image frame according to the pixel value distribution of the pixel points in the image frame, and determine feature information of each key point in the first image frame;

[0032] A first matching module is configured to match each first image frame with a plurality of image frames corresponding to a plurality of locations in an image database according to feature information corresponding to each image frame; when there are at least a preset number of groups of image frames that successfully match each first image frame with a plurality of image frames corresponding to a second location in the image database, determine to obtain a first matching result that satisfies a preset location matching condition; wherein the location indicated by the first matching result is the second location; wherein the image database is used to store feature information of image frames at a plurality of first-category locations, and each image frame at a first-category location includes image frames in a plurality of directions around the vehicle at the location;

[0033] The first determining module is configured to determine a location from the second locations that points to the same location as the first location.

[0034] Optionally, the image database is further used to store feature information of image frames at multiple second-category locations, each image frame at the second-category location includes an image frame in one direction around the vehicle at the location; the device further includes:

[0035] The second matching module is configured to, after determining the feature information of each key point in the first image frame, match each first image frame with an image frame at each location in the image database according to the feature information corresponding to each image frame; when at least one first image frame among the first image frames successfully matches the image frame in the image database, determine to obtain a second matching result that satisfies a preset location matching condition; wherein the third location indicated by the second matching result is: the location corresponding to the successfully matched image frame in the image database;

[0036] The second determining module is configured to determine a location from the third location that points to the same location as the first location.

[0037] Optionally, the device further comprises:

[0038] a stitching module configured to stitch the first image frames after acquiring the first image frames captured by the image acquisition device at the first location to obtain a first panoramic image frame at the first location;

[0039] A second extraction module is configured to extract key points in the first panoramic image frame according to pixel value distribution of pixel points in the image frame, and determine feature information of each key point in the first panoramic image frame;

[0040] a third matching module, configured to match the first panoramic image frame with panoramic image frames corresponding to a plurality of locations in an image database according to feature information corresponding to each panoramic image frame; when the first panoramic image frame successfully matches with a panoramic image frame in the image database, determine to obtain a third matching result that satisfies a preset location matching condition, wherein a fourth location indicated by the third matching result is: a location corresponding to the successfully matched panoramic image frame in the image database; wherein the image database is used to store feature information of panoramic image frames at a plurality of locations;

[0041] The third determining module is configured to determine a location from the fourth location that points to the same location as the first location.

[0042] Optionally, when there are multiple second locations, the first determining module is specifically configured to:

[0043] Acquire a fifth location indicated by a matching result determined from the image database at a previous location before the first location;

[0044] Determine whether there is a target image frame that is continuous with the image frame corresponding to the fifth location among the plurality of image frames corresponding to the second location in the image database;

[0045] If it exists, the second location corresponding to the target image frame is determined as a location pointing to the same location as the first location.

[0046] Optionally, the first matching module is specifically configured as follows:

[0047] For each first image frame and each image frame in the image database, each feature information of the first image frame is compared with each feature information of the image frame to obtain the degree of overlap of the feature information between the first image frame and the image frame; when the degree of overlap of the feature information is greater than a preset overlap threshold, it is determined that the first image frame matches the image frame successfully.

[0048] Optionally, the device further comprises:

[0049] The storage module is configured to store feature information of each first image frame at the first location in the image database when it is determined that a matching result satisfying the preset location matching condition cannot be obtained.

[0050] Optionally, when the first extraction module determines the feature information of each key point in the first image frame, it includes:

[0051] For each key point in the first image frame, determine a descriptor corresponding to the key point according to the key point and pixel points at preset positions around the key point;

[0052] The word corresponding to the descriptor of each key point in the first image frame is determined from a preset descriptor dictionary, and the words corresponding to all the key points in the first image frame are used as feature information of the first image frame.

[0053] In a third aspect, an embodiment of the present invention discloses a vehicle-mounted terminal, including: a processor and an image acquisition device; the processor includes: an acquisition module, a first extraction module, a first matching module and a first determination module;

[0054] The acquisition module is used to acquire first image frames in multiple directions around the vehicle acquired by the image acquisition device at a first location;

[0055] The first extraction module is used to extract key points in each first image frame according to the pixel value distribution of the pixel points in the image frame, and determine the feature information of each key point in the first image frame;

[0056] The first matching module is used to match each first image frame with multiple image frames corresponding to multiple locations in the image database according to the feature information corresponding to each image frame; when there are at least a preset number of groups of image frames that successfully match each first image frame with the multiple image frames corresponding to the second location in the image database, determine to obtain a first matching result that meets the preset location matching condition; wherein the location indicated by the first matching result is the second location; wherein the image database is used to store feature information of image frames at multiple first-category locations, and each image frame at the first-category location includes image frames in multiple directions around the vehicle at the location;

[0057] The first determining module is used to determine a location from the second locations that points to the same location as the first location.

[0058] Optionally, the image database is further used to store feature information of image frames at multiple second-category locations, each image frame at the second-category location includes an image frame in one direction around the vehicle at the location; the processor further includes:

[0059] A second matching module is used for matching each first image frame with an image frame at each location in the image database according to the feature information corresponding to each image frame after determining the feature information of each key point in the first image frame; when at least one first image frame among the first image frames successfully matches the image frame in the image database, determining to obtain a second matching result that satisfies a preset location matching condition; wherein the third location indicated by the second matching result is: the location corresponding to the successfully matched image frame in the image database;

[0060] The second determining module is used to determine a location from the third location that points to the same location as the first location.

[0061] Optionally, the processor further includes:

[0062] a stitching module, configured to stitch the first image frames acquired by the image acquisition device at the first location to obtain a first panoramic image frame at the first location;

[0063] A second extraction module, configured to extract key points in the first panoramic image frame according to pixel value distribution of pixel points in the image frame, and determine feature information of each key point in the first panoramic image frame;

[0064] a third matching module, configured to match the first panoramic image frame with panoramic image frames corresponding to a plurality of locations in an image database according to feature information corresponding to each panoramic image frame; when the first panoramic image frame successfully matches with a panoramic image frame in the image database, determine to obtain a third matching result that satisfies a preset location matching condition, wherein a fourth location indicated by the third matching result is: a location corresponding to the successfully matched panoramic image frame in the image database; wherein the image database is configured to store feature information of panoramic image frames at a plurality of locations;

[0065] The third determining module is used to determine a location from the fourth location that points to the same location as the first location.

[0066] Optionally, when there are multiple second locations, the first determining module is specifically configured to:

[0067] Acquire a fifth location indicated by a matching result determined from the image database at a previous location before the first location;

[0068] Determine whether there is a target image frame that is continuous with the image frame corresponding to the fifth location among the plurality of image frames corresponding to the second location in the image database;

[0069] If it exists, the second location corresponding to the target image frame is determined as a location pointing to the same location as the first location.

[0070] Optionally, the first matching module is specifically used for:

[0071] For each first image frame and each image frame in the image database, each feature information of the first image frame is compared with each feature information of the image frame to obtain the degree of overlap of the feature information between the first image frame and the image frame; when the degree of overlap of the feature information is greater than a preset overlap threshold, it is determined that the first image frame matches the image frame successfully.

[0072] Optionally, the processor further includes:

[0073] The storage module is used to store the feature information of each first image frame at the first location in the image database when it is determined that a matching result that meets the preset location matching condition cannot be obtained.

[0074] Optionally, when the first extraction module determines the feature information of each key point in the first image frame, it includes:

[0075] For each key point in the first image frame, determine a descriptor corresponding to the key point according to the key point and pixel points at preset positions around the key point;

[0076] The word corresponding to the descriptor of each key point in the first image frame is determined from a preset descriptor dictionary, and the words corresponding to all the key points in the first image frame are used as feature information of the first image frame.

[0077] From the above content, it can be seen that the vehicle-based loop detection method and device, and the vehicle-mounted terminal provided by the embodiments of the present invention can obtain the first image frames in multiple directions around the vehicle collected at the first location, and when matching with the image frames in the image database, multiple image frames between every two locations are matched. In the related art, each location corresponds to an image frame, and the matching may fail when the angle of the image frame taken again changes. In the embodiment of the present invention, image frames in multiple directions are collected at each location, and the image frames in multiple directions can reflect more comprehensive feature information at the location. During matching, when there are at least a preset number of groups of successfully matched image frames, it is determined that the matching result that meets the preset location matching conditions is obtained, so that the matching process is more accurate. Therefore, the embodiment of the present invention can improve the accuracy of loop detection.

[0078] The innovative features of the embodiments of the present invention include:

[0079] 1. Each location corresponds to image frames in multiple directions. In the case where each location in the image database contains image frames in various directions, when there are at least a preset number of groups of successfully matched image frames, a matching result that meets the preset location matching conditions is determined. This makes the matching in loop detection not limited by the shooting angle, and the accuracy of matching the same location is higher.

[0080] 2. When each location in the image database contains an image frame, when at least one first image frame successfully matches the image frame in the image database, a matching result that satisfies the preset location matching condition is determined. This makes loop detection compatible with the situation where each location in the database corresponds to one image frame, thereby improving compatibility.

[0081] 3. Splice multiple image frames corresponding to each location into a panoramic image frame, which can reduce the number of matches, shorten the matching process, and improve matching efficiency.

[0082] 4. Judging based on image sequences can improve the accuracy of loop detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work.

[0084] Figure 1 A schematic diagram of a flow chart of a vehicle-based loop detection method provided by an embodiment of the present invention;

[0085] Figure 2 A schematic diagram of a generation process of a descriptor dictionary provided in an embodiment of the present invention;

[0086] Figure 3 A schematic flow chart of another vehicle-based loop detection method provided by an embodiment of the present invention;

[0087] Figure 4 A schematic flow chart of another vehicle-based loop detection method provided by an embodiment of the present invention;

[0088] Figure 5 to Figure 7 Schematic diagram of the structure of a vehicle-based loop detection device in several embodiments of the present invention;

[0089] Figure 8 A schematic structural diagram of a vehicle-mounted terminal provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0090] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0091] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0092] The embodiment of the present invention discloses a vehicle-based loop detection method and device, and a vehicle-mounted terminal, which can improve the accuracy of loop detection. The embodiment of the present invention is described in detail below.

[0093] Figure 1 A flow chart of a vehicle-based loop detection method provided in an embodiment of the present invention. The method is applied to an electronic device. The electronic device may be a common computer, a server, or an intelligent mobile terminal. The electronic device may also be a device installed in a vehicle for computing and processing. The method specifically includes the following steps.

[0094] S110: Acquire first image frames in multiple directions around the vehicle captured by an image capture device at a first location.

[0095] There may be multiple image acquisition devices installed at different positions of the vehicle, and each image acquisition device acquires image frames in one direction around the vehicle. For example, a camera may be installed in the front, rear, left, and right directions of the vehicle, respectively. In this case, four first image frames may be acquired at each location. The image acquisition device may be a normal lens camera or a fisheye lens camera.

[0096] The electronic device may receive a first image frame captured by each image acquisition device when the vehicle is located at a first location. At the same time and at the same location, each image acquisition device acquires a first image frame. The electronic device may associate each first image frame with the first location. The first location may be represented by two-dimensional or three-dimensional coordinates.

[0097] The image acquisition device can acquire image frames at a preset frame rate. When performing loop detection, the electronic device can obtain a key frame (I frame) in the image frame sequence acquired by the image acquisition device as the first image frame. This can reduce the amount of calculation. During the positioning process, the movement of the vehicle is slower than the acquisition speed of the image acquisition device, so there is no need to perform loop detection on each image frame. Selecting the key frame in the image frame for loop detection can reduce the processing burden of the system. This can also maintain the validity of the information.

[0098] When acquiring first image frames in multiple directions around the vehicle acquired by the image acquisition device at the first location, it may specifically include: acquiring initial image frames in multiple directions around the vehicle acquired by the image acquisition device at the first location, converting each initial image frame into a grayscale image, and obtaining each first image frame. The above embodiment is applicable to the case where the image directly acquired by the image acquisition device is a color image.

[0099] S120: extracting key points in each first image frame according to pixel value distribution of pixel points in the image frame, and determining feature information of each key point in the first image frame.

[0100] Among them, key points may include object corners, points with large pixel changes, boundary points, contour points, bright spots in darker areas, and dark spots in brighter areas in the image frame. Each first image frame may contain multiple key points. Specifically, the FAST (Features From Accelerated Segment Test, FAST) algorithm may be used to extract key points in the first image frame. When detecting key points, based on a pixel point, the pixel values ​​of other pixel points around the pixel point may be detected. If there are more than a specified number of other pixel points around the pixel point, and the difference in pixel value between the pixel point and the pixel point is greater than a preset pixel value threshold, the pixel point is determined to be a key point.

[0101] The feature information of each key point may be information including the positional relationship and pixel value relationship between the key point and the pixels around the key point. Each key point has a corresponding feature information, and the feature information of the first image frame includes the feature information of all key points in the first image frame.

[0102] S130: Match each first image frame with a plurality of image frames corresponding to a plurality of locations in an image database according to feature information corresponding to each image frame.

[0103] The image database is used to store feature information of image frames at multiple first-category locations, and each image frame at the first-category location includes image frames in multiple directions around the vehicle at the location.

[0104] When there are at least a preset number of groups of image frames that successfully match each first image frame with a plurality of image frames corresponding to the second location in the image database, it is determined that a first matching result that satisfies a preset location matching condition is obtained. A matching process between a first location and each first-category location in the image database includes a process of pairwise matching between each first image frame corresponding to the first location and each image frame corresponding to the first-category location.

[0105] The location indicated by the first matching result is the second location, and the second location is a location in the first category of locations. The second location can be represented by two-dimensional or three-dimensional coordinates.

[0106] The preset number may be a number pre-determined according to the total number of first image frames included in each first location. For example, when the total number of first image frames is 4, the preset number may be a value of 2 to 4.

[0107] For example, the first location corresponds to 4 first image frames, and each first-category location in the image database corresponds to 4 image frames. During matching, the feature information of each first image frame can be matched with each image frame in each first-category location. In other words, for each first location and each first-category location, 4*4 image matching can be performed for a total of 8 times. When there are more than a preset number of groups of image frames that are successfully matched, it is considered that the match between the first location and the first-category location meets the preset location matching condition, and the first-category location is the second location. When the number of matched image groups is less than the preset number of groups, it is considered that the match between the first location and the first-category location does not meet the preset location matching condition, that is, there is no matching result that meets the preset location matching condition.

[0108] When it is determined that a matching result satisfying the preset location matching condition cannot be obtained, feature information of each first image frame at the first location may be stored in an image database for subsequent loop detection.

[0109] S140: Determine a location from the second locations that points to the same location as the first location.

[0110] When the number of the second locations is one, the second location can be directly determined as the location pointing to the same location as the first location. When the number of the second locations is greater than one, one location can be selected from the second locations as the location pointing to the same location as the first location, or all the second locations can be determined as the locations pointing to the same location as the first location.

[0111] A location that points to the same location as the first location may be understood as a location that may have different coordinates from the first location, but both locations represent the same location.

[0112] After determining a location that points to the same location as the first location, the first location may be corrected according to the determined location.

[0113] In one application scenario, in some special places where there is no Global Positioning System (GPS) signal, the vehicle cannot determine the specific location of the vehicle based on the GPS signal and cannot complete the map construction. For example, in places such as underground garages, it is impossible to rely on GPS signals to obtain the global location information of the vehicle. In this case, the current vehicle posture can only be determined based on odometer data and the previous vehicle posture. The cumulative error of the map constructed using this method is large. In order to reduce the cumulative error, an image database can be constructed to store the image frames captured by the camera during the vehicle's movement for use in loop detection.

[0114] When the vehicle moves to the first location, if a first matching result that meets the preset location matching conditions can be obtained from the image database, the first location can be corrected according to the determined location, and then the vehicle posture at the first location can be corrected to reduce the cumulative error in the positioning process.

[0115] It can be seen from the above content that the present embodiment can obtain the first image frames in multiple directions around the vehicle collected at the first location, and when matching with the image frames in the image database, multiple image frames between every two locations are matched. In the related art, each location corresponds to an image frame, and the change in the angle of the image frame taken again may cause unsuccessful matching. In the present embodiment, image frames in multiple directions are collected at each location, and the image frames in multiple directions can reflect more comprehensive feature information at the location, and when matching, when there are at least a preset number of groups of successfully matched image frames, it is determined that the matching result that meets the preset location matching condition is obtained, so that the matching process is more accurate. Therefore, the present embodiment can improve the accuracy of loop detection.

[0116] In another embodiment of the present invention, based on Figure 1 In the illustrated embodiment, when there are multiple second locations, step S140, the step of determining a location from the second locations that points to the same location as the first location, may specifically include steps 1a to 3a.

[0117] Step 1a: Acquire a fifth location indicated by a matching result determined from the image database at a previous location before the first location.

[0118] In each loop detection, multiple second locations matching the first location can be determined from the image database. In addition, the frame numbers of the image frames matched from the image database at the continuous locations are also continuous. When the image frames use key frames, the continuous frame numbers refer to the continuous key frame numbers. In order to more accurately determine the location pointing to the same location as the first location from the multiple second locations, the multiple second locations can be screened based on the continuous locations, that is, based on the continuous image sequence.

[0119] When the vehicle is in a continuous moving process, the image acquisition device can acquire key frames with continuous frame numbers. At the same location, the frame numbers of the first image frames acquired by the image acquisition devices on the same vehicle are the same.

[0120] The previous location may be one or more locations, that is, matching results at one or more locations before the first location may be obtained.

[0121] Step 2a: Determine whether there is a target image frame continuous with the image frame corresponding to the fifth location among the image frames corresponding to the plurality of second locations in the image database. If so, execute step 3a; if not, do not process it.

[0122] The image frame corresponding to the fifth location refers to an image frame corresponding to a location pointing to the same location in an image database corresponding to the fifth location. The fifth location may be one or more.

[0123] For example, the current location is location 3, and the vehicle moves in the order of location 1→location 2→location 3, and location 1, location 2 and location 3 are three consecutive locations. At location 3, the locations (multiple second locations) determined from the image database that point to the same location as location 3 include: location a—image frame a, location b—image frame b, and location c—image frame c. Relative to location 3, the previous location may include location 1 and location 2. When it is obtained that the vehicle is at location 2, the location determined from the image database that points to the same location as location 2 is location m, and the image frame number corresponding to location m in the image database is m; when the vehicle is at location 1, the location determined from the image database that points to the same location as location 1 is location n, and the image frame number corresponding to location n in the image database is n. After judgment, the image frame corresponding to location 3 (second location) includes image a, image b and image c, where the frame numbers of image a, image frame m and image n are continuous, and image a can be determined as the target image frame.

[0124] When loop detection is performed for each location, if the frame numbers of image frames in the image database corresponding to several consecutive locations are continuous, the location in the loop detection can be determined more accurately.

[0125] Step 3a: Determine the second location corresponding to the target image frame as a location that points to the same location as the first location.

[0126] In summary, this embodiment makes a judgment based on the image sequence to determine the location that points to the same location as the first location. Compared with matching based on a single frame, this embodiment can avoid false matching caused by similar content of single-frame images at different locations, and can improve the accuracy of loop detection.

[0127] In another embodiment of the present invention, based on Figure 1 In the illustrated embodiment, step S130, the step of matching each first image frame with a plurality of image frames corresponding to a plurality of locations in an image database according to feature information corresponding to each image frame, may specifically include:

[0128] For each first image frame and each image frame in the image database, each feature information of the first image frame is compared with each feature information of the image frame to obtain the degree of overlap of the feature information between the first image frame and the image frame.

[0129] When the feature information coincidence is greater than the preset coincidence threshold, it is determined that the first image frame matches the image frame successfully. When the feature information coincidence is not greater than the preset coincidence threshold, it is determined that the first image frame matches the image frame unsuccessfully.

[0130] The preset overlap threshold may be a specific value or a ratio value, and the obtained feature information overlap may be a value or a ratio value.

[0131] For example, by comparing each feature information of the first image frame with each feature information of the image frame, the number of identical feature information can be obtained, and the ratio of the number of identical feature information to the total number of feature information of the first image frame is used as the ratio value.

[0132] For example, a first image frame includes 100 key points, that is, the first image frame includes 100 feature information, and a certain image frame in the image database includes 120 feature information. When comparing feature information, the 100 feature information of the first image frame is compared with the 120 feature information of the image frame, and 90 feature information are the same, then the overlap degree of feature information between the two can be 90 or 90 / 100.

[0133] In summary, this embodiment provides a specific implementation method for matching each first image frame with each image frame in the image database, which can improve the matching efficiency.

[0134] In another embodiment of the present invention, based on Figure 1 In the illustrated embodiment, the step of determining the feature information of each key point in the first image frame in step S120 may specifically include steps 1b and 2b.

[0135] Step 1b: For each key point in the first image frame, determine a descriptor corresponding to the key point based on the key point and pixel points at preset positions around the key point.

[0136] The descriptor is a multi-dimensional feature vector, and the descriptor is used to describe the distribution characteristics of the pixels around the key point. When determining the descriptor corresponding to the key point, N pixel pairs can be selected around the key point P in a certain pattern, and the comparison results of the N pixel pairs can be combined as the descriptor. N is an integer. Specifically, the BRIEF (Binary Robust Independent Elementary Features) algorithm can be used to determine the descriptor corresponding to the key point.

[0137] Step 2b: Determine the word corresponding to the descriptor of each key point in the first image frame from a preset descriptor dictionary, and use the words corresponding to all key points in the first image frame as feature information of the first image frame.

[0138] The descriptor dictionary is used to store the correspondence between descriptors and words. Words are smaller in data volume than descriptors, and converting descriptors into corresponding words can reduce the amount of calculation during matching.

[0139] The descriptor dictionary can be pre-established in the following manner: a large number of scene images in an underground garage are collected, and key points in the multiple scene images are extracted, and the descriptor of each key point is determined based on each key point and the pixel points at a preset position around the key point; after obtaining the descriptors of a large number of key points, the position of each descriptor in the description space can be targeted, and clustering can be performed so that similar descriptors in the description space correspond to the same words.

[0140] See also Figure 2 As shown, after collecting a large number of scene images, a large number of descriptors in these scene images are determined, and a large number of descriptors in the description space are clustered, and the descriptors belonging to the same cluster are set to correspond to the same word, so that a descriptor dictionary containing multiple descriptors belonging to the same cluster corresponding to one word is obtained. Each black dot represents a descriptor.

[0141] In summary, in this embodiment, descriptors of key points are determined, and each descriptor is converted into words. All words in each image frame constitute the feature information of the image frame. The feature information determined in this way can reduce the amount of calculation during feature information matching and improve calculation efficiency.

[0142] In another embodiment of the present invention, based on Figure 1 The embodiment shown can obtain Figure 3 The embodiment shown. Figure 3 In the illustrated embodiment, the image database is further used to store feature information of image frames at a plurality of second-type locations, and each image frame at the second-type location includes image frames in one direction around the vehicle at the location. The method includes the following steps.

[0143] S310: Acquire first image frames in multiple directions around the vehicle captured by an image capture device at a first location.

[0144] S320: extracting key points in each first image frame according to pixel value distribution of pixel points in the image frame, and determining feature information of each key point in the first image frame.

[0145] In this embodiment, the above steps S310 and S320 are respectively Figure 1 Steps S110 and S120 in the embodiment shown are the same, and detailed descriptions can be found in Figure 1 The embodiment shown.

[0146] S330: Match each first image frame with an image frame at each location in the image database according to the feature information corresponding to each image frame. When at least one first image frame among the first image frames successfully matches the image frame in the image database, a second matching result that satisfies the preset location matching condition is determined.

[0147] The third location indicated by the second matching result is: a location corresponding to an image frame that has been successfully matched in the image database.

[0148] In this embodiment, each location in the image database corresponds to an image frame. During matching, for the first location and each location in the image database, each first image frame is matched with the image frame corresponding to the location in the image database. When at least one first image frame successfully matches the image frame, it is considered that the first location and the location in the image database meet the preset location matching condition. The above is a location matching process, and the first location can be matched with all locations in the image database respectively.

[0149] When it is determined that a matching result satisfying the preset location matching condition cannot be obtained, the correspondence between the first location, the first image frame and the feature information may be stored in an image database for use in subsequent matching.

[0150] S340: Determine a location from the third location that points to the same location as the first location.

[0151] The specific implementation of this step can refer to the description of step S140, which will not be repeated here.

[0152] In summary, in this embodiment, when each location in the image database corresponds to an image frame, each first image frame corresponding to the first location can be matched with the image frame corresponding to each location in the image database, and when at least one first image frame successfully matches the image frame in the image database, a second matching result that satisfies the preset location matching condition is determined. Therefore, this embodiment can take into account the situation of single-view image frames in the image database and improve the accuracy of loop detection.

[0153] In another embodiment of the present invention, based on Figure 1 The embodiment shown can obtain Figure 4 The embodiment shown. Figure 4 The method of the illustrated embodiment may include the following steps.

[0154] S410: Acquire first image frames in multiple directions around the vehicle captured by an image capture device at a first location. The specific description of this step can be found in step S110, which will not be repeated here.

[0155] S420: stitching the first image frames together to obtain a first panoramic image frame at the first location.

[0156] When the first image frames are stitched together, the first image frames may be stitched together in a preset position order according to the preset positional relationship of the image acquisition devices to obtain a first panoramic image frame including the surrounding environment at the first location.

[0157] S430: extracting key points in the first panoramic image frame according to pixel value distribution of pixel points in the image frame, and determining feature information of each key point in the first panoramic image frame.

[0158] The specific implementation method of determining the characteristic information of the first panoramic image frame in this step may refer to the implementation method of determining the characteristic information of the first image frame in step S120, which will not be repeated here.

[0159] S440: Matching the first panoramic image frame with panoramic image frames corresponding to a plurality of locations in an image database according to feature information corresponding to each panoramic image frame.

[0160] When the first panoramic image frame successfully matches the panoramic image frame in the image database, a third matching result satisfying the preset location matching condition is determined, and the fourth location indicated by the third matching result is: the location corresponding to the successfully matched panoramic image frame in the image database.

[0161] The image database is used to store feature information of panoramic image frames at multiple locations.

[0162] In this embodiment, each location corresponds to a panoramic image frame. During matching, the first panoramic image frame at the first location is matched with the panoramic image frames at each location in the image database, and when the matching is successful, a third matching result that meets the preset location matching condition is determined.

[0163] S450: Determine a location from the fourth location that points to the same location as the first location. The specific implementation of this step can refer to the description in step S140.

[0164] In summary, in this embodiment, the first image frames at the first location are spliced ​​to obtain a first panoramic image frame, and during matching, the first panoramic image frame is matched with the panoramic image frames at each location in the image database. Splicing the first image frames at the first location can reduce the number of matches between the panoramic image frame and each panoramic image frame in the image database, shorten the matching process, and improve matching efficiency.

[0165] In another embodiment of the present invention, when determining the feature information of each key point in the first panoramic image frame in S430, steps 1c and 2c may be specifically included.

[0166] Step 1c: for each key point in the first panoramic image frame, determine a descriptor corresponding to the key point according to the key point and pixel points at preset positions around the key point.

[0167] Step 2c: Determine the word corresponding to the descriptor of each key point in the first panoramic image frame from a preset descriptor dictionary, and use the words corresponding to all key points in the first panoramic image frame as feature information of the first panoramic image frame.

[0168] For the specific description of steps 1c and 2c in this embodiment, please refer to steps 1b and 2b, which will not be repeated here.

[0169] Figure 5 A vehicle-based loop detection device is provided in an embodiment of the present invention. Figure 1 The method embodiment shown corresponds to the embodiment shown. The device includes:

[0170] An acquisition module 510 is configured to acquire first image frames in multiple directions around the vehicle acquired by an image acquisition device at a first location;

[0171] The first extraction module 520 is configured to extract key points in each first image frame according to the pixel value distribution of the pixel points in the image frame, and determine the feature information of each key point in the first image frame;

[0172] The first matching module 530 is configured to match each first image frame with a plurality of image frames corresponding to a plurality of locations in an image database according to feature information corresponding to each image frame; when there are at least a preset number of groups of image frames that successfully match each first image frame with a plurality of image frames corresponding to a second location in the image database, determine to obtain a first matching result that satisfies a preset location matching condition; wherein the location indicated by the first matching result is the second location; wherein the image database is used to store feature information of image frames at a plurality of first-category locations, and each image frame at a first-category location includes image frames in a plurality of directions around the vehicle at the location;

[0173] The first determining module 540 is configured to determine a location from the second locations that points to the same location as the first location.

[0174] In another embodiment of the present invention, based on Figure 5 In the illustrated embodiment, when there are multiple second locations, the first determining module 540 is specifically configured to:

[0175] Acquire a fifth location indicated by a matching result determined from the image database at a previous location before the first location;

[0176] Determine whether there is a target image frame that is continuous with the image frame corresponding to the fifth location among the plurality of image frames corresponding to the second location in the image database;

[0177] If it exists, the second location corresponding to the target image frame is determined as a location that points to the same location as the first location.

[0178] In another embodiment of the present invention, based on Figure 5 In the illustrated embodiment, the first matching module 530 is specifically configured as follows:

[0179] For each first image frame and each image frame in the image database, each feature information of the first image frame is compared with each feature information of the image frame to obtain the degree of overlap of the feature information between the first image frame and the image frame; when the degree of overlap of the feature information is greater than a preset overlap threshold, it is determined that the first image frame matches the image frame successfully.

[0180] In another embodiment of the present invention, based on Figure 5 In the embodiment shown, the device may further include:

[0181] The storage module (not shown in the figure) is configured to store the feature information of each first image frame at the first location in the image database when it is determined that a matching result that meets the preset location matching condition cannot be obtained.

[0182] In another embodiment of the present invention, based on Figure 5 In the illustrated embodiment, the first extraction module 520, when determining the feature information of each key point in the first image frame, includes:

[0183] For each key point in the first image frame, determine a descriptor corresponding to the key point according to the key point and pixel points at preset positions around the key point;

[0184] The word corresponding to the descriptor of each key point in the first image frame is determined from a preset descriptor dictionary, and the words corresponding to all the key points in the first image frame are used as feature information of the first image frame.

[0185] In another embodiment of the present invention, based on Figure 5 The embodiment shown can obtain Figure 6 In this embodiment, the image database is also used to store feature information of image frames at multiple second-type locations, and each image frame at the second-type location includes an image frame in one direction around the vehicle at the location. Figure 3 The device includes: an acquisition module 610, a first extraction module 620, a second matching module 630 and a second determination module 640. The acquisition module 610 and the first extraction module 620 are respectively Figure 5 The acquisition module 510 and the first extraction module 520 in the illustrated embodiment are the same, and the specific descriptions are omitted.

[0186] The second matching module 630 is configured to, after determining the feature information of each key point in the first image frame, match each first image frame with an image frame at each location in the image database according to the feature information corresponding to each image frame; when at least one first image frame among the first image frames successfully matches an image frame in the image database, determine to obtain a second matching result that satisfies a preset location matching condition; wherein the third location indicated by the second matching result is: the location corresponding to the successfully matched image frame in the image database;

[0187] The second determining module 640 is configured to determine a location from the third location that points to the same location as the first location.

[0188] In another embodiment of the present invention, based on Figure 5 The embodiment shown can obtain Figure 7 The device embodiment shown in FIG. Figure 4 The device further includes: an acquisition module 710, a splicing module 720, a second extraction module 730, a third matching module 740 and a third determination module 750. The acquisition module 710 and the Figure 5The acquisition module 510 in the illustrated embodiment is the same, and the specific description is not repeated here.

[0189] The stitching module 720 is configured to stitch the first image frames acquired by the image acquisition device at the first location to obtain a first panoramic image frame at the first location.

[0190] A second extraction module 730 is configured to extract key points in the first panoramic image frame according to the pixel value distribution of the pixel points in the image frame, and determine feature information of each key point in the first panoramic image frame;

[0191] The third matching module 740 is configured to match the first panoramic image frame with panoramic image frames corresponding to multiple locations in an image database according to feature information corresponding to each panoramic image frame; when the first panoramic image frame successfully matches the panoramic image frame in the image database, determine to obtain a third matching result that satisfies a preset location matching condition, and the fourth location indicated by the third matching result is: the location corresponding to the successfully matched panoramic image frame in the image database; wherein the image database is used to store feature information of panoramic image frames at multiple locations;

[0192] The third determining module 750 is configured to determine a location from the fourth location that points to the same location as the first location.

[0193] The above device embodiment corresponds to the method embodiment and has the same technical effect as the method embodiment. For specific description, please refer to the method embodiment. The device embodiment is obtained based on the method embodiment. For specific description, please refer to the method embodiment part, which will not be repeated here.

[0194] Figure 8 The schematic diagram of the structure of a vehicle-mounted terminal provided by an embodiment of the present invention includes: a processor 810 and an image acquisition device 820. The processor 810 includes: an acquisition module 11, a first extraction module 12, a first matching module 13 and a first determination module 14.

[0195] An acquisition module 11 is used to acquire first image frames in multiple directions around the vehicle acquired by the image acquisition device 820 at a first location;

[0196] A first extraction module 12, used to extract key points in each first image frame according to the pixel value distribution of the pixel points in the image frame, and determine the feature information of each key point in the first image frame;

[0197] The first matching module 13 is used to match each first image frame with multiple image frames corresponding to multiple locations in the image database according to the feature information corresponding to each image frame; when there are at least a preset number of image frames that successfully match each first image frame with the multiple image frames corresponding to the second location in the image database, determine to obtain a first matching result that meets the preset location matching condition; wherein the location indicated by the first matching result is the second location; wherein the image database is used to store the feature information of the image frames at multiple first-category locations, and each image frame at the first-category location includes image frames in multiple directions around the vehicle at the location;

[0198] The first determining module 14 is configured to determine a location from the second locations that points to the same location as the first location.

[0199] In another embodiment of the present invention, based on Figure 8 In the illustrated embodiment, the image database is further used to store feature information of image frames at a plurality of second-type locations, each image frame at the second-type location includes an image frame in one direction around the vehicle at the location; the processor 810 further includes:

[0200] A second matching module (not shown in the figure) is used to match each first image frame with an image frame at each location in the image database according to the feature information corresponding to each image frame after determining the feature information of each key point in the first image frame; when at least one first image frame among the first image frames successfully matches the image frame in the image database, determine to obtain a second matching result that satisfies a preset location matching condition; wherein the third location indicated by the second matching result is: the location corresponding to the successfully matched image frame in the image database;

[0201] The second determining module (not shown in the figure) is used to determine a location from the third location that points to the same location as the first location.

[0202] In another embodiment of the present invention, based on Figure 8 In the illustrated embodiment, the processor 810 further includes:

[0203] A stitching module (not shown in the figure) is used to stitch the first image frames acquired by the image acquisition device at the first location to obtain a first panoramic image frame at the first location;

[0204] A second extraction module (not shown in the figure), used to extract key points in the first panoramic image frame according to the pixel value distribution of the pixel points in the image frame, and determine the feature information of each key point in the first panoramic image frame;

[0205] a third matching module (not shown in the figure), for matching the first panoramic image frame with panoramic image frames corresponding to a plurality of locations in an image database according to feature information corresponding to each panoramic image frame; when the first panoramic image frame matches the panoramic image frame in the image database successfully, determining to obtain a third matching result that satisfies a preset location matching condition, and the fourth location indicated by the third matching result is: the location corresponding to the successfully matched panoramic image frame in the image database; wherein the image database is used to store feature information of the panoramic image frames at the plurality of locations;

[0206] The third determining module (not shown in the figure) is used to determine a location from the fourth locations that points to the same location as the first location.

[0207] In another embodiment of the present invention, based on Figure 8 In the illustrated embodiment, when there are multiple second locations, the first determining module 14 is specifically configured to:

[0208] Acquire a fifth location indicated by a matching result determined from the image database at a previous location before the first location;

[0209] Determine whether there is a target image frame that is continuous with the image frame corresponding to the fifth location among a plurality of image frames corresponding to the second location in the image database;

[0210] If it exists, the second location corresponding to the target image frame is determined as a location that points to the same location as the first location.

[0211] In another embodiment of the present invention, based on Figure 8 In the illustrated embodiment, the first matching module 13 is specifically used for:

[0212] For each first image frame and each image frame in the image database, each feature information of the first image frame is compared with each feature information of the image frame to obtain the degree of overlap of the feature information between the first image frame and the image frame; when the degree of overlap of the feature information is greater than a preset overlap threshold, it is determined that the first image frame matches the image frame successfully.

[0213] In another embodiment of the present invention, based on Figure 8 In the illustrated embodiment, the processor 810 further includes:

[0214] A storage module (not shown in the figure) is used to store the feature information of each first image frame at the first location in the image database when it is determined that a matching result that meets the preset location matching condition cannot be obtained.

[0215] In another embodiment of the present invention, based on Figure 8In the illustrated embodiment, the first extraction module 12, when determining the feature information of each key point in the first image frame, includes:

[0216] For each key point in the first image frame, determine a descriptor corresponding to the key point according to the key point and pixel points at preset positions around the key point;

[0217] The word corresponding to the descriptor of each key point in the first image frame is determined from a preset descriptor dictionary, and the words corresponding to all the key points in the first image frame are used as feature information of the first image frame.

[0218] The terminal embodiment and Figure 1 The method embodiments shown are embodiments based on the same inventive concept, and relevant parts can be referenced to each other. The above terminal embodiments correspond to the method embodiments, and have the same technical effects as the method embodiments. For specific descriptions, refer to the method embodiments.

[0219] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0220] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle-based loop detection method, characterized in that: include: Acquire first image frames in multiple directions around the vehicle captured by an image acquisition device at a first location, wherein each of the image acquisition devices acquires one first image frame at the same time and location; According to the pixel value distribution of the pixel points in the image frame, the key points in each first image frame are extracted, and the feature information of each key point in the first image frame is determined. The step of determining the feature information of each key point in the first image frame includes: for each key point in the first image frame, according to the key point and the pixel points at the preset position around the key point, determining the descriptor corresponding to the key point; determining the word corresponding to the descriptor of each key point in the first image frame from the preset descriptor dictionary, and taking the words corresponding to all the key points in the first image frame as the feature information of the first image frame, wherein the method for establishing the descriptor dictionary includes: collecting multiple scene images in the underground garage, and extracting the key points in the multiple scene images, according to each key point and the pixel points at the preset position around it, determining the descriptor of each key point, after obtaining the descriptor of each key point, according to the position of each descriptor in the description space, clustering so that the similar descriptors in the description space correspond to the same word; According to the feature information corresponding to each image frame, each first image frame is matched with a plurality of image frames corresponding to a plurality of locations in an image database; when there are at least a preset number of groups of image frames that successfully match each first image frame with a plurality of image frames corresponding to a second location in the image database, a first matching result that satisfies a preset location matching condition is determined; wherein the location indicated by the first matching result is the second location; wherein the image database is used to store feature information of image frames at a plurality of first-category locations, and each image frame at a first-category location includes image frames in a plurality of directions around the vehicle at the location; A location pointing to the same location as the first location is determined from the second locations.

2. The method according to claim 1, characterized in that The image database is further used to store feature information of image frames at a plurality of second-category locations, each image frame at the second-category location including image frames in one direction around the vehicle at the location; After determining the feature information of each key point in the first image frame, the method further includes: According to the feature information corresponding to each image frame, each first image frame is matched with an image frame at each location in the image database; when at least one first image frame among the first image frames successfully matches the image frame in the image database, a second matching result satisfying a preset location matching condition is determined; wherein the third location indicated by the second matching result is: the location corresponding to the successfully matched image frame in the image database; A location pointing to the same location as the first location is determined from the third locations.

3. The method according to any one of claims 1 to 2, characterized in that: After acquiring each first image frame captured by the image acquisition device at the first location, the method further includes: splicing the first image frames to obtain a first panoramic image frame at the first location; Extracting key points in the first panoramic image frame according to pixel value distribution of pixel points in the image frame, and determining feature information of each key point in the first panoramic image frame; According to the feature information corresponding to each panoramic image frame, the first panoramic image frame is matched with the panoramic image frames corresponding to multiple locations in the image database; when the first panoramic image frame is successfully matched with the panoramic image frames in the image database, a third matching result satisfying a preset location matching condition is determined, and the fourth location indicated by the third matching result is: the location corresponding to the successfully matched panoramic image frame in the image database; wherein the image database is used to store the feature information of the panoramic image frames at multiple locations; A location pointing to the same location as the first location is determined from the fourth locations.

4. The method according to any one of claims 1 to 3, characterized in that: When there are multiple second locations, the step of determining a location from the second locations that points to the same location as the first location includes: Acquire a fifth location indicated by a matching result determined from the image database at a previous location before the first location; Determine whether there is a target image frame that is continuous with the image frame corresponding to the fifth location among the plurality of image frames corresponding to the second location in the image database; If it exists, the second location corresponding to the target image frame is determined as a location pointing to the same location as the first location.

5. The method according to claim 1, characterized in that The step of matching each first image frame with a plurality of image frames corresponding to a plurality of locations in the image database according to the feature information corresponding to each image frame comprises: For each first image frame and each image frame in the image database, each feature information of the first image frame is compared with each feature information of the image frame to obtain the degree of overlap of the feature information between the first image frame and the image frame; when the degree of overlap of the feature information is greater than a preset overlap threshold, it is determined that the first image frame matches the image frame successfully.

6. The method according to claim 1, characterized in that When it is determined that no matching result satisfying the preset location matching condition can be obtained, the method further includes: The feature information of each first image frame at the first location is stored in the image database.

7. A vehicle-based loop detection device, characterized in that: include: An acquisition module is configured to acquire first image frames in multiple directions around the vehicle acquired by an image acquisition device at a first location, wherein each of the image acquisition devices acquires one first image frame at the same time and location; The first extraction module is configured to extract key points in each first image frame according to the pixel value distribution of the pixel points in the image frame, and determine the feature information of each key point in the first image frame, wherein the step of determining the feature information of each key point in the first image frame comprises: for each key point in the first image frame, determining the descriptor corresponding to the key point according to the key point and the pixel points at the preset position around the key point; determining the word corresponding to the descriptor of each key point in the first image frame from the preset descriptor dictionary, and taking the words corresponding to all the key points in the first image frame as the feature information of the first image frame, wherein the method for establishing the descriptor dictionary comprises: collecting multiple scene images in the underground garage, and extracting key points in the multiple scene images, determining the descriptor of each key point according to each key point and the pixel points at the preset position around the key point, and after obtaining the descriptor of each key point, clustering the position of each descriptor in the description space so that the similar descriptors in the description space correspond to the same word; A first matching module is configured to match each first image frame with a plurality of image frames corresponding to a plurality of locations in an image database according to feature information corresponding to each image frame; when there are at least a preset number of groups of image frames that successfully match each first image frame with a plurality of image frames corresponding to a second location in the image database, determine to obtain a first matching result that satisfies a preset location matching condition; wherein the location indicated by the first matching result is the second location; wherein the image database is used to store feature information of image frames at a plurality of first-category locations, and each image frame at a first-category location includes image frames in a plurality of directions around the vehicle at the location; The first determining module is configured to determine a location from the second locations that points to the same location as the first location.

8. The device according to claim 7, characterized in that The device also includes: a stitching module configured to stitch the first image frames after acquiring the first image frames captured by the image acquisition device at the first location to obtain a first panoramic image frame at the first location; A second extraction module is configured to extract key points in the first panoramic image frame according to pixel value distribution of pixel points in the image frame, and determine feature information of each key point in the first panoramic image frame; a third matching module, configured to match the first panoramic image frame with panoramic image frames corresponding to a plurality of locations in an image database according to feature information corresponding to each panoramic image frame; when the first panoramic image frame successfully matches with a panoramic image frame in the image database, determine to obtain a third matching result that satisfies a preset location matching condition, wherein a fourth location indicated by the third matching result is: a location corresponding to the successfully matched panoramic image frame in the image database; wherein the image database is used to store feature information of panoramic image frames at a plurality of locations; The third determination module is configured to determine a location from the fourth location that points to the same location as the first location.

9. A vehicle-mounted terminal, characterized in that: include: processor and image acquisition device; The processor includes: an acquisition module, a first extraction module, a first matching module and a first determination module; The acquisition module is used to acquire first image frames in multiple directions around the vehicle acquired by the image acquisition device at a first location, and each of the image acquisition devices acquires one first image frame at the same time and location; The first extraction module is used to extract key points in each first image frame according to the pixel value distribution of the pixel points in the image frame, and determine the feature information of each key point in the first image frame. The step of determining the feature information of each key point in the first image frame includes: for each key point in the first image frame, determining the descriptor corresponding to the key point according to the key point and the pixel points at the preset position around the key point; determining the word corresponding to the descriptor of each key point in the first image frame from a preset descriptor dictionary, and taking the words corresponding to all key points in the first image frame as the feature information of the first image frame, wherein the method for establishing the descriptor dictionary includes: collecting multiple scene images in the underground garage, and extracting key points in the multiple scene images, determining the descriptor of each key point according to each key point and the pixel points at the preset position around it, and after obtaining the descriptor of each key point, clustering the position of each descriptor in the description space so that the similar descriptors in the description space correspond to the same word; The first matching module is used to match each first image frame with multiple image frames corresponding to multiple locations in the image database according to the feature information corresponding to each image frame; when there are at least a preset number of groups of image frames that successfully match each first image frame with the multiple image frames corresponding to the second location in the image database, determine to obtain a first matching result that meets the preset location matching condition; wherein the location indicated by the first matching result is the second location; wherein the image database is used to store feature information of image frames at multiple first-category locations, and each image frame at the first-category location includes image frames in multiple directions around the vehicle at the location; The first determining module is used to determine a location from the second locations that points to the same location as the first location.

Images