Image localization methods, devices, equipment and storage media

Abstract

This application discloses an image localization method, apparatus, device, and storage medium. The method includes: acquiring consecutive frame images including multiple first images to be localized, each first image corresponding to reference position information; inputting the first images to be localized into a target recognition model to obtain a second image to be localized with a corresponding target label; extracting features from the second image to be localized to obtain localization feature information; performing image matching processing on adjacent images in the second image to be localized based on the localization feature information to obtain multiple matching feature points; determining a target matching feature point from the multiple matching feature points according to the target label; and determining the localization result based on the target matching feature point and the reference position information. This application obtains the localization result by combining only the matching feature points and the reference position information of the capturing device, which effectively improves the efficiency of obtaining image localization results compared to image localization schemes based on a large amount of prior information.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of image processing technology, and in particular to an image positioning method, apparatus, device, and storage medium. Background Technology

[0002] Existing image localization methods for aerial images mainly include the following approaches: target localization methods based on coordinate transformation and target localization methods based on image matching. However, ground target localization methods based on coordinate transformation rely on and are limited by the measurement accuracy of camera angle information such as pitch angle, yaw angle, and roll angle, as well as the stability of the flight platform, resulting in low accuracy of the localization results. Compared with target localization methods based on coordinate transformation, target localization methods based on image matching improve the accuracy of image localization. However, image matching requires the prior acquisition of a large number of reference images, such as satellite images, and the acquisition of their prior information as a data basis, resulting in a large computational load for obtaining image localization results and affecting the efficiency of obtaining localization results. Summary of the Invention

[0003] This application provides an image localization method, apparatus, device, and storage medium, which can effectively improve the efficiency of obtaining image localization results.

[0004] In a first aspect, embodiments of this application provide an image localization method, including:

[0005] Acquire continuous frame images, which represent ground images acquired by the UAV during flight. The continuous frame images include multiple first images to be located, each of which corresponds to reference position information. The reference position information represents the real-time position information of the shooting device used to acquire each of the first images to be located.

[0006] Each of the first images to be located is input into a pre-trained target recognition model for image recognition processing to obtain each of the second images to be located, and each of the second images to be located has a corresponding target label.

[0007] Feature extraction is performed on each of the second images to be located to obtain the feature information of each image to be located;

[0008] Based on the feature information to be located, image matching processing is performed on adjacent images in the second image to be located according to a preset image matching algorithm to obtain multiple matching feature points;

[0009] The target matching feature point is determined from the plurality of matching feature points based on the target label;

[0010] The location result of the target object corresponding to the target label is determined based on the target matching feature points and the reference location information.

[0011] In some embodiments, the feature information to be located includes image feature descriptors, and the step of extracting features from each of the second images to be located to obtain the feature information to be located includes:

[0012] Based on the preset corner detection FAST algorithm, image feature points are extracted from each of the second images to be located;

[0013] Each image feature descriptor is generated based on the image feature points and the preset BRIEF algorithm.

[0014] In some embodiments, the plurality of second images to be located includes at least a first image and a second image, wherein the first image is a neighboring frame image of the second image, the second image corresponds to a matching feature point that successfully matches the feature point of the first image, and the target label includes bounding box location information. The step of determining the target matching feature point from the plurality of matching feature points based on the target label includes:

[0015] The coordinates of each center point are calculated based on the position information of each recognition box in the first image. The coordinates of each center point represent the coordinates of the center point of the recognition box corresponding to each target label in the first image.

[0016] According to a preset nearest neighbor search algorithm, the target matching feature point is determined from the matching feature points of the second image. The target matching feature point represents the feature point in the matching feature points of the second image that is the nearest neighbor to each of the center point coordinates.

[0017] In some embodiments, the number of target matching feature points is at least two, and determining the location result of the target object corresponding to the target label based on the target matching feature points and the reference location information includes:

[0018] Determine the pixel coordinates of each of the target matching feature points, and determine the average value of all the pixel coordinates as the target pixel difference;

[0019] Obtain the target geographic distance difference, which is obtained based on the reference location information of the first image and the reference location information of the second image;

[0020] Calculate the ratio of the target geographical distance difference to the target pixel difference to obtain a pixel scale;

[0021] Determine the image coordinate system of the second image to be located, and calculate the rotation angle between the preset reference coordinate system and the image coordinate system;

[0022] The positioning result of the target object corresponding to the target label is determined based on the pixel scale and the rotation angle.

[0023] In some embodiments, the number of target matching feature points is at least four, and determining the location result of the target object corresponding to the target label based on the target matching feature points and the reference location information includes:

[0024] When the shooting angle of the shooting device does not meet the preset shooting angle conditions, an affine transformation matrix is ​​calculated based on at least four target matching feature points;

[0025] The positioning result of the target object corresponding to the target label is determined based on the affine transformation matrix, the pixel scale, and the rotation angle.

[0026] In some embodiments, after performing image matching processing on adjacent images in the second image to be located based on the feature information to be located and obtaining multiple matching feature points according to a preset image matching algorithm, the method further includes:

[0027] According to the preset Random Sampling Consensus (RANSAC) algorithm, outliers are removed from multiple matching feature points to obtain matching feature points after removing outliers.

[0028] In some embodiments, the target recognition model is the Yolov5 model.

[0029] Secondly, embodiments of this application provide an image positioning device, comprising:

[0030] A continuous frame image acquisition module is used to acquire continuous frame images, which represent ground images acquired by the UAV during flight. The continuous frame images include multiple first images to be located, each of which corresponds to reference position information. The reference position information represents the real-time position information of the shooting device used to acquire each of the first images to be located.

[0031] The image recognition module is used to input each of the first images to be located into a pre-trained target recognition model for image recognition processing to obtain each of the second images to be located, and each of the second images to be located has a corresponding target label;

[0032] The feature extraction module is used to extract features from each of the second images to be located, and obtain the feature information of each image to be located.

[0033] The image matching module is used to perform image matching processing on adjacent images in the second image to be located based on the feature information to be located and according to a preset image matching algorithm to obtain multiple matching feature points;

[0034] The target matching feature point acquisition module is used to determine the target matching feature point from the plurality of matching feature points based on the target label;

[0035] The positioning result determination module is used to determine the positioning result of the target object corresponding to the target label based on the target matching feature points and the reference position information.

[0036] Thirdly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image positioning method as described in the first aspect.

[0037] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for performing the image localization method as described in the first aspect.

[0038] This application provides an image localization method, apparatus, device, and storage medium. The method includes: acquiring consecutive frame images, wherein the consecutive frame images represent ground images acquired by a UAV during flight, the consecutive frame images include multiple first images to be localized, each first image to be localized corresponding to reference position information, the reference position information representing the real-time position information of the capturing device used to acquire each first image to be localized; inputting each first image to be localized into a pre-trained target recognition model for image recognition processing to obtain each second image to be localized, each second image to be localized corresponding to a target label; performing feature extraction on each second image to be localized to obtain each feature information to be localized; based on the feature information to be localized, performing image matching processing on adjacent images in the second images to be localized according to a preset image matching algorithm to obtain multiple matching feature points; determining a target matching feature point from the multiple matching feature points according to the target label; and determining the localization result of the target object corresponding to the target label according to the target matching feature point and the reference position information. This application performs image matching on real-time acquired continuous frame images, and obtains the positioning result by combining the matching feature points with the reference position information of the shooting device. Compared with image positioning schemes that rely on a large amount of prior information for image matching, this effectively improves the efficiency of obtaining image positioning results. Attached Figure Description

[0039] Figure 1 This is a flowchart of the steps of an image localization method provided in one embodiment of this application;

[0040] Figure 2 This is a flowchart of the steps for obtaining the feature information to be located, provided in another embodiment of this application;

[0041] Figure 3 This is a flowchart of the steps for determining target matching feature points provided in another embodiment of this application;

[0042] Figure 4 This is a flowchart of the steps for determining the positioning result provided in another embodiment of this application;

[0043] Figure 5 This is a flowchart of the steps for determining the positioning result provided in another embodiment of this application;

[0044] Figure 6 This is a flowchart of the steps for performing image matching processing on adjacent images in the second image to be located, provided in another embodiment of this application;

[0045] Figure 7 This is a flowchart of the steps of an image positioning device provided in another embodiment of this application;

[0046] Figure 8 This is a structural diagram of an electronic device provided in another embodiment of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] It is understandable that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, or the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0049] This application provides an image localization method, apparatus, device, and storage medium. The method includes: acquiring consecutive frame images, wherein the consecutive frame images represent ground images acquired by a UAV during flight, the consecutive frame images include multiple first images to be localized, each first image to be localized corresponding to reference position information, the reference position information representing the real-time position information of the capturing device used to acquire each first image to be localized; inputting each first image to be localized into a pre-trained target recognition model for image recognition processing to obtain each second image to be localized, each second image to be localized corresponding to a target label; performing feature extraction on each second image to be localized to obtain each feature information to be localized; based on the feature information to be localized, performing image matching processing on adjacent images in the second images to be localized according to a preset image matching algorithm to obtain multiple matching feature points; determining a target matching feature point from the multiple matching feature points according to the target label; and determining the localization result of the target object corresponding to the target label according to the target matching feature point and the reference position information. This application performs image matching on real-time acquired continuous frame images, and obtains the positioning result by combining the matching feature points with the reference position information of the shooting device. Compared with image positioning schemes that rely on a large amount of prior information for image matching, this effectively improves the efficiency of obtaining image positioning results.

[0050] The embodiments of this application will be further described below with reference to the accompanying drawings.

[0051] like Figure 1 As shown, Figure 1 This is a flowchart illustrating the steps of an image localization method provided in one embodiment of this application. This application provides an image localization method, which includes, but is not limited to, the following steps:

[0052] Step S110: Acquire continuous frame images. The continuous frame images represent ground images acquired by the UAV during flight. The continuous frame images include multiple first images to be located. Each first image to be located corresponds to reference position information. The reference position information represents the real-time position information of the shooting device used to acquire each first image to be located.

[0053] Step S120: Input each first image to be located into a pre-trained target recognition model for image recognition processing to obtain each second image to be located, and each second image to be located has a corresponding target label.

[0054] Step S130: Extract features from each of the second images to be located to obtain the feature information of each image to be located;

[0055] Step S140: Based on the feature information to be located, perform image matching processing on adjacent images in the second image to be located according to a preset image matching algorithm to obtain multiple matching feature points;

[0056] Step S150: Determine the target matching feature point from multiple matching feature points based on the target label;

[0057] Step S160: Determine the location result of the target object corresponding to the target label based on the target matching feature points and reference position information.

[0058] It should be noted that the embodiments of this application do not limit the specific method of acquiring continuous frame images. It can be that before the control drone takes off, the first frame of ground image is taken by an aerial flight platform used to place the drone. After acquiring the first frame of ground image, the control drone takes off from the aerial flight platform according to a preset flight route and takes ground images in real time during the flight. The continuous image frames of the embodiments of this application include the first frame of ground image taken by the aerial flight platform and the ground image taken by the drone. Alternatively, it can be that the continuous image frames are acquired by taking real-time pictures only by the drone.

[0059] It should be noted that the drone or aerial platform in this application embodiment can acquire continuous frame images by carrying different types of shooting devices such as remote sensing equipment or image sensors. No restrictions are imposed here. Furthermore, this application embodiment does not limit the shooting angle of the shooting device. The shooting angle in this embodiment can be controlled by keeping the pitch angle of the shooting device perpendicular to the horizontal plane downward. Those skilled in the art can adjust it according to actual needs.

[0060] It should be noted that the embodiments of this application do not limit the specific target recognition model. The target recognition model in this embodiment can be the Yolov5 model, or it can be the Mask R-CNN model or the SSD model.

[0061] It should be noted that any type of drone or aerial flight platform can be used in this embodiment, as long as sufficient endurance is ensured to acquire continuous frame images. This embodiment does not modify the specific hardware structure of the drone or aerial flight platform; furthermore, this embodiment does not restrict the flight route of the drone, and those skilled in the art can make adjustments according to actual needs.

[0062] It should be noted that the embodiments of this application do not limit the storage and retrieval method of intermediate data generated during image localization based on continuous frame images. It can be that each first image to be located and the reference position information corresponding to each first image to be located in the continuous frame images are stored in queue Q1 in chronological order. After each first image to be located is read from queue Q1 and processed by the target recognition model, each second image to be located and each target label are stored in queue Q2 in chronological order. Those skilled in the art can determine the method according to actual needs.

[0063] It is understood that, in this embodiment of the application, continuous frame images are acquired in real time. These continuous frame images represent ground images acquired by the UAV during flight. Each continuous frame image includes multiple first images to be located, and each first image to be located corresponds to reference position information. The reference position information represents the real-time position information of the capturing device used to acquire each first image to be located (i.e., the position information corresponding to the current first image to be located when the aerial flight platform or UAV acquires the current first image to be located). Each first image to be located is input into a pre-trained target recognition model for image recognition processing to obtain each second image to be located. Each second image to be located corresponds to a target label. The target object corresponding to the target label can be a ground target detected by the target recognition model, such as a pedestrian or a vehicle. Feature extraction is performed on each second image to be located to obtain each feature information to be located. Based on the feature information to be located, image matching processing is performed on adjacent images in the second images to be located according to a preset image matching algorithm to obtain multiple matching feature points. The target matching feature point is determined from the multiple matching feature points according to the target label. The positioning result of the target object corresponding to the target label is determined according to the target matching feature point and the reference position information. This application performs image matching on continuously acquired frames in real time, and obtains the localization result by combining the matching feature points with the reference position information of the shooting device. Compared with image localization schemes that rely on a large amount of prior information for image matching, this reduces the cost of acquiring a large dataset and improves the efficiency of acquiring image localization results while ensuring image localization accuracy.

[0064] Additionally, refer to Figure 2 In one embodiment, the feature information to be located includes image feature descriptors. Figure 1 Step S130 in the illustrated embodiment also includes, but is not limited to, the following steps:

[0065] Step S210: Extract image feature points from each of the second images to be located according to the preset corner detection FAST algorithm;

[0066] Step S220: Generate each image feature descriptor based on the image feature points and the preset BRIEF algorithm.

[0067] Understandably, by extracting image feature points from each of the second images to be located using the preset corner detection FAST algorithm, and generating image feature descriptors based on the image feature points and the preset BRIEF algorithm, an effective data foundation can be provided for subsequent image matching processing.

[0068] It should be noted that, in the embodiments of this application, image feature points can also be extracted from each second image to be located using the Scale Invariant Feature Transform (SIFT) algorithm and the SURF algorithm. In the embodiments of this application, each image feature descriptor can also be generated based on the ORB descriptor extraction algorithm and the image feature points. No further limitations are imposed here.

[0069] Additionally, refer to Figure 3 In one embodiment, the plurality of second images to be located includes at least a first image and a second image. The first image is an adjacent frame image of the second image, and the second image corresponds to matching feature points that successfully match the feature points of the first image. The target label includes bounding box location information. Figure 1 Step S150 in the illustrated embodiment includes, but is not limited to, the following steps:

[0070] Step S310: Calculate the coordinates of each center point based on the position information of each recognition box in the first image. The center point coordinates represent the coordinates of the center point of the recognition box corresponding to each target label in the first image.

[0071] Step S320: According to the preset nearest neighbor search algorithm, the target matching feature point is determined from the matching feature points of the second image. The target matching feature point represents the feature point in the matching feature points of the second image that is the nearest neighbor to each center point coordinate.

[0072] Understandably, the coordinates of each center point are calculated based on the position information of each recognition box in the first image (different recognition boxes correspond to different types of target objects identified in the first image). The center point coordinates represent the coordinates of the center point of the recognition box corresponding to each target label in the first image. According to the preset nearest neighbor search algorithm, the target matching feature points are determined from the matching feature points in the second image. The target matching feature points represent the feature points in the matching feature points of the second image that are the nearest neighbors to each center point coordinate, thereby providing an effective data foundation for determining the location result of the target object corresponding to the target label.

[0073] It should be noted that the embodiments of this application do not limit the specific nearest neighbor search algorithm. It can be the KDtree nearest neighbor search algorithm or the K nearest neighbor search algorithm. Those skilled in the art can determine the algorithm according to the actual situation.

[0074] In another embodiment, the number of target matching feature points is at least two. Figure 1Step S160 in the illustrated embodiment includes, but is not limited to, the following steps:

[0075] Step S410: Determine the pixel coordinates of each target matching feature point, and determine the average value of all pixel coordinates as the target pixel difference.

[0076] Step S420: Obtain the target geographic distance difference, which is obtained based on the reference position information of the first image and the reference position information of the second image;

[0077] Step S430: Calculate the ratio of the target geographic distance difference to the target pixel difference to obtain the pixel scale.

[0078] Step S440: Determine the image coordinate system of the second image to be located, and calculate the rotation angle between the preset reference coordinate system and the image coordinate system;

[0079] Step S450: Determine the positioning result of the target object corresponding to the target label based on the pixel scale and rotation angle.

[0080] It should be noted that in the embodiments of this application, the preset reference coordinate system can be the xoy northeast-northeast ground plane coordinate system. Those skilled in the art can determine it according to actual needs, and no restrictions are imposed here.

[0081] It should be noted that the embodiments of this application do not limit the specific data format of the reference location information. The data format of the reference location information is determined by the location information acquisition module of the drone or aerial flight platform equipped with the shooting equipment. The location information acquisition module can be a GPS locator or a Beidou locator.

[0082] It is understood that this embodiment determines the pixel coordinates of each target matching feature point, sets the average of all pixel coordinates as the target pixel difference, obtains the target geographic distance difference based on the reference position information of the first and second images, calculates the ratio of the target geographic distance difference to the target pixel difference to obtain a pixel scale, determines the image coordinate system of the second image to be located, calculates the rotation angle between the preset reference coordinate system and the image coordinate system, and determines the longitude and latitude values ​​of all pixels in the image frame based on the pixel scale and the rotation angle. From the longitude and latitude values ​​of all pixels, the pixel corresponding to the target object is determined, thereby enabling the geographic location of each target object. Based on the technical solution of this embodiment, image matching is performed on continuously acquired frames in real time, and the positioning result is obtained only by combining the matching feature points with the reference position information of the shooting device. It does not rely on sensors such as gyroscopes to provide angle information, eliminating the influence of angle errors, nor does it rely on prior information such as satellite maps. Its application scenarios are not limited, it can reduce the cost of acquiring large datasets, and improve the efficiency of acquiring image positioning results while ensuring image positioning accuracy.

[0083] Additionally, refer to Figure 5 In one embodiment, the number of target matching feature points is at least four. Figure 1 Step S160 in the illustrated embodiment includes, but is not limited to, the following steps:

[0084] Step S510: When the shooting angle of the shooting device does not meet the preset shooting angle conditions, calculate the affine transformation matrix based on at least four target matching feature points.

[0085] Step S520: Determine the positioning result of the target object corresponding to the target label based on the affine transformation matrix, pixel scale, and rotation angle.

[0086] It should be noted that in this embodiment, the preset shooting angle condition can be that the difference between the shooting angle and the preset angle is within a preset error range. The preset angle is the angle corresponding to the pitch angle of the shooting device perpendicular to the ground. When it is detected that the difference between the current shooting angle and the preset angle exceeds the preset error range, it means that the shooting angle of the shooting device does not meet the preset shooting angle condition. In this case, the affine transformation matrix is ​​calculated based on the target matching feature points (i.e. the feature points that are successfully matched in the first image and the second image. The specific number of selected feature points can be determined according to actual needs, such as selecting 4 pairs of matching feature points). The positioning result of the target object corresponding to the target label is determined based on the affine transformation matrix, the pixel scale, and the rotation angle.

[0087] It should be noted that the specific methods for calculating the affine transformation matrix are well known to those skilled in the art, and will not be elaborated upon here.

[0088] Additionally, in one embodiment, reference Figure 6 In execution Figure 1 After step S140, the image localization method provided in this application embodiment also includes, but is not limited to, the following steps:

[0089] Step S610: According to the preset Random Sampling Consensus (RANSAC) algorithm, outliers in multiple matching feature points are removed to obtain matching feature points after removing outliers.

[0090] It should be noted that this embodiment can also remove outliers by comparing the similarity between multiple matching feature points and removing matching feature points with low similarity. Those skilled in the art can select a method to remove outliers from multiple matching feature points according to the actual situation.

[0091] Understandably, by using the pre-defined Random Sampling Consensus (RANSAC) algorithm to remove outliers from multiple matching feature points, the resulting matching feature points can improve the reliability of the matching feature point data, thereby enhancing the accuracy of image localization.

[0092] In addition, during execution Figure 1 Before step S120, the image positioning method of this application embodiment may further include: performing data preprocessing on each first image to be positioned according to a preset image preprocessing rule.

[0093] Understandably, the main purpose of image preprocessing on the first image to be located is to eliminate irrelevant information in the first image to be located, restore useful real information, enhance the detectability of relevant information and simplify the data to the maximum extent, thereby improving the reliability of subsequent applications of the first image to be located.

[0094] It should be noted that the embodiments of this application do not limit the specific method of image preprocessing for the first image to be located. It can be image random rotation processing, normalization processing, or image binarization processing, etc. Those skilled in the art can select the specific method of image preprocessing according to the actual situation.

[0095] Additionally, refer to Figure 7 , Figure 7 This is a schematic diagram of a module of an image positioning device provided in another embodiment of this application. In one embodiment, this application provides an image positioning device 700, which includes:

[0096] The continuous frame image acquisition module 710 is used to acquire continuous frame images, which represent ground images acquired by the UAV during flight. The continuous frame images include multiple first images to be located, and each first image to be located corresponds to reference position information. The reference position information represents the real-time position information of the shooting device used to acquire each first image to be located.

[0097] The image recognition module 720 is used to input each first image to be located into a pre-trained target recognition model for image recognition processing to obtain each second image to be located, and each second image to be located has a corresponding target label.

[0098] The feature extraction module 730 is used to extract features from each of the second images to be located, and obtain the feature information of each image to be located.

[0099] The image matching module 740 is used to perform image matching processing on adjacent images in the second image to be located based on the feature information to be located and according to a preset image matching algorithm to obtain multiple matching feature points.

[0100] The target matching feature point acquisition module 750 is used to determine the target matching feature point from multiple matching feature points based on the target label;

[0101] The positioning result determination module 760 is used to determine the positioning result of the target object corresponding to the target label based on the target matching feature points and reference position information.

[0102] The specific implementation of this image positioning device is basically the same as the specific embodiment of the image positioning method described above, and will not be repeated here.

[0103] Additionally, refer to Figure 8 , Figure 8 This is a structural diagram of an image positioning device provided in another embodiment of this application. An embodiment of this application also provides an image positioning device 800, which includes: a memory 810, a processor 820, and a computer program stored in the memory 810 and executable on the processor 820.

[0104] The processor 820 and memory 810 can be connected via a bus or other means.

[0105] The non-transient software program and instructions required to implement the image localization method of the above embodiments are stored in the memory 810. When executed by the processor 820, the image localization method applied to the image localization device 800 in the above embodiments is executed, for example, the method described above is executed. Figure 1 Method steps S110 to S160, Figure 2 Method steps S210 to S220, Figure 3Method steps S310 to S320 Figure 4 Method steps S410 to S450 Figure 5 Method steps S510 to S520 Figure 6 Method step S610.

[0106] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0107] Furthermore, one embodiment of this application also provides a computer-readable storage medium storing computer-executable instructions that are executed by a processor or controller, for example, by a processor in the above-described image positioning device 800 embodiment, causing the processor to perform the image positioning method applied to the image positioning device in the above-described embodiment, for example, performing the above-described... Figure 1 Method steps S110 to S160, Figure 2 Method steps S210 to S220, Figure 3 Method steps S310 to S320 Figure 4 Method steps S410 to S450 Figure 5 Method steps S510 to S520 Figure 6Method step S610. Those skilled in the art will understand that all or some of the steps in the methods disclosed above, and the system, can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

Claims

1. An image localization method, characterized in that, include: Acquire continuous frame images, which represent ground images acquired by the UAV during flight. The continuous frame images include multiple first images to be located, each of which corresponds to reference position information. The reference position information represents the real-time position information of the shooting device used to acquire each of the first images to be located. Each of the first images to be located is input into a pre-trained target recognition model for image recognition processing to obtain each of the second images to be located, and each of the second images to be located has a corresponding target label. Feature extraction is performed on each of the second images to be located to obtain the feature information of each image to be located; Based on the feature information to be located, image matching processing is performed on adjacent images in the second image to be located according to a preset image matching algorithm to obtain multiple matching feature points; The target matching feature point is determined from the plurality of matching feature points based on the target label; The location result of the target object corresponding to the target label is determined based on the target matching feature points and the reference location information; The plurality of second images to be located include at least a first image and a second image, wherein the first image is an adjacent frame image of the second image, and the second image corresponds to a matching feature point that successfully matches the feature point of the first image. The target label includes bounding box location information. The step of determining the target matching feature point from the plurality of matching feature points based on the target label includes: The coordinates of each center point are calculated based on the position information of each recognition box in the first image. The coordinates of each center point represent the coordinates of the center point of the recognition box corresponding to each target label in the first image. According to a preset nearest neighbor search algorithm, the target matching feature point is determined from the matching feature points of the second image. The target matching feature point represents the feature point in the matching feature points of the second image that is the nearest neighbor to each of the center point coordinates.

2. The image localization method according to claim 1, characterized in that, The feature information to be located includes image feature descriptors. The step of extracting features from each of the second images to be located to obtain the feature information to be located includes: Based on the preset corner detection FAST algorithm, image feature points are extracted from each of the second images to be located; Each image feature descriptor is generated based on the image feature points and the preset BRIEF algorithm.

3. The image localization method according to claim 1, characterized in that, The number of target matching feature points is at least two, and the step of determining the location result of the target object corresponding to the target label based on the target matching feature points and the reference location information includes: Determine the pixel coordinates of each of the target matching feature points, and determine the average value of all the pixel coordinates as the target pixel difference; Obtain the target geographic distance difference, which is obtained based on the reference location information of the first image and the reference location information of the second image; Calculate the ratio of the target geographical distance difference to the target pixel difference to obtain a pixel scale; Determine the image coordinate system of the second image to be located, and calculate the rotation angle between the preset reference coordinate system and the image coordinate system; The positioning result of the target object corresponding to the target label is determined based on the pixel scale and the rotation angle.

4. The image localization method according to claim 3, characterized in that, The number of target matching feature points is at least four, and the step of determining the location result of the target object corresponding to the target label based on the target matching feature points and the reference location information includes: When the shooting angle of the shooting device does not meet the preset shooting angle conditions, an affine transformation matrix is ​​calculated based on at least four target matching feature points; The positioning result of the target object corresponding to the target label is determined based on the affine transformation matrix, the pixel scale, and the rotation angle.

5. The image localization method according to claim 1, characterized in that, After obtaining multiple matching feature points by performing image matching processing on adjacent images in the second image to be located based on the feature information to be located and according to a preset image matching algorithm, the method further includes: According to the preset Random Sampling Consensus (RANSAC) algorithm, outliers are removed from multiple matching feature points to obtain matching feature points after removing outliers.

6. The image localization method according to claim 1, characterized in that, The target recognition model is the Yolov5 model.

7. An image positioning device, characterized in that, include: A continuous frame image acquisition module is used to acquire continuous frame images, which represent ground images acquired by the UAV during flight. The continuous frame images include multiple first images to be located, each of which corresponds to reference position information. The reference position information represents the real-time position information of the shooting device used to acquire each of the first images to be located. The image recognition module is used to input each of the first images to be located into a pre-trained target recognition model for image recognition processing to obtain each of the second images to be located, and each of the second images to be located has a corresponding target label; The feature extraction module is used to extract features from each of the second images to be located, and obtain the feature information of each image to be located. The image matching module is used to perform image matching processing on adjacent images in the second image to be located based on the feature information to be located and according to a preset image matching algorithm to obtain multiple matching feature points; The target matching feature point acquisition module is used to determine the target matching feature point from the plurality of matching feature points based on the target label; The positioning result determination module is used to determine the positioning result of the target object corresponding to the target label based on the target matching feature points and the reference position information; The plurality of second images to be located include at least a first image and a second image, wherein the first image is an adjacent frame image of the second image, and the second image corresponds to a matching feature point that successfully matches the feature point of the first image. The target label includes bounding box location information. The step of determining the target matching feature point from the plurality of matching feature points based on the target label includes: The coordinates of each center point are calculated based on the position information of each recognition box in the first image. The coordinates of each center point represent the coordinates of the center point of the recognition box corresponding to each target label in the first image. According to a preset nearest neighbor search algorithm, the target matching feature point is determined from the matching feature points of the second image. The target matching feature point represents the feature point in the matching feature points of the second image that is the nearest neighbor to each of the center point coordinates.

8. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the image positioning method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing computer-executable instructions for performing the image localization method as described in any one of claims 1 to 6.

Images