Picture calibration method, device, equipment and storage medium

Abstract

The application discloses a picture calibration method, device, equipment and storage medium. The picture calibration method comprises the following steps: acquiring a source picture and a picture to be calibrated; the source picture comprises identification information of a control point; image feature extraction is performed on the source picture and the picture to be calibrated respectively to obtain first feature data and second feature data groups; similar feature data corresponding to the first feature data is determined from a plurality of second feature data; the first feature data, the similar feature data and a surveying and mapping characteristic value of the picture to be calibrated are input into a position prediction model to obtain first position information of the control point in the picture to be calibrated; the surveying and mapping characteristic value comprises parameter information associated with the picture to be calibrated or the control point. According to the embodiment of the application, the automatic calibration of the control point of the picture to be calibrated can be performed according to the source picture, manual piercing operation is replaced, human resources and cost are reduced, and piercing calibration efficiency is improved.

Description

Technical Field

[0001] This application belongs to the field of surveying and mapping technology, and in particular relates to a method, apparatus, device and storage medium for image calibration. Background Technology

[0002] Currently, in the process of 3D image modeling, the triangulation method is commonly used. This involves pre-processing with oblique photogrammetry using drones or other aerial photography equipment to capture images of areas with control points (COPs). The corresponding positions of the COPs are then determined from these images. After performing the triangulation operation on a sufficient number of images, aerial triangulation and final modeling can be achieved.

[0003] Traditional pixelation operations typically involve manual intervention, with personnel manually marking control points in various images. To reduce the workload and improve efficiency, some technologies combine manual and software-based pixelation, first manually marking the control points and then automatically marking them using software. However, in large-scale real-world modeling tasks, even a small amount of manual pixelation is often time-consuming and labor-intensive, increasing labor costs and reducing pixelation efficiency. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for image calibration, which can solve the technical problem of low efficiency in image control point puncturing operations in the prior art.

[0005] In a first aspect, embodiments of this application provide a method for calibrating an image, the method comprising:

[0006] Obtain the source image and the image to be calibrated; the source image includes the identification information of the control points.

[0007] Image features are extracted from the source image and the image to be calibrated to obtain a first feature data set and a second feature data set. The first feature data set consists of the image features corresponding to the source image. The second feature data set includes multiple second feature data sets, which are image features corresponding to a portion of the image to be calibrated.

[0008] From multiple second feature data, identify similar feature data that correspond to the first feature data;

[0009] The first feature data, similar feature data, and the mapping feature values ​​of the image to be calibrated are input into the location prediction model to obtain the first location information of the control point in the image to be calibrated; the mapping feature values ​​include parameter information associated with the image to be calibrated or the control point.

[0010] According to some embodiments of this application, similar feature data corresponding to the first feature data is determined from a plurality of second feature data, including:

[0011] The first feature data and the second feature data set are input into the image comparison model;

[0012] The first feature data is compared with multiple second feature data using an image comparison model to obtain similar feature data; the similar feature data is the second feature data with the smallest feature comparison difference with the first feature data.

[0013] According to some embodiments of this application, an image comparison model is used to compare first feature data with multiple second feature data to obtain similar feature data, including:

[0014] An image comparison model is used to compare the first feature data with multiple second feature data to obtain multiple similarity values; the similarity values ​​are used to represent the degree of difference between the feature comparisons of the first feature data and the second feature data;

[0015] Similar feature data are determined based on the similarity values ​​corresponding to multiple secondary feature data.

[0016] According to some embodiments of this application, determining similar feature data based on similarity values ​​corresponding to multiple second feature data further includes:

[0017] The image comparison judgment value is determined based on the similarity values ​​corresponding to multiple second feature data respectively;

[0018] When the image comparison judgment value is greater than the judgment threshold, the similarity feature data is the second feature data corresponding to the largest similarity value;

[0019] If the image comparison judgment value is less than the judgment threshold, it is determined that the image to be calibrated does not contain control points.

[0020] According to some embodiments of this application, image features are extracted from the source image and the image to be calibrated to obtain a first feature data set and a second feature data set, including:

[0021] Image features are extracted from the source image to obtain the first feature data;

[0022] The image to be calibrated is divided into multiple comparison regions;

[0023] Image features were extracted from multiple comparison regions to obtain multiple second feature data.

[0024] According to some embodiments of this application, in the multiple comparison regions obtained by dividing the image to be calibrated, at least two comparison regions partially overlap.

[0025] According to some embodiments of this application, the first feature data, similar feature data, and the mapping feature values ​​of the image to be calibrated are input into the location prediction model to obtain the first location information of the control points in the image to be calibrated, including:

[0026] Input the first feature data, similar feature data, and the mapping feature values ​​of the image to be calibrated into the location prediction model;

[0027] The second location information of the control point in the similarity comparison area is obtained by using the prediction layer of the location prediction model; the similarity comparison area is the comparison area corresponding to similar feature data.

[0028] The transformation layer of the location prediction model is used to convert the second location information of the control point in the similarity comparison area into the first location information of the control point in the image to be calibrated.

[0029] Secondly, embodiments of this application provide an image calibration device, the device comprising:

[0030] The image acquisition module is used to acquire source images and images to be calibrated; the source image includes the identification information of control points.

[0031] The feature extraction module is used to extract image features from the source image and the image to be calibrated, respectively, to obtain a first feature data and a second feature data set; the first feature data is the image feature corresponding to the source image; the second feature data set includes multiple second feature data, which are image features corresponding to a portion of the image to be calibrated;

[0032] The similarity comparison module is used to determine similar feature data corresponding to the first feature data from multiple second feature data;

[0033] The location calibration module is used to input the first feature data, similar feature data, and the mapping feature values ​​of the image to be calibrated into the location prediction model to obtain the first location information of the control point in the image to be calibrated; the mapping feature values ​​include parameter information associated with the image to be calibrated or the control point.

[0034] Thirdly, embodiments of this application provide an image calibration device, which includes: a processor and a memory storing computer program instructions;

[0035] When the processor executes computer program instructions, it implements the image calibration method as described in the above embodiment.

[0036] Fourthly, embodiments of this application provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement the image labeling method described in the above embodiments.

[0037] Compared with existing technologies, the image calibration method, apparatus, device, and storage medium provided in this application, after extracting image features from a source image containing control point (CNP) identification information to obtain first feature data, can extract features from the image to be calibrated to obtain multiple second feature data. From these multiple second feature data, the most similar feature data to the first feature data can be determined. Based on the similar feature data and the mapping feature values ​​during mapping of the image to be calibrated, the position prediction model can predict the first position information of the CNP in the image to be calibrated using the first feature data and the similar feature data, thereby determining the pixel coordinates of the CNP in the image to be calibrated and realizing the puncturing operation of the image to be calibrated. By providing a source image containing CNP identification information, the image to be calibrated can be identified to determine whether the image to be calibrated contains CNP and the specific position information of the CNP. During the puncturing operation of the image to be calibrated, automated point measurement can be achieved, eliminating the need for manual puncturing, reducing labor costs, and improving puncturing efficiency. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic flowchart of an image calibration method provided in an embodiment of this application;

[0040] Figure 2 This is a flowchart illustrating a method for calibrating images according to another embodiment of this application;

[0041] Figure 3 This is a flowchart illustrating a method for calibrating images according to another embodiment of this application;

[0042] Figure 4 This is a schematic flowchart of a method for calibrating images provided in another embodiment of this application;

[0043] Figure 5 This is a schematic diagram of the calibration device shown in the images provided in the embodiments of this application;

[0044] Figure 6 This is a schematic diagram of the calibration device shown in the images provided in the embodiments of this application. Detailed Implementation

[0045] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples of this application.

[0046] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0047] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The embodiments will now be described in detail with reference to the accompanying drawings.

[0048] Currently, in the process of 3D image modeling, the triangulation method is commonly used. This involves pre-processing with oblique photogrammetry using drones or other aerial photography equipment to capture images of areas with control points (COPs). The corresponding positions of the COPs are then determined from these images. After performing the triangulation operation on a sufficient number of images, aerial triangulation and final modeling can be achieved.

[0049] Current image control point (ADC) manipulation techniques typically employ manual intervention, with personnel manually marking control points in each image. This manual method incurs high labor costs and significantly extends the marking time, leading to reduced modeling efficiency. To improve modeling efficiency and reduce the workload of manual marking, related technologies combine manual and automated marking. A portion of the control points are marked manually first, and then software automatically marks the remaining images based on these marked control points. This automated marking method saves substantial manpower and improves marking efficiency while maintaining a certain level of accuracy.

[0050] However, in the aforementioned combination of manual and automated image manipulation, the initial manipulation process still requires manual intervention. In large modeling tasks, even if only a small portion of the images need manual manipulation, the number of images is substantial, consuming significant human resources. Furthermore, when the image resolution is high, even manually manipulating a single image often requires considerable time and effort, significantly increasing labor costs and greatly reducing manipulation efficiency.

[0051] To address the aforementioned technical problems, embodiments of this application provide a method, apparatus, device, and storage medium for image calibration. The image calibration method provided in this application embodiment will be described first below.

[0052] Figure 1 A schematic diagram of an image calibration method according to an embodiment of this application is shown. The image calibration method includes:

[0053] S110, acquire the source image and the image to be calibrated; the source image includes the identification information of the control points;

[0054] S120, Image features are extracted from the source image and the image to be calibrated respectively to obtain a first feature data and a second feature data group; the first feature data is the image feature corresponding to the source image; the second feature data group includes multiple second feature data, and the second feature data is the image feature corresponding to a part of the image to be calibrated.

[0055] S130, determine similar feature data corresponding to the first feature data from multiple second feature data;

[0056] S140, the first feature data, similar feature data, and the mapping feature values ​​of the image to be calibrated are input into the location prediction model to obtain the first location information of the control point in the image to be calibrated; the mapping feature values ​​include parameter information associated with the image to be calibrated or the control point.

[0057] The image calibration method provided in this application can be applied to an image calibration device, which can be a mobile device, such as a smart mobile terminal, tablet computer, or drone. Alternatively, the image calibration device can also be a non-mobile device, such as a server, industrial computer, or various edge computing units. This embodiment does not limit the specific form of the image calibration device.

[0058] In this embodiment, after extracting image features from the source image containing control point (CPI) identification information to obtain first feature data, multiple second feature data can be obtained by extracting features from the image to be calibrated. From these second feature data, the most similar feature data to the first feature data can be determined. Based on the similar feature data and the mapping feature values ​​of the image to be calibrated, the position prediction model can predict the first position information of the CPI in the image to be calibrated using the first feature data and the similar feature data, thereby determining the pixel coordinates of the CPI in the image to be calibrated and realizing the point-pricking operation on the image to be calibrated. By providing a source image containing CPI identification information, the image to be calibrated can be identified to determine whether it contains CPI and its specific position information. During the point-pricking operation on the image to be calibrated, automated point measurement can be achieved, eliminating the need for manual point-pricking, reducing human resource costs, and improving point-pricking calibration efficiency.

[0059] In S110, the device can acquire a source image and an image to be calibrated. The source image may contain control point (CPI) identification information. Based on the CPI identification information in the source image, it can be determined whether CPIs exist in the image to be calibrated, and further, the specific location of the CPIs in the image to be calibrated can be determined.

[0060] After ground-based image control points (VCS points) are deployed, they can be photographed using imaging equipment to obtain source images containing their identification information. This identification information can include the VCS point's shape and color. For example, a VCS point can be circular, L-shaped, cross-shaped, or funnel-shaped, and its color can be a single color or a mixture of two or more colors. When setting the color of the VCS point, its color characteristics should differ somewhat from the surrounding environment.

[0061] When taking source images, relevant personnel can take pictures of the control points at the locations where they are set up. For example, field personnel can take pictures of the control points directly after they are set up. Alternatively, aerial photography equipment, such as drones or aircraft, can be used for high-altitude photography.

[0062] It's important to note that to minimize interference in the source image, the shooting range is typically controlled during capture. This ensures the source image includes all the control point's identification information while minimizing other environmental factors. For example, when personnel directly photograph control points, the shooting range can be adjusted to ensure the control points occupy most of the frame. For drones or other aerial photography equipment, while maintaining high resolution, the shooting range can be reduced by zooming or other methods to minimize interference from factors other than control points. Furthermore, after obtaining the source image, further processing such as cropping can be performed to retain the control point's identification information while reducing the impact of other factors.

[0063] In step S120, after acquiring the source image and the image to be calibrated, image features can be extracted from both images. Extracting image features from the source image yields the first set of feature data, while extracting image features from the image to be calibrated yields the second set of feature data.

[0064] Understandably, after obtaining the image to be calibrated, it can be divided into multiple sub-images, each containing a portion of the image to be calibrated. By extracting image features from each sub-image separately, multiple second feature data corresponding to each sub-image can be obtained, thus forming a second feature data set.

[0065] The image feature extraction methods mentioned above can be SIFT (Scale-invariant feature transform), VLAD (vector of locally aggregated descriptors), HOG (Histogram of Oriented Gradient), LBP (Local Binary Pattern), Haar feature algorithm, or other feature extraction models.

[0066] In this embodiment, one or more of the above-mentioned image feature extraction algorithms can be used to extract image features from the source image and the image to be calibrated. By performing image feature extraction, various image features can be extracted from the image. For example, image features may include geometric features, shape features, amplitude features, color features, histogram features, and local binary pattern features, etc. It is understood that the feature data extracted by the image feature extraction method used in this embodiment should at least include partial identification information of control points, such as the shape features and color features of the control points.

[0067] Please refer to Figure 2 In some embodiments, the above-described S120 may further include:

[0068] S210, perform image feature extraction on the source image to obtain the first feature data;

[0069] S220, divides the image to be calibrated into multiple comparison regions;

[0070] S230, image features are extracted from multiple comparison regions to obtain multiple second feature data.

[0071] In this embodiment, image feature extraction can be directly performed on the source image to obtain the first feature data. For the image to be calibrated, due to its large shooting range and small control points, directly calibrating the image will result in feature data containing a large number of invalid features, leading to insufficient or missing feature data for control points. Therefore, by dividing the image to be calibrated, image feature extraction can be performed on multiple comparison regions separately, ensuring that the extracted feature data in the comparison regions containing control points accurately includes their features. This allows for accurate identification of control point features in the feature comparison analysis of the first and second feature data, improving the accuracy of the feature comparison analysis.

[0072] In S210, after obtaining the source image and the image to be calibrated, image features can be extracted directly from the source image to obtain the first feature data corresponding to the source image.

[0073] The source image contains the identification information of control points. By extracting image features from the source image, the identification information of control points can be converted into first feature data. Based on the second feature data in the image to be calibrated, it can be determined whether the image to be calibrated contains control points through feature comparison, thereby realizing the calibration of control points.

[0074] Understandably, when control points occupy a large portion of the source image, image feature extraction can be performed directly on the source image. The resulting first feature data will contain detailed information about the control points and reduce the influence of other factors in the source image besides control points. If the source image has a large shooting area and control points only occupy a portion of the image, the source image can be enlarged or cropped first to reduce the amount of image content other than control points before image feature extraction.

[0075] In S220, the images to be calibrated are typically taken from high altitudes using drones or other aerial photography equipment via oblique photography or other shooting methods. These images usually cover a large area. When control points (APIs) are captured within the image, they occupy only a small portion of the image. Directly extracting features from these APIs would be insufficient because their small size would prevent the extracted features from fully reflecting their identification information, thus increasing the difficulty of comparing the calibrated image with the source image. Therefore, after acquiring the calibrated image, it is usually divided into multiple comparison regions, each corresponding to a sub-image. When extracting features from the sub-images, the area occupied by APIs is significantly larger than their area in the complete calibrated image. This allows the extracted feature data to contain more accurate API feature data, facilitating comparison and judgment between the sub-image and the source image.

[0076] In S230, after dividing the image to be calibrated into multiple comparison regions, image features can be extracted from the sub-images corresponding to each comparison region to obtain the second feature data corresponding to each comparison region. Multiple second feature data can form a second feature data group.

[0077] In some embodiments, when dividing the image to be calibrated, a pre-set division method can be used to obtain multiple comparison regions. Among these multiple comparison regions, at least two comparison regions have partial overlap.

[0078] If the image to be calibrated is divided using a method where the comparison regions do not overlap, it is possible that control points in the image to be calibrated may be divided into two comparison regions. This could result in the inability to identify control points when they are present in the image. Taking two comparison regions as an example, to avoid splitting control points into two regions during region division and affecting control point calibration, the division method can be adjusted so that there is partial overlap between the two comparison regions. When a control point is located at the boundary between the two comparison regions, both regions can completely contain the control point, thus preventing the control point from being segmented and causing calibration failure.

[0079] It is understandable that the division method described above, which involves partial overlap between two comparison regions, can be applied to more comparison regions. For example, when dividing an image to be calibrated, adjacent comparison regions can partially overlap, allowing control points to fall completely within a single comparison region.

[0080] In S130, after obtaining the first feature data and multiple second feature data through image feature extraction, similar feature data corresponding to the first feature data can be determined from the multiple second feature data.

[0081] By performing feature comparison analysis between each second feature data point and the first feature data point, the similarity between the sub-image corresponding to the second feature data point and the source image corresponding to the first feature data point can be determined. For example, the method for determining the similarity can be determined based on different image feature extraction methods. For instance, the first and second feature data points can be feature matrices; by comparing and analyzing the feature matrices, the similarity between the first and second feature data points can be obtained. After comparing each second feature data point with the first feature data point, the similarity level corresponding to each second feature data point can be obtained, and similar feature data points can be determined from multiple second feature data points based on each similarity level.

[0082] Understandably, since similar feature data is the second feature data most similar to the first feature data, if the image to be labeled contains control points, then those control points should typically be located in the region corresponding to the similar feature data. After determining the similar feature data, the sub-images corresponding to those similar feature data can be further determined, and the positional information of the control points can be labeled within the sub-images.

[0083] Please refer to Figure 3 In some embodiments, the above-described S130 may further include:

[0084] S310, input the first feature data and the second feature data group into the image comparison model;

[0085] S320, using an image comparison model, the first feature data is compared with multiple second feature data to obtain similar feature data; the similar feature data is the second feature data with the smallest feature comparison difference with the first feature data.

[0086] In this embodiment, an image comparison model can be used to compare and analyze the first feature data and the second feature data. After each second feature data is compared with the first feature data, the corresponding feature comparison difference can be determined. The second feature data with the smallest feature comparison difference from the first feature data can be selected from multiple second feature data and thus regarded as similar feature data.

[0087] In step S310, after obtaining the first feature data and multiple second feature data, the multiple second feature data can be compared with the first feature data respectively. After determining a certain second feature data from the multiple second feature data, this second feature data and the first feature data can be input into the image comparison model, and the image comparison model is used to calibrate control points. The image comparison model can be pre-trained. For example, by setting training sets, test sets, and validation sets, the image comparison model is trained and optimized, and the trained image comparison model is used to predict the positions of the first and second feature data.

[0088] In step S320, after inputting the first feature data and the second feature data into the image comparison model, the model can be used to perform feature comparison between the first feature data and the second feature data to obtain the similarity between the first feature data and a certain second feature data. After comparing each second feature data with the first feature data separately, the similarity level corresponding to each second feature data can be obtained. Based on the various similarity levels, the second feature data with the smallest feature comparison difference from the first feature data can be determined from multiple second feature data, and this second feature data is taken as the similar feature data.

[0089] Please refer to Figure 4 In some embodiments, the above-described S320 may further include:

[0090] S410, The first feature data is compared with multiple second feature data using an image comparison model to obtain multiple similarity values; the similarity values ​​are used to represent the degree of difference between the feature comparison of the first feature data and the second feature data;

[0091] S420, determine similar feature data based on the similarity values ​​corresponding to multiple second feature data respectively.

[0092] In this embodiment, when each second feature data is compared with the first feature data using an image comparison model, a corresponding similarity value can be obtained. Based on the magnitude of the similarity value, similar feature data can be determined from multiple second feature data.

[0093] In S410, after the first feature data and the second feature data are input into the image comparison model, the image comparison model can be used to perform feature comparison in order to obtain the similarity value between the first feature data and each of the second feature data.

[0094] The similarity value can be a real number within a pre-defined range, used to characterize the degree of difference between the first feature data and the second feature data when comparing features. For example, within the range of 0-100, a higher similarity value indicates a smaller difference between the second feature data and the first feature data corresponding to that similarity value.

[0095] In S420, after performing feature comparison analysis between each second feature data and the first feature data using an image comparison model, a similarity value corresponding to each second feature data can be obtained. Since the similarity value reflects the difference between the first and second feature data, a suitable extreme value can be determined from multiple similarity values ​​to identify the corresponding second feature data as similar feature data. For example, within a given range, a higher similarity value indicates a smaller feature difference between the two feature data, so the second feature data corresponding to the highest similarity value can be selected as similar feature data. Conversely, within a given range, a lower similarity value indicates a smaller feature difference between the two feature data, so the second feature data corresponding to the lowest similarity value can be selected as similar feature data.

[0096] In some embodiments, the above-described S420 may further include:

[0097] S510, determine the image comparison judgment value based on the similarity values ​​corresponding to multiple second feature data respectively;

[0098] S520, when the image comparison judgment value is greater than the judgment threshold, the similarity feature data is the second feature data corresponding to the largest similarity value;

[0099] S530, if the image comparison judgment value is less than the judgment threshold, determine that the image to be calibrated does not contain control points.

[0100] In this embodiment, an image comparison determination value can be calculated using the similarity values ​​of multiple second feature data. When the image comparison determination value is greater than the determination threshold, it can be determined that there are sub-images containing control points among these sub-images. In this case, the second feature data with the highest similarity value can be used as the similarity feature data. When the image comparison determination value is less than the determination threshold, it can be determined that these sub-images all have significant differences from the source image, thus determining that the image to be calibrated does not contain control points. When it is determined that the image to be calibrated does not contain control points, the location prediction model can no longer be used to predict the location of control points, thereby reducing the computational load in the calibration process.

[0101] In S510, after obtaining the similarity values ​​corresponding to multiple second feature data using the image comparison model, an image comparison judgment value can be determined based on each similarity value. By comparing the image comparison judgment value with the judgment threshold, it can be determined whether the image to be calibrated contains control points.

[0102] The method described above, which calculates an image comparison judgment value based on multiple similarity values ​​and compares it with a judgment threshold, can be applied when multiple similarity values ​​indicate significant feature differences. For example, when the similarity values ​​range from 0 to 100, and a higher similarity value indicates a smaller feature difference, if multiple similarity values ​​are all low, such as all below 20, an image comparison judgment value can be calculated based on these multiple similarity values. This image comparison judgment value can be the mean, median, or mode of the multiple similarity values, or it can be the standard deviation, weighted average, or a numerical result calculated using other preset algorithms.

[0103] In S520, the calculated image comparison judgment value is compared with a pre-set judgment threshold to determine whether the image comparison judgment value is greater than or less than the judgment threshold. If the image comparison judgment value is greater than the judgment threshold, it means that there may still be comparison regions containing control points in multiple comparison regions. In this case, the second feature data corresponding to the largest similarity value can be used as the similarity feature data.

[0104] In S530, when the image comparison judgment value is less than the judgment threshold, it is determined that multiple second feature data are significantly different from the first feature data, that is, no control points are included in each comparison area, and no control points are included in the image to be calibrated.

[0105] When it is determined that the image to be calibrated does not contain any control points, the calibration process can be terminated and the corresponding judgment result can be output.

[0106] Understandably, the method described above for determining image comparison criteria based on various similarity values ​​is generally applicable when multiple similarity values ​​are relatively low. If one or more similarity values ​​are high, for example, a similarity value of 80 or higher, then the second feature data with the highest similarity value can be directly used as the similarity feature data.

[0107] In one specific implementation, if it is determined that the image to be calibrated does not contain control points, the next image to be calibrated can be selected to continue calibrating the control points.

[0108] It should be noted that the comparison between the image comparison determination value and the determination threshold in the above embodiments can also be determined based on the calculation method of the image comparison determination value. For example, when there is a correlation between the image comparison determination value and the standard deviation of each similarity value, the larger the image comparison determination value, the greater the difference between each similarity value. It can be understood that if each similarity value is low and the difference between each similarity value is also small, it means that each comparison region is relatively close, and the probability that each comparison region does not contain control points is relatively high. However, when the difference between each similarity value is large, it means that there is also a certain difference between each comparison region, and at this time, there may be a comparison region containing control points. In this calculation method, after obtaining the corresponding determination threshold, when the image comparison determination value is greater than the determination threshold, the second feature data corresponding to the largest similarity value should be used as the similarity feature data; when the image comparison determination value is less than the determination threshold, it is determined that the image to be labeled does not contain control points.

[0109] In step S140, after inputting the first feature data, similar feature data, and the mapping feature values ​​of the image to be calibrated into the location prediction model, the model can be used to predict the specific location of the control points in the image to be calibrated, thereby obtaining the first location information of the control points. The first location information can be the coordinate information or pixel information corresponding to the control points in the image to be calibrated. Based on the first location information, the specific location of the control points can be determined from the image to be calibrated, thus achieving the calibration of the control points in the image to be calibrated.

[0110] The aforementioned mapping feature values ​​may include parameter information associated with the image to be calibrated or the control point. Based on this associated parameter information, the approximate orientation or location of the control point in the image to be calibrated can be roughly determined. It is understandable that because the location prediction model needs to perform massive data calculations to directly calculate the predicted location information of the control point in the image to be calibrated based on the first feature data and similar feature data, it not only places high demands on the hardware of the equipment but also leads to high computational resource consumption, significantly increasing the calibration cost of the control point. Therefore, when performing location prediction through the location prediction model, mapping feature values ​​can also be provided to the model to further reduce the prediction range, decrease the computational load, and improve the prediction efficiency of location information with the assistance of these mapping feature values.

[0111] In some embodiments, S140 may further include:

[0112] S610, input the first feature data, similar feature data and the mapping feature values ​​of the image to be calibrated into the location prediction model;

[0113] S620, the prediction layer of the location prediction model is used to obtain the second location information of the control point in the similar comparison area; the similar comparison area is the comparison area corresponding to similar feature data;

[0114] S630 uses the transformation layer of the location prediction model to convert the second location information of the control point in the similarity comparison area into the first location information of the control point in the image to be calibrated.

[0115] In this embodiment, the location prediction model may include a prediction layer and a transformation layer. The prediction layer predicts the position of the control point in the corresponding comparison area based on the mapping feature values, first feature data, and similar feature data, to obtain the second location information of the control point. The transformation layer determines the position of the control point in the image to be calibrated based on the corresponding position of the comparison area in the image to be calibrated and the second location information of the control point. Through the prediction layer and the transformation layer, the first location information corresponding to the control point in the image to be calibrated can be directly output.

[0116] In S610, after determining similar feature data from multiple second feature data, the first feature data, similar feature data, and the mapping feature values ​​of the image to be calibrated can be input into the position prediction model, and the position prediction model can be used to predict the position information of the control point in the image to be calibrated.

[0117] In S620, the location prediction model includes a prediction layer and a transformation layer. The prediction layer can predict the location information of the control point in the comparison area corresponding to the second feature data based on the first feature data and similar feature data, combined with the mapping feature values ​​of the image to be calibrated, so as to obtain the specific location coordinates of the control point in the similar comparison area.

[0118] The mapping feature values ​​of the aforementioned image to be calibrated are associated with the image or control points (APIs). When predicting the location information of APIs, this association helps the location prediction model reduce computational load and improve prediction efficiency. For example, mapping feature values ​​can be shooting equipment parameters such as the positioning information of the shooting equipment, shooting tilt angle, equipment altitude, and shooting range when the image to be calibrated was taken. They can also be the approximate location information of APIs provided by field personnel, or the location and shape information of specific objects around the APIs. With the assistance of these mapping feature values, the location prediction model can further narrow down the possible range of the API's location when predicting its location, thereby reducing computational load and improving prediction efficiency and accuracy.

[0119] It is understandable that the prediction layer of the aforementioned location prediction model receives input parameters of the first feature data and similar feature data. Therefore, the second location information of the control point predicted should be the location information of the control point within the similarity comparison region. This similarity comparison region is the comparison region corresponding to the similar feature data.

[0120] In S630, after determining the second location information of the control point, the transformation layer of the location prediction model can be used to transform the location information of the control point, converting the second location information into the first location information. This first location information is the first location information corresponding to the control point in the image to be calibrated.

[0121] Understandably, the similarity comparison region is the shooting area corresponding to the sub-image obtained after dividing the image to be calibrated. Based on the division method of the image to be calibrated, the relative position of the shooting area corresponding to the sub-image within the complete region of the image to be calibrated can be determined. Based on the second position information of the control points in the similarity comparison region and the relative position information of the similarity comparison region within the complete region, the first position information of the control points in the image to be calibrated can be determined.

[0122] After determining the first position information of the control points, the puncture point calibration of the control points in the image to be calibrated is completed.

[0123] In some embodiments, the sample data input to the location prediction model for training can be pre-calibrated with artificial prick marks to obtain the artificial prick mark results for each sample data. During the training of the location prediction model, based on the differences between the prediction results of the location prediction model for each sample data and the artificial prick mark results for each sample data, a corresponding optimization function can be fitted and generated. It is understood that adding an optimization function to the location prediction model can compensate for the prediction results generated by the model, making the prediction results more accurate. During the training process of the location prediction model, a training termination condition can be set: the difference between the prediction results of the location prediction model and the artificial prick mark results is reduced to within the required accuracy range.

[0124] In some embodiments, the first location information includes pixel coordinates; after S140 above, it may further include:

[0125] S710, obtain the coordinate range of the image to be calibrated;

[0126] S720, determine whether the pixel coordinates of the control points in the image to be calibrated are within the coordinate range;

[0127] S730 determines that the image to be calibrated does not contain a control point if the pixel coordinates of the control point are outside the coordinate range.

[0128] In this embodiment, after obtaining the pixel coordinates of the control points in the image to be calibrated through the location prediction model, if the pixel coordinates are outside the coordinate range of the image to be calibrated, it can be determined that the image to be calibrated does not contain control points.

[0129] In the S710, the first position information of the image control point can be the pixel coordinates of the image control point in the image to be calibrated. Based on the shooting parameters of the image to be calibrated, the coordinate range of the image to be calibrated in the image plane can be determined.

[0130] In S720, based on the coordinate range of the image to be calibrated in the image plane and the pixel coordinates of the control point, it can be determined whether the pixel coordinates are within the coordinate range of the image to be calibrated. For example, the coordinate range of the image to be calibrated may include the coordinate range of the x-axis and the coordinate range of the y-axis. If the x-coordinate of the control point exceeds the coordinate range of the x-axis, or the y-coordinate exceeds the coordinate range of the y-axis, then it can be determined that the pixel coordinates of the control point are outside the coordinate range of the image to be calibrated. Alternatively, directly comparing the pixel coordinates of the control point with the coordinate range of the image to be calibrated can also determine whether the control point is within the image to be calibrated.

[0131] In S730, when the pixel coordinates of the control point are determined to be outside the coordinate range of the image to be calibrated, it means that the position of the control point predicted by the position prediction model is not within the image to be calibrated. At this time, it can be determined that the image to be calibrated does not contain the control point.

[0132] In some embodiments, after acquiring the source image and the image to be calibrated, the first position information of the control points in the image to be calibrated can be determined by the method proposed in the above embodiments, thereby realizing the calibration of the control points of the image to be calibrated.

[0133] After the image to be calibrated is completed, if it is necessary to continue calibrating the same type of control points, the next image to be calibrated can be obtained and the calibration can continue without changing the source image.

[0134] If the source image remains unchanged, when calibrating the next image to be calibrated, it is not necessary to repeatedly extract image features from the source image; instead, the already generated first feature data can be directly obtained, thereby reducing the computational load during the calibration process. Similarly, since the imaging device capturing a large number of images to be calibrated is continuously shooting, there are many identical regions between adjacent images. When extracting image features from the current image to be calibrated, if the second feature data obtained after dividing the image to be calibrated is highly consistent with the similar feature data generated from previous images, the position information of the control points in the current image to be calibrated can be quickly determined based on the position information of the control points in the similar feature data, thereby improving calibration efficiency.

[0135] When calibrating the same control point, if the number of calibrated images reaches a preset number, the next control point can be calibrated. That is, the source image of the current control point is replaced with the source image of the next control point, and the corresponding image to be calibrated is obtained for control point calibration.

[0136] In some embodiments, after calibrating each control point a certain number of times to obtain the position information of each control point in multiple images to be calibrated, the control points can continue to be calibrated to fulfill the control point calibration requirements in the reality modeling task.

[0137] In addition, after a certain number of calibrations are completed at each control point (e.g., three images to be calibrated at each control point), automated puncturing can be performed on the remaining images based on the already calibrated images. This method, combined with automated puncturing, can quickly perform automated puncturing on the remaining images while maintaining a certain level of accuracy, thus improving the efficiency of puncturing in the modeling task. Since automated puncturing requires a certain number of already calibrated images, the above embodiment of calibrating the source images and images to be calibrated allows for the rapid completion of puncturing on the remaining images after calibrating a portion of the images. Because the already calibrated images required by automated puncturing can be obtained from the source images, images to be calibrated, and the location prediction model, it can replace manual operation for calibrating the initial portion of the images, thereby avoiding the consumption of human resources and improving modeling efficiency.

[0138] Understandably, automated image puncturing in software has a minimum requirement for the number of images that have already been punctured. Beyond that, the more punctured images provided, the higher the accuracy of automated puncturing of the remaining images. To ensure high accuracy, a calibration method using both source and target images can be employed to puncture more images, resulting in more punctured images and thus improving the accuracy of automated software puncturing.

[0139] like Figure 5 As shown in the figure, this application embodiment also provides an image calibration device, the device including:

[0140] Image acquisition module 501 is used to acquire source image and image to be calibrated; the source image includes the identification information of control points;

[0141] The feature extraction module 502 is used to extract image features from the source image and the image to be calibrated, respectively, to obtain a first feature data and a second feature data set; the first feature data is the image feature corresponding to the source image; the second feature data set includes multiple second feature data, which are image features corresponding to a portion of the image to be calibrated.

[0142] The similarity comparison module 503 is used to determine similar feature data corresponding to the first feature data from multiple second feature data;

[0143] The location calibration module 504 is used to input the first feature data, similar feature data, and the mapping feature values ​​of the image to be calibrated into the location prediction model to obtain the first location information of the control point in the image to be calibrated; the mapping feature values ​​include parameter information associated with the image to be calibrated or the control point.

[0144] It should be noted that the image calibration device is the same as the image calibration method described above. All implementation methods in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effect.

[0145] Figure 6 The image provided in this application shows a schematic diagram of the hardware structure of the calibration device.

[0146] The image calibration device may include a processor 601 and a memory 602 storing computer program instructions.

[0147] Specifically, the processor 601 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0148] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where suitable, memory 602 may include removable or non-removable (or fixed) media. Where suitable, memory 602 may be internal or external to the image calibration device. In a particular embodiment, memory 602 is a non-volatile solid-state memory.

[0149] In a particular embodiment, memory 602 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.

[0150] The processor 601 implements any of the image calibration methods described in the above embodiments by reading and executing computer program instructions stored in the memory 602.

[0151] In one example, the image calibration device may further include a communication interface 603 and a bus 610. Wherein, as Figure 3 As shown, the processor 601, memory 602, and communication interface 603 are connected through bus 610 and complete communication with each other.

[0152] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0153] Bus 610 includes hardware, software, or both, that couples components of a picture-annotating device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 610 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0154] Furthermore, in conjunction with the image calibration methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the image calibration methods in the above embodiments.

[0155] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0156] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0157] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0158] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0159] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for calibrating an image, characterized in that, The method includes: Acquire source images and images to be calibrated; the source images include the identification information of the control points, and the source images are obtained by taking pictures of the control points at the locations where the control points are deployed; After obtaining the source image and the image to be calibrated, the process also includes: The source image is cropped while retaining the identification information of the control points; Image features are extracted from the source image and the image to be calibrated to obtain a first feature data and a second feature data set; the first feature data is the image feature corresponding to the source image; the second feature data set includes multiple second feature data, which are image features corresponding to a portion of the image to be calibrated; The step of extracting image features from the source image and the image to be calibrated to obtain a first feature data set and a second feature data set includes: Image feature extraction is performed on the source image to obtain the first feature data; The image to be calibrated is divided into multiple comparison regions; Image features are extracted from the multiple comparison regions to obtain multiple second feature data; From multiple second feature data, determine similar feature data that corresponds to the first feature data; The first feature data, the similar feature data, and the mapping feature value of the image to be calibrated are input into the location prediction model to obtain the first location information of the control point in the image to be calibrated; the mapping feature value includes parameter information associated with the image to be calibrated or the control point, and the associated parameter information is the location information or shape information of specific objects around the control point; The step of inputting the first feature data, the similar feature data, and the mapping feature values ​​of the image to be calibrated into the location prediction model to obtain the first location information of the control point in the image to be calibrated includes: The first feature data, the similar feature data, and the mapping feature values ​​of the image to be calibrated are input into the location prediction model; The second location information of the control point in the similarity comparison region is obtained by using the prediction layer of the location prediction model; the similarity comparison region is the comparison region corresponding to the similar feature data. The transformation layer of the location prediction model is used to convert the second location information of the control point in the similarity comparison area into the first location information of the control point in the image to be calibrated.

2. The image calibration method according to claim 1, characterized in that, The step of determining similar feature data corresponding to the first feature data from multiple second feature data includes: The first feature data and the second feature data group are input into the image comparison model; The image comparison model is used to compare the first feature data with multiple second feature data to obtain similar feature data; the similar feature data is the second feature data with the smallest feature comparison difference from the first feature data.

3. The image calibration method according to claim 2, characterized in that, The step of using the image comparison model to perform feature comparison between the first feature data and multiple second feature data to obtain similar feature data includes: The image comparison model is used to compare the first feature data with multiple second feature data to obtain multiple similarity values; the similarity values ​​are used to represent the degree of difference between the feature comparisons of the first feature data and the second feature data. Similar feature data are determined based on the similarity values ​​corresponding to multiple secondary feature data.

4. The image calibration method according to claim 3, characterized in that, The step of determining similar feature data based on the similarity values ​​corresponding to multiple second feature data further includes: The image comparison judgment value is determined based on the similarity values ​​corresponding to multiple second feature data respectively; If the image comparison determination value is greater than the determination threshold, the similarity feature data is the second feature data corresponding to the largest similarity value; If the image comparison determination value is less than the determination threshold, it is determined that the image to be calibrated does not contain the control point.

5. The image calibration method according to claim 1, characterized in that, In the multiple comparison regions obtained by dividing the image to be calibrated, at least two comparison regions partially overlap.

6. The image calibration method according to claim 1, characterized in that, The first location information includes pixel coordinates; after inputting the first feature data, the similar feature data, and the mapping feature values ​​of the image to be calibrated into the location prediction model to obtain the first location information corresponding to the control point in the image to be calibrated, the method further includes: Obtain the coordinate range of the image to be calibrated; Determine whether the pixel coordinates of the control point in the image to be calibrated are within the coordinate range; If the pixel coordinates of the control point are outside the coordinate range, it is determined that the image to be calibrated does not contain the control point.

7. A device for calibrating images, characterized in that, The device includes: The image acquisition module is used to acquire source images and images to be calibrated; the source image includes the identification information of the control points, and the source image is obtained by taking pictures of the control points at the locations where the control points are deployed; The cropping module is used to crop the source image while retaining the identification information of the control points; The feature extraction module is used to extract image features from the source image and the image to be calibrated, respectively, to obtain a first feature data and a second feature data group; the first feature data is the image feature corresponding to the source image; the second feature data group includes multiple second feature data, and the second feature data is the image feature corresponding to a part of the image to be calibrated; The feature extraction module is also used for: Image feature extraction is performed on the source image to obtain the first feature data; The image to be calibrated is divided into multiple comparison regions; Image features are extracted from the multiple comparison regions to obtain multiple second feature data; A similarity comparison module is used to determine similar feature data corresponding to the first feature data from multiple second feature data; The location calibration module is used to input the first feature data, the similar feature data, and the mapping feature value of the image to be calibrated into the location prediction model to obtain the first location information of the control point in the image to be calibrated; the mapping feature value includes parameter information associated with the image to be calibrated or the control point, and the associated parameter information is the location information or shape information of a specific object around the control point; The position calibration module is also used for: The first feature data, the similar feature data, and the mapping feature values ​​of the image to be calibrated are input into the location prediction model; The second location information of the control point in the similarity comparison region is obtained by using the prediction layer of the location prediction model; the similarity comparison region is the comparison region corresponding to the similar feature data. The conversion layer of the location prediction model is used to convert the second location information of the control point in the similarity comparison area into the first location information of the control point in the image to be calibrated.

8. A device for calibrating images, characterized in that, The image calibration device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the image calibration method as described in any one of claims 1-6.

9. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the image calibration method as described in any one of claims 1-6.

Images