Empirical photo conformity judgment method and device, equipment and medium

Abstract

The invention provides a proof photo conformity judgment method. The proof photo conformity judgment method comprises the steps of obtaining a reported land class and n proof photos matched with the reported land class; using a recognition model to perform land type recognition on the n proof photos, and averaging recognition results to obtain a quasi-recognition result, the quasi-recognition result including r prediction probabilities indicating r land types; judging the confidence coefficient of the quasi-recognition result, if the quasi-recognition result is credible, constructing a nearest land class set by using k land classes indicated by the maximum k prediction probabilities in the quasi-recognition result, and judging whether the reported land class falls into the nearest land class set, and if so, proving that the photo is consistent with the reported land class; and if not, turning to manual auditing. According to the method, land type identification is directly carried out on the proof photo through the identification model, the manual workload can be reduced, the conformity judgment efficiency is improved, the pattern spot land type is determined based on a group of proof photo identification results, judgment is carried out by utilizing the closest land type set, and the conformity judgment accuracy can be improved.

Description

Technical Field

[0001] This invention relates to the field of land use change survey technology, and in particular to a method, apparatus, equipment and medium for judging the conformity of evidence photographs. Background Technology

[0002] Land use change surveys are an important means of supervising and managing the use of land resources. These surveys involve annual nationwide investigations, verifications, and analyses of land use types, land ownership, and land management information to achieve tracking and management of land resources. In land use change survey operations, the receiving party needs to interpret the supporting photographs of the land features reported by the local survey department to verify whether the types of land features captured in the photographs are consistent with the reported types of land features.

[0003] Previously, the conformity assessment of evidence photos was mainly conducted manually, resulting in a large workload and long processing time. Currently, there are methods using artificial intelligence to assist in the conformity assessment of evidence photos, but these are only applicable to evidence photos that meet certain quality requirements. They require prior screening of the evidence photos, which increases computational and time costs and reduces efficiency. Summary of the Invention

[0004] This invention provides a method for judging the conformity of evidence photos. By recognizing the category of evidence photos through a recognition model, the manual workload can be reduced. At the same time, by directly using the information of each evidence photo in a set of evidence photos for land category recognition, the efficiency and accuracy of evidence photo conformity judgment can be improved simultaneously.

[0005] According to the first aspect, a method for determining the conformity of evidentiary photographs is provided, characterized in that it includes:

[0006] S1: Obtain the reported land type and n evidence photos matching the reported land type, where n is a positive integer;

[0007] S2: Use the recognition model to identify the land use category of each of the n evidence photos, and obtain n recognition results;

[0008] S3: Calculate the average of the n identification results to obtain a quasi-identification result, wherein the quasi-identification result includes r predicted probabilities, and the r predicted probabilities respectively indicate r land types, where r is a positive integer;

[0009] S4: Determine the confidence level of the quasi-identification result. If the quasi-identification result is reliable, proceed to S5; if the quasi-identification result is unreliable, proceed to manual review.

[0010] S5: Construct the closest land category set using the k land categories indicated by the largest k predicted probabilities in the quasi-identification results, and determine whether the reported land category falls into the closest land category set. If yes, determine that the evidence photo is consistent with the reported land category; otherwise, proceed to manual review, where k∈[1,r].

[0011] According to the second aspect, a device for determining the conformity of evidentiary photographs is provided, characterized in that it includes:

[0012] The acquisition module is used to acquire the reported land type and n evidence photos that match the reported land type, where n is a positive integer;

[0013] The land category identification module is used to identify the land category of each of the n evidence photos using an identification model, and obtain n identification results;

[0014] The averaging module is used to perform an averaging operation on the n recognition results to obtain a quasi-recognition result, wherein the quasi-recognition result includes r predicted probabilities, and the r predicted probabilities respectively indicate r land types, where r is a positive integer;

[0015] The confidence level judgment module is used to judge the confidence level of the quasi-identification result. If the quasi-identification result is credible, the conformity judgment module is executed; if the quasi-identification result is unreliable, it is transferred to manual review.

[0016] The conformity judgment module is used to construct the closest land category set using the k land categories indicated by the largest k predicted probabilities in the quasi-identification results, and to determine whether the reported land category falls into the closest land category set. If so, the evidence photo is determined to be consistent with the reported land category; otherwise, it is transferred to manual review, where k∈[1,r].

[0017] According to a third aspect, an electronic device is provided, characterized in that the electronic device comprises:

[0018] The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the above-described method for determining the conformity of the evidence photographs.

[0019] According to a fourth aspect, a computer-readable storage medium is provided, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the steps of the above-described method for judging the conformity of evidence photos.

[0020] The beneficial effects of the present invention include at least the following:

[0021] 1. This invention uses an identification model to identify the land use category of the evidence photos, which can reduce the amount of manual work and directly use the evidence photos without pre-screening them, thus improving the efficiency of conformity judgment.

[0022] 2. This invention obtains a quasi-identification result by averaging the identification results of a set of evidence photos. It can make full use of the information of each evidence photo in a set of evidence photos to determine the land type of the land parcel, reduce errors caused by inaccurate identification based on a single photo, greatly improve the accuracy of land type identification of evidence photos, and thus improve the accuracy of conformity judgment. At the same time, by combining the judgment with the closest land type set, the accuracy of conformity judgment is further improved.

[0023] 3. This invention further assesses the confidence level of the alignment recognition results, avoiding compliance judgments based on unreliable quasi-identification results, thus ensuring the accuracy and reliability of the compliance judgment. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating a method for determining the conformity of photographs used as evidence according to the present invention.

[0026] Figure 2 This is a schematic diagram of the frame of a photographic conformity judgment device according to the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0028] Figure 1 The diagram illustrates a flowchart of a method for determining the conformity of evidentiary photographs provided by the present invention. This method includes:

[0029] S1: Obtain the reported land type and n evidence photos matching the reported land type, where n is a positive integer;

[0030] S2: Use the recognition model to identify the land use category of each of the n evidence photos, and obtain n recognition results;

[0031] S3: Calculate the average of the n identification results to obtain a quasi-identification result, wherein the quasi-identification result includes r predicted probabilities, and the r predicted probabilities respectively indicate r land types, where r is a positive integer;

[0032] S4: Determine the confidence level of the quasi-identification result. If the quasi-identification result is reliable, proceed to S5; if the quasi-identification result is unreliable, proceed to manual review.

[0033] S5: Construct the closest land category set using the k land categories indicated by the largest k predicted probabilities in the quasi-identification results, and determine whether the reported land category falls into the closest land category set. If yes, determine that the evidence photo is consistent with the reported land category; otherwise, proceed to manual review, where k∈[1,r].

[0034] Step S1 is described in detail based on an embodiment of the present invention.

[0035] Specifically, the verification database is retrieved, and from the database, a reported land category t and n supporting photographs f1, f2, ..., f3 corresponding to the reported land category t are retrieved. n , where n is a positive integer.

[0036] The verification database refers to the database submitted by local authorities for internal verification during land use change surveys. It stores attribute data for multiple land parcels and a set of photographs of the parcels to be used for evidence. The attribute data for each parcel includes the reported land type and coordinate location information. A land parcel refers to land use information for a specific area obtained from remote sensing imagery acquired using satellite imagery technology; parcels within a single area share the same land use attributes.

[0037] Reported land categories refer to the land categories of map features reported by local authorities. Assume there are a total of *r* land categories for map features in the land use change survey, where *r* is a positive integer. For example, land categories include 14 types: "paddy fields," "water surfaces," "irrigated land," "marshland," "garden land," "grassland," "mangroves," "tidal flats," "bulwark areas," "areas with unfinished demolitions," "photovoltaic panel areas," "construction land," "facility agricultural land and factory farming land," and "unused land." Correspondingly, each reported land category belongs to one of these 14 types.

[0038] Evidence photographs are photos taken by investigators on-site corresponding to the land parcels. They serve as evidence for the reported land use category of each parcel, and each reported land use category has a matching set of evidence photographs. Generally, the initial set of evidence photographs matching each reported land use category includes photographs taken from multiple different angles around the perimeter of the parcel, facing the center of the parcel. For example, multiple photographs taken from eight azimuth angles (the four vertices and the centers of the four sides) towards the center of the parcel. It should be understood that the number of evidence photographs (n) matching different reported land use categories is not necessarily the same.

[0039] Step S2 is described in detail based on an embodiment of the present invention.

[0040] Specifically, the recognition model is used to analyze n evidence photos f1, f2, ..., f n Perform feature identification to obtain n evidence photos f1, f2, ..., f n The n recognition results c1, c2, ..., c are in one-to-one correspondence. n The identification results are probability vectors, with each result including r identification probabilities. Each probability indicates a land type, representing the probability that the evidence photo belongs to that type. r is the total number of land types, and the r identification probabilities represent the probability that the evidence photo belongs to each of the r land types, where each land type refers to a primary land type. The total number of land type types r can be determined based on the actual change survey requirements, for example, 14.

[0041] The recognition model is implemented through a visual language model. As an optional implementation, the recognition model employs the second version of the Self-Distillation with No Labels (DINO) model (DINO-V2). DINO is a self-supervised learning network based on the Transformer architecture, which learns visual feature representations through self-supervised training on unlabeled images. DINO-V2 is trained on top of DINO with more data and more techniques. DINO-V2 uses unsupervised learning methods to train the image encoder on large image datasets, obtaining visual features with universal semantics, applicable to various image distributions and tasks.

[0042] The DINO-V2 model is trained using evidence photos to obtain a recognition model. Specifically, a subset of high-quality evidence photos are selected from the verification database, and after data labeling, tagged evidence photos are obtained and stored in the sample database. The DINO-V2 model is then trained using the evidence photos in the sample database to obtain the trained DINO-V2 model as the recognition model. The method for training the DINO-V2 model using evidence photos can be found in relevant technical documents and will not be elaborated here.

[0043] This invention uses a recognition model to identify the land use category of evidence photos, which can automatically classify the evidence photos, reducing manual workload and time costs.

[0044] Based on an embodiment of the present invention, step S3 will be described in detail.

[0045] Specifically, for n recognition results c1, c2, ..., c n The quasi-recognition result c is obtained by performing an averaging operation. a The calculation formula is as follows:

[0046]

[0047] Understandably, since the recognition result is a probability vector, averaging is done by averaging n probability vectors, and the resulting near-recognition result is also a probability vector. Each recognition result includes r recognition probabilities, and the averaging operation specifically includes:

[0048] The average of the n identification probabilities indicating the same land type from the n identification results is used to obtain a predicted probability; for r land types, r predicted probabilities p1, p2, ..., p3 are obtained respectively indicating the r land types. r Given r predicted probabilities p1, p2, ..., p r The quasi-identification result c constitutes a .

[0049] This invention directly identifies the land use category of a set of evidence photos by using an identification model, eliminating the need for complex screening processes such as coverage judgment and quality evaluation of the evidence photos beforehand. This saves computing power and time and improves the efficiency of conformity judgment.

[0050] This invention obtains a quasi-identification result by averaging the identification results of a set of evidence photos. Based on the quasi-identification result obtained from the common evidence photos, the land type of the corresponding patch is determined. This invention can make full use of the information of each evidence photo to obtain a more accurate land type identification result, which greatly improves the accuracy of land type identification of evidence photos.

[0051] Since the recognition model may make misjudgments, in order to avoid affecting the accuracy by making conformity judgments based on incorrect recognition results, this invention first performs a confidence judgment on the quasi-recognition result before using the quasi-recognition result, and provides a method for confidence judgment.

[0052] According to one embodiment of the present invention, in step S4, determining the confidence level of the quasi-identification result includes:

[0053] Obtain the maximum predicted probability among the r predicted probabilities, and determine whether the maximum predicted probability is greater than or equal to the threshold probability. If it is, the quasi-identification result is determined to be reliable; otherwise, the quasi-identification result is determined to be unreliable.

[0054] Specifically, regarding the quasi-identification result c a r predicted probabilities p1, p2, ..., p r Sort the data and select the one with the highest predicted probability p from the sorting results. max And determine the maximum prediction probability p max Is the probability p greater than or equal to the threshold? ref If yes, the quasi-identification result is deemed credible and proceeds to the next step; otherwise, the quasi-identification result is deemed unreliable and proceeds to manual review.

[0055] Wherein, the threshold probability p ref This is a pre-set probability threshold, ranging from (0,1), which can be set according to actual needs. Preferably, it is set to 0.5. The manual review method and steps are described below.

[0056] This invention further assesses the confidence level of the alignment recognition results, avoiding compliance judgments based on unreliable quasi-identification results, thus ensuring the accuracy of compliance judgments.

[0057] According to one embodiment of the present invention, step S5 includes:

[0058] S51: Obtain the k largest predicted probabilities in the quasi-identification results, and associate them with the k land types indicated by the k largest predicted probabilities. Use the k land types to construct the closest land type set.

[0059] Regarding the quasi-identification result c a r predicted probabilities p1, p2, ..., p r Sort the probabilities from largest to smallest and obtain the top k predicted probabilities p. m1 ,p m2 …,p mk It associates the k land categories d1, d2, ..., d with the k predicted probabilities that are ranked first. k Using the k land types, construct the set of closest land types {d1, d2, ..., dk}. k}, where k∈[1,r]. Preferably, k is 3.

[0060] S52: Determine whether the reported land type falls within the set of closest land types. If yes, determine that the evidence photo is consistent with the reported land type. If not, proceed to manual review, where k∈[1,r].

[0061] Specifically, determine whether the reported land type t falls into the nearest land type set {d1, d2, ..., d}. k If so, then determine whether the reported land type t matches the corresponding n evidence photos f1, f2, ..., f n If the compliance is consistent, then proceed to manual review.

[0062] Understandably, the closest land category set is a set of the k land categories with the highest confidence level obtained after comprehensively judging the land feature characteristics of n evidence photos. If the reported land category falls into the closest land category set, it means that most of the n evidence photos or the n evidence photos as a whole can correctly represent the land category of the patch. The land category of the patch reflected by the group of evidence photos as a whole is consistent with the reported land category, so it can be considered that the conformity is consistent.

[0063] This invention, based on obtaining a quasi-identification result by averaging n identification results, combines the construction of a set of the closest land types corresponding to the k land types with the largest predicted probabilities in the quasi-identification results, and judges the conformity by whether the reported land type falls into this set, thereby further improving the accuracy of conformity judgment.

[0064] According to one optional implementation method, manual review includes: manually determining the true land type of the land parcel based on n evidence photos, comparing the consistency between the reported land type, the quasi-identification result and the true land type, and further processing based on the comparison results.

[0065] Specifically, the actual land use type of the land parcel is determined manually based on n evidence photos; the reported land use type is compared with the actual land use type to determine if they are the same, and the land use type indicated by the accurate identification result is compared with the actual land use type; further processing is carried out based on the comparison results.

[0066] According to one optional implementation, further processing based on the comparison results includes:

[0067] 1) If the reported land type is the same as the actual land type, but the quasi-identification result is different from the actual land type, then it is determined that the reported land type is consistent with the n evidence photos, and the identification model has difficulty identifying the evidence photos. Then, the n evidence photos are manually labeled and added to the supplementary sample library.

[0068] 2) If the reported land category is different from the actual land category, but the quasi-identification result is the same as the actual land category, then it is determined that the reported land category is inconsistent with the n evidence photos and the reported land category is incorrect. In this case, the reported land category in the verification database is manually corrected. At the same time, the evidence photos can be tagged and added to the supplementary sample database.

[0069] 3) If the reported land type is different from the actual land type, and the quasi-identification result is different from the actual land type, then it is determined that the reported land type is inconsistent with the n evidence photos, and the identification model has difficulty identifying the evidence photos, and the reported land type is incorrect. In this case, the reported land type in the verification database is manually corrected, and the evidence photos are tagged and added to the supplementary sample database.

[0070] By labeling the evidence photos after manual review and storing them in a supplementary sample library, the recognition model can be further trained and updated to improve its generalization ability and robustness.

[0071] Please see Figure 2 In one embodiment of the present invention, an evidentiary photograph conformity judgment device 200 is provided, characterized in that the device comprises:

[0072] The acquisition module 201 is used to acquire the reported land type and n evidence photos matching the reported land type, where n is a positive integer;

[0073] Land category identification module 202 is used to identify the land category of each of the n evidence photos using an identification model, and obtain n identification results;

[0074] The averaging module 203 is used to perform an averaging operation on the n recognition results to obtain a quasi-recognition result, wherein the quasi-recognition result includes r predicted probabilities, and the r predicted probabilities respectively indicate r land types, where r is a positive integer;

[0075] The confidence level judgment module 204 is used to judge the confidence level of the quasi-identification result. If the quasi-identification result is credible, the compliance judgment module 205 is executed; if the quasi-identification result is unreliable, the process is transferred to manual review.

[0076] The conformity judgment module 205 is used to construct the closest land type set using the k land types indicated by the largest k predicted probabilities in the quasi-identification results, and to determine whether the reported land type falls into the closest land type set. If so, the evidence photo is determined to be consistent with the reported land type; otherwise, it is transferred to manual review, where k∈[1,r].

[0077] This invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the method described in any of the foregoing method embodiments. Further, the electronic device further includes: at least one input device and at least one output device. The memory, processor, input device, and output device are connected via a bus. Specifically, the input device may be a camera, touch panel, physical button, or mouse, etc. The output device may specifically be a display screen. The memory may be high-speed random access memory (RAM) or non-volatile memory, such as disk storage. The memory stores a set of executable program code, and the processor is coupled to the memory.

[0078] This invention also provides a computer-readable storage medium, which may be disposed in the electronic device described in the foregoing embodiments. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of the foregoing method embodiments. Further, the computer-readable storage medium may also be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other medium capable of storing program code.

[0079] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0080] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining the conformity of evidentiary photographs, characterized in that, include: S1: Obtain the reported land type and n evidence photos matching the reported land type, where n is a positive integer; S2: Use the recognition model to identify the land use category of each of the n evidence photos, and obtain n recognition results; S3: Calculate the average of the n identification results to obtain a quasi-identification result, wherein the quasi-identification result includes r predicted probabilities, and the r predicted probabilities respectively indicate r land types, where r is a positive integer; S4: Determine the confidence level of the quasi-identification result. If the quasi-identification result is reliable, proceed to S5; if the quasi-identification result is unreliable, proceed to manual review. S5: Construct the closest land category set using the k land categories indicated by the largest k predicted probabilities in the quasi-identification results, and determine whether the reported land category falls into the closest land category set. If yes, determine that the evidence photo is consistent with the reported land category; otherwise, proceed to manual review, where k∈[1,r].

2. The method according to claim 1, characterized in that, The identification model was obtained by training the second version of the unlabeled self-distillation model using evidence photos.

3. The method according to claim 1, characterized in that, The step of averaging the n recognition results to obtain the quasi-recognition result includes: The average of the n identification probabilities indicating the same land type from the n identification results is used to obtain a predicted probability. For each of the r land types, r predicted probabilities are obtained, and the quasi-identification result is constituted by the r predicted probabilities.

4. The method according to claim 1, characterized in that, The confidence level for determining the quasi-identification result includes: Obtain the maximum predicted probability among the r predicted probabilities, and determine whether the maximum predicted probability is greater than or equal to the threshold probability. If it is, the quasi-identification result is reliable; otherwise, the quasi-identification result is unreliable.

5. The method according to claim 4, characterized in that, The threshold probability is 0.

5.

6. The method according to claim 1, characterized in that, The value of k is 3.

7. The method according to claim 1, characterized in that, The evidence photos that have undergone manual review are tagged and stored in a supplementary sample library, which is used to continue training the recognition model.

8. A device for determining the conformity of evidentiary photographs, characterized in that, include: The acquisition module is used to acquire the reported land type and n evidence photos that match the reported land type, where n is a positive integer; The land category identification module is used to identify the land category of each of the n evidence photos using an identification model, and obtain n identification results; The averaging module is used to perform an averaging operation on the n recognition results to obtain a quasi-recognition result, wherein the quasi-recognition result includes r predicted probabilities, and the r predicted probabilities respectively indicate r land types, where r is a positive integer; The confidence level judgment module is used to judge the confidence level of the quasi-identification result. If the quasi-identification result is credible, the conformity judgment module is executed; if the quasi-identification result is unreliable, it is transferred to manual review. The conformity judgment module is used to construct the closest land category set using the k land categories indicated by the largest k predicted probabilities in the quasi-identification results, and to determine whether the reported land category falls into the closest land category set. If so, the evidence photo is determined to be consistent with the reported land category; otherwise, it is transferred to manual review, where k∈[1,r].

9. An electronic device, characterized in that, The electronic device includes: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-7.

Images