Entity matching method and entity matching program

Abstract

To provide a method or the like which can improve determination accuracy whether objects such as solar panels or the like reflected on each of plural picked-up images are the same.SOLUTION: Pair candidates for specified objects detected from respective picked-up images on which plural objects (solar panels P) are reflected are specified. Determination processing is executed with a pair of specified objects forming the pair candidates in order according to a first similarity as targets. The first similarity is the similarity in a relative arrangement mode between the pair of respective specified objects in the pair of respective picked-up images and the other specified objects. In the determination processing, the pair of specified objects are determined as the same specified objects with a fact that a second similarity is greater than or equal to a threshold value as a requirement. The second similarity is the similarity of the specified attributes of the pair of respective specified objects. Further, if a contradiction arises even when the requirements are satisfied, the pair of specified objects are determined as different specified objects.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to a technology for identifying objects that appear in a plurality of captured images. [Background technology]

[0002] A technique has been proposed for determining whether or not there is an abnormality in each of a plurality of photovoltaic power generation panels (or modules) arranged vertically and horizontally in a matrix (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-117466 Summary of the Invention [Problem to be solved by the invention]

[0004] To identify panels determined to have an abnormality (abnormal panels), for example, aerial photographs of multiple aligned panels are converted into orthoimages, and icons indicating the abnormal panels are superimposed on the orthoimages. Specifically, as shown in the upper part of Figure 12, multiple images img1, img2, ... obtained by overlapping photography are converted into multiple orthoimages, and the multiple orthoimages are aligned and integrated to generate an integrated image IMG, as shown in the lower part of Figure 12. In this case, as shown in the lower part of Figure 12, the positions of the same abnormal panels in the integrated image may be shifted, and multiple icons or marks M1, M2, ... indicating the same abnormal panel may be displayed side by side.

[0005] Therefore, an object of the present invention is to provide a method etc. that can improve the accuracy of determining whether or not an object such as a solar panel reflected in each of a plurality of captured images is the same. [Means for solving the problem]

[0006] The name matching method of the present invention comprises: Identifying, among primary pair candidates for a designated object detected as an object having a designated attribute from each of a pair of photographed images among a plurality of photographed images in which a plurality of objects are reflected, secondary pair candidates for the designated object whose distance is equal to or less than a first predetermined distance; calculating a first similarity which is a similarity between a relative arrangement of each of the pair of designated objects constituting the second pair candidates in each of the pair of captured images and other designated objects, and a second similarity which is a similarity between the designated attributes of each of the pair of designated objects constituting the second pair candidates; A pair of designated objects constituting the secondary pair candidates is targeted in an order according to the first similarity, and the pair of designated objects is determined to be the same designated object, provided that the second similarity is equal to or greater than a threshold value. However, if a contradiction occurs in which different designated objects extracted from one captured image are determined to be the same designated object even when the requirement is met, a determination process is executed to determine that the pair of designated objects constituting the secondary pair candidates are different designated objects.

[0007] According to the name matching method of the present invention, among primary pair candidates of designated objects detected as objects having a designated attribute from each of a pair of photographed images among a plurality of photographed images that include a plurality of objects (including a plurality of aligned objects and a plurality of irregularly arranged objects), secondary pair candidates of designated objects whose distance is equal to or less than a first predetermined distance are identified. This prevents a pair of designated objects that are unlikely to be the same object in view of the distance from each other from being determined to be the same object.

[0008] In the determination process, the pair of designated objects is determined to be the same designated object if the second similarity is equal to or greater than a threshold. The "second similarity" is the similarity of the designated attributes of the pair of designated objects constituting the secondary pair candidate. This prevents a pair of designated objects that are unlikely to be the same object in light of the similarity of the designated attributes from being determined to be the same object. Furthermore, even if the requirement is met, if a contradiction occurs in which different designated objects extracted from a single captured image are determined to be the same designated object, the pair of designated objects is determined to be different designated objects.

[0009] The determination process is performed on pairs of designated objects constituting secondary pair candidates in order according to the first similarity. The "first similarity" is the similarity between the relative placement of each of the pair of designated objects constituting the secondary pair candidates in each of the pair of captured images and other designated objects. This makes it possible to avoid a situation where a pair of designated objects that is highly likely to be the same object in light of the similarity in the designated attributes is determined to be the same object, while a pair of designated objects that would cause such a contradiction is determined to be the same object.

[0010] As a result, it is possible to improve the accuracy of determining whether or not the object, such as a solar panel, reflected in each of the multiple captured images is the same. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating the configuration of a name matching system. [Figure 2] 1 is a flowchart showing the steps of a name matching method. [Figure 3] FIG. 10 is an explanatory diagram regarding the selection process of a primary panel pair. [Figure 4] FIG. 10 is an explanatory diagram regarding the selection process of a secondary panel pair. [Figure 5] Diagram illustrating the selection process for the second panel pair (continued). [Figure 6] FIG. 10 is an explanatory diagram regarding an evaluation process of an abnormal similarity. [Figure 7]FIG. 10 is an explanatory diagram regarding the evaluation result of the abnormal similarity. [Figure 8] FIG. 1 is an explanatory diagram regarding the definition of an RPI vector. [Figure 9] FIG. 10 is an explanatory diagram of an example of an RPI vector. [Figure 10] FIG. 10 is an explanatory diagram regarding pairing processing of an abnormal panel. [Figure 11] FIG. 10 is an explanatory diagram of the name matching results of the abnormal panel. [Figure 12] FIG. DETAILED DESCRIPTION OF THE INVENTION

[0012] (Configuration of the name matching system) A name matching method according to one embodiment of the present invention is implemented using a name matching system that is composed of a name matching device 10 shown in Figure 1, an unmanned airplane 20, and a camera 22 mounted on the unmanned airplane 20.

[0013] The name matching device 10 is composed of an information processing terminal or computer configured to be able to communicate with the unmanned airplane 20, and is configured so that an arithmetic processing device (such as a CPU) reads out data and a name matching program stored in a storage device (memory), and performs the name matching process described below on the data in accordance with the program.

[0014] (Preliminary processing of name matching method) While the unmanned airplane 20 is moving, the camera 22 mounted on the unmanned airplane 20 sequentially photographs or acquires from the air images imgn (n=1, 2, ..., N) that show multiple panels P (objects) that are arranged in an aligned manner (FIG. 2 / STEP 10). Note that multiple images img may be acquired using one or multiple cameras 22 mounted on a single unmanned airplane 20, or multiple images img may be acquired using one or multiple cameras 22 mounted on each of multiple unmanned airplanes 20. Images imgn that show other objects such as vehicles as multiple objects that are arranged in an aligned manner may also be acquired.

[0015] Image analysis of the captured image (img) determines whether or not there is an abnormality in each panel P, and also determines the arrangement pattern of the abnormal parts in each abnormal panel P (a panel determined to have an abnormality) (Fig. 2 / STEP 12).

[0016] As a result, for example, as shown in the upper left panel (1) of Figure 7, abnormal panels P1 and P2 are detected in the captured image img1. Furthermore, as shown in the lower left panel (2) of Figure 7, the arrangement pattern of abnormal areas (white cells) in each of the abnormal panels P1 and P2 (e.g., a 16x16 cell area) is identified. Similarly, as shown in the upper right panel (3) of Figure 7, abnormal panels P3 and P4 are detected in the captured image img2. Furthermore, as shown in the lower right panel (4) of Figure 7, the arrangement pattern of abnormal areas (white cells) in each of the abnormal panels P3 and P4 (e.g., a 16x16 cell area) is identified.

[0017] Based on the latitude and longitude of the unmanned airplane 20 measured by the positioning device mounted on the unmanned airplane 20, the latitude and longitude of a representative point (e.g., the center) of each captured image img are determined (Fig. 2 / STEP 13). Furthermore, based on the latitude and longitude of the unmanned airplane 20, the latitude and longitude of the abnormal panel P reflected in each captured image img are determined (Fig. 2 / STEP 14).

[0018] An image identifier (imgID) for identifying each individual captured image is assigned to each of the multiple captured images (FIG. 2 / STEP 15). Furthermore, a panel identifier (globalID) for identifying each individual abnormal panel P is assigned to each abnormal panel P (FIG. 2 / STEP 16).

[0019] As a result, an M×N sparse matrix A can be defined in which True is entered in the components corresponding to the combination of the image identifier imgID of the captured image img and the panel identifier globalID of the abnormal panel P detected in the captured image, as shown in the simulation in Figure 3. "N" is the total number of captured images img. "M" is the total number of abnormal panels P.

[0020] (Procedure for name matching method) First, from multiple captured images imgn, the distance of the representative point (= {(latitude deviation) 2 +(longitude deviation) 2} 1 / 2 A pair of captured images in which the distance between the first and second predetermined distances is equal to or less than the first specified distance (second predetermined distance (for example, 50 m)) are extracted as a captured image pair (FIG. 2 / STEP 20).

[0021] This allows for the definition of a sparse matrix B in which True is input into components corresponding to the combination of the image identifiers imgID of a pair of captured images that make up a captured image pair, as shown in the schematic diagram of Fig. 3. For example, if a captured image with image identifier imgID=n1 and a captured image with image identifier imgID=n2 make up a captured image pair, "True" is input into the (n1, n2) component of the sparse matrix B.

[0022] Next, a pair of abnormal panels P in each of a pair of captured images img constituting the captured image pair is identified as a "primary panel pair candidate" (FIG. 2 / STEP 21).

[0023] As shown diagrammatically in Figure 3, a sparse matrix A and its transpose t A and sparse matrix B are used to define a sparse matrix C=A·B· tA pair of abnormal panels P identified by a combination of panel identifier globalIDs corresponding to components with a value of True in A are identified as "primary panel pair candidates." When performing numerical calculations, True is replaced with the specific numerical value 1.

[0024] Furthermore, from among the primary panel pair candidates, a primary panel pair consisting of a pair of abnormal panels P whose distance is less than or equal to a second specified distance (first predetermined distance (e.g., 7.5 m)) is selected as a "secondary panel pair candidate" (Figure 2 / STEP 22).

[0025] As shown in Fig. 4(1), the sparse matrix C (upper triangular matrix (see Fig. 3)) and its transpose matrix t C (lower triangular matrix) combination matrix (C+ t C) and a vector γ representing the latitude of each panel identified by the panel identifier globalID. lat The Hadamard product is calculated with the vector γ(M×1), after expanding the values ​​of each row in the column direction so that the number of columns becomes M. The Hadamard product is calculated with the matrix (C+ t In (C), the elements with the value "True" are shown in black in Figure 4(1). This results in the M × M sparse matrix Γ shown in Figure 4(2). lat The matrix Γ is obtained. lat The (m1,m2) elements that have non-zero values ​​in are shown in black in Figure 4(2). lat The (m1, m2) component of the abnormal panel P identified by the panel identifier globalID=m1 m1 As shown in Fig. 4(2), the sparse matrix Γ lat and its transpose matrix t Gamma lat The difference matrix (Γ lat - t Gamma lat ) is calculated. lat - t Gamma lat The (m1, m2) component of the (m1, m2) element is the abnormal panel P identified by the panel identifier globalID=m1.m1 and the anomalous panel P identified by the panel identifier globalID=m2. m2 As shown in Figure 4(3), the difference matrix (Γ lat - t Gamma lat ) as the value of each element. lat For longitude, the same procedure as for latitude is used to obtain the matrix Δ lng is obtained.

[0026] As shown in Fig. 5(4), the matrix Δ lat and Δ lng By adding these together, the abnormal panel P m1 and P m2 The sum of the square of the latitude deviation and the square of the longitude deviation (i.e., the anomaly panel P m1 and P m2 The matrix Δ has the (m1,m2) element value as lat +Δ lng Among the (m1, m2) elements of the matrix shown in the simulation in FIG. 5(4), elements having a value equal to or less than the square of the second specified distance are selected. As a result, a matrix Δ having the selected elements is obtained, as shown in the simulation in FIG. 5(5). Then, only the upper triangular elements of the matrix Δ are left, and as shown in the simulation in FIG. 5(6), the upper triangular matrix Δ is obtained. upper The upper triangular matrix Δ upper A pair of abnormal panels P corresponds to the (m1, m2) components (shown in black) that have non-zero values ​​in m1 and P m2 This will form the "secondary panel pair candidates."

[0027] The abnormal similarity S of a pair of abnormal panels P that constitutes a secondary panel pair candidate ab is evaluated or calculated (Figure 2 / STEP 23). The abnormality similarity is defined in the range of 0 to 1, and the closer to "1" the more similar the abnormal patterns (arrangement patterns of abnormal areas) of a pair of abnormal panels P are, the closer it is to "1."

[0028] The pair of abnormal panels P that constitute the secondary panel pair candidates shown in Figure 6(1) m1 and P m2 An example of an evaluation method for the abnormal similarity of the abnormal panel P will be described. m1 and P m2 The normal regions (corresponding to normal cells) in each of the m1 ) and F(P m2 ) (corresponding to cells where an abnormality has occurred) are represented by white areas. In this example, m1 and P m2 is represented as an 8x8 cell region, it may also be represented by a cell region of a different size (and / or shape), such as a 16x16 cell region.

[0029] First, abnormal panel P m1 and P m2 Then, zero padding is performed on each of the panels. As a result, the abnormal panel P m1 and P m2 The abnormal panel P m1 and P m2 By expanding each of these, the expanded abnormal panel P m1 _E and P m2 Each of _E is generated.

[0030] Next, the extended abnormal panel P m1 _E and P m2 For each of the vertices _E, a (3,3) dilation (MAX filter) process (or averaging process) is performed. As a result, for example, as shown in FIG. 6(3), the abnormal region F(P m1 ) and F(P m2 ) are expanded, and the abnormal region F(P m1 )_E and F(P m2 ) _E with each of the expanded abnormal panel P m1 _E and P m2Each of _E is generated.

[0031] Next, the filtered expanded abnormal panel P m1 _E and P m2 _E are integrated (stacked). This results in the expanded abnormal panel P m1 _E and P m2 The cells that were white in at least one of the cells UP_E are defined as white areas, and the remaining cells are defined as black areas to generate an integrated cell area UP_E.

[0032] Then, based on the integrated cell area UP_E, the abnormal panel P m1 and P m2 The anomalous similarity S ab The IoU (Intersection over Union) is calculated as follows. As a result, the white area (F(P m1 )_E∪F(P m2 )_E) for the number of cells constituting the area (=15) m1 _E and P m2 In both the white area (F(P m1 )_E∪F(P m2 )_E) and the ratio (=9 / 15) = 0.6 of the number of cells and area (=9) is abnormal panel P m1 and P m2 The anomalous similarity S ab Instead of IoU, the Jaccard coefficient is used to calculate the anomaly similarity S ab It may be calculated as:

[0033] Figure 7 shows a pair of abnormal panels P when the abnormal panel is represented by a 16x16 cell area. m1 and P m2 The anomalous similarity S ab The calculation results of (=IoU) are shown. One abnormal panel P shown in the upper left of Figure 7 (1) m1 The abnormal region F(P m1 ) and the other abnormal panel P m2The abnormal region F(P m2 ) (white area) are completely identical, so the abnormal similarity S ab The upper limit was "1". One of the abnormal panels P shown in the lower left part of Figure 7 (2) m1 The abnormal region F(P m1 ) and the other abnormal panel P m2 The abnormal region F(P m2 ) are relatively similar in placement pattern, so the anomalous similarity S ab The value of the abnormal panel P shown in the upper right panel (3) and the lower right panel (4) of Figure 7 was relatively high at 0.65. m1 The abnormal region F(P m1 ) and the other abnormal panel P m2 The abnormal region F(P m2 ) are not very similar, so the abnormal similarity S ab were relatively low at 0.05 and 0.17.

[0034] The similarity S of the relative position in an image (RPI) of a pair of abnormal panels that make up a secondary panel pair RPI is calculated (Figure 2 / STEP 24).

[0035] Specifically, first, the RPI vector η of each anomalous panel is defined according to a gravity model. According to the gravity model, each anomalous panel is regarded as a celestial body of unit mass, and the gravitational force that an anomalous panel receives from other anomalous panels in the vicinity is expressed by an RPI vector ("direction" and "magnitude"). The number of other anomalous panels may be adjusted so that they are equal to or less than a designated number selected in order of the shortest distance from the anomalous panel. For example, as shown in FIG. 8, if there are three anomalous panels P m0 , P m1 and P m2 If detected, an abnormal panel P m0 RPI vector η m0 I will explain how to calculate the abnormal panel P m0 From the representative point of other abnormal panel Pm1 , P m2 to the representative point of P, the vector η 01 and η 02 , and, based on their lengths (norms) |η 01 | and |η 02 |, according to relational expression (02), the RPI vector η m0 of a single abnormal panel P m0 is defined.

[0036] η m0 = η 01 / |η 01 | q+1 + η 02 / |η 02 | q+1 ‥(02).

[0037] The RPI vector is defined in the latitude-longitude coordinate system. As described above, for example, the latitude-longitude of the center of gravity of an abnormal panel is defined as the latitude-longitude of the abnormal panel. The component vectors of the RPI vector η m0 with the positions of two abnormal panels as the starting point and the ending point, respectively, η 01 and / or η 02 are defined to be inversely proportional to the q-th power (1 < q (for example, q = 2)) of the distance between the two abnormal panels.

[0038] Thus, for each of the abnormal panels P1, P2, and P3 detected in the captured image img1 as shown in the upper left of FIG. 9, the RPI vectors η1, η2, and η3 are defined. Similarly, for each of the abnormal panels P4 and P5 detected in the captured image img2 as shown in the upper right of FIG. 9, the RPI vectors η4 and η5 are defined. And the similarity of a pair of RPI vectors η m1 and η m2 constituting the secondary panel pair candidate is obtained as the RPI similarity S RPI .

[0039] Two RPI vectors η m1 and η m2The similarity of each "direction" is expressed as the cosine similarity S cos It is defined by

[0040] S cos =(η m1 η m2 ) / (|η m1 |·|η m2 |)‥(11).

[0041] As is clear from the relation (11), the cosine similarity S cos is in the range of -1 to 1, and the two RPI vectors η m1 and η m2 If the directions of the two RPI vectors η are the same, the value is "1" m1 and η m2 is "0" when the two RPI vectors η m1 and η m2 The two RPI vectors η are defined to be "-1" when the directions of η are opposite. m1 and η m2 If at least one of is a zero vector, the cosine similarity S cos is defined to be "0".

[0042] Two RPI vectors η m1 and η m2 The similarity of each "size" is expressed by the following relational expression (12), the magnitude similarity S mag It is defined by

[0043] S mag =(|η m1 |+|η m2 |) / 2max(|η m1 |,|η m2 |)‥(12).

[0044] As is clear from the relation (12), the magnitude similarity S mag is defined to be in the range of 0.5 to 1. The magnitude similarity S magSince the lower limit of RPI similarity S is set to 0.5, RPI The impact on has been adjusted.

[0045] In addition, one of the RPI vectors η m1 The norm of |η m1 | and the other RPI vector η m2 The norm of |η m2 Absolute value of deviation from ||η m1 |-|η m2 The magnitude similarity S is calculated so that it approaches 1 as || decreases. mag For example, the magnitude similarity S mag may be defined by the relation (121).

[0046] S mag =exp(-C||η m1 |-|η m2 ||) (0 <C)‥(121)。 RPI similarity S RPI is the cosine similarity S cos and magnitude similarity S mag is defined according to the relation (14) based on

[0047] S RPI =(S cos ×S mag +1) / 2 ‥(14).

[0048] As is clear from the relation (14), the RPI similarity S RPI is defined to be in the range 0 to 1.

[0049] RPI similarity S RPI is the cosine similarity S cos and magnitude similarity S mag may be defined according to the relation (141) as a weighted sum of

[0050] S RPI =αS cos +(1-α)S mag(0<α<1)‥(141).

[0051] Two indicators, a priority indicator PI and a decision indicator DI, are calculated as pairing indicators (Figure 2 / STEP 25).

[0052] The priority index PI is calculated by the anomaly similarity S ab and RPI similarity S RPI is defined according to the relation (21) based on

[0053] PI=S ab ×S RPI ‥(twenty one).

[0054] The priority index PI is calculated by the anomaly similarity S ab and RPI similarity S RPI may be defined according to the relation (211) as a weighted sum of

[0055] PI=βS ab +(1-β)S RPI (0<β<0.5)‥(211).

[0056] The priority index PI is calculated by the RPI similarity S RPI For example, the priority index PI may be defined according to the relational expression (212).

[0057] PI=S RPI ...(212).

[0058] The decision index DI (pair establishment decision index) is the anomaly similarity S ab For example, the decision index DI is defined according to the relation (22).

[0059] DI=S ab ‥(twenty two).

[0060] Based on the priority index PI and the decision index DI, a pairing process for abnormal panels is performed (FIG. 2 / STEP 26).

[0061] Here, we will explain the case where the RPI vectors η1 to η5 of the abnormal panels P1 to P5 are defined as shown in the upper part of Figure 9, and the arrangement pattern of the abnormal areas (white areas) of each of the abnormal panels P1, P2, P4, and P5 is obtained as shown in the lower part of Figure 9.

[0062] In this case, as shown in Figure 10(1), for each of the secondary panel pair candidates (P1, P4), (P1, P5), (P2, P4) and (P2, P5), the abnormal similarity S ab and RPI similarity S RPI is calculated. Furthermore, as shown in FIG. 10(2), a priority index PI and a determination index DI are calculated. Then, as shown in FIG. 10(3), the secondary panel pair candidates (P1,P4), (P1,P5), (P2,P4), and (P2,P5) are ranked in order of priority in the order of largest priority index PI: (P2,P5) → (P1,P4) → (P2,P4) → (P1,P5).

[0063] In this state, first, the threshold value DI of the decision index DI th For example, when the priority index PI is set to "0.75", pairing of abnormal panel candidates is attempted in descending order of priority index PI. The decision index DI (= 1.0) of the secondary panel pair candidates (P2, P5) is set to the threshold DI th Since the above results are true, the abnormal panels P2 and P5 are paired. The decision index DI (=0.5) of the abnormal panel pair candidate (P1, P4) is greater than the threshold DI th Since the determination index DI (=1.0) of the candidate abnormal panel pair (P2, P4) is less than the threshold DI thThat's all. However, if abnormal panels P2 and P4 are paired, the result will be contradictory, as P4 and P5, which are separate abnormal panels in the captured image img2, are the same panel, and the pairing will be rejected. In other words, the same abnormal panels are grouped together, and the formation of subsequent pairs that contradict existing groups is restricted. The decision index DI (=0.5) of the candidate abnormal panel pair (P1, P5) is below the threshold DI th Since the value is less than the threshold, the pairing of the abnormal panels P1 and P5 is suspended.

[0064] Next, the threshold value DI of the decision index DI th is lowered to, for example, "0.50", and in this situation, pairing of unpaired abnormal panel candidates is attempted in descending order of priority index PI. The decision index DI (=0.5) of the abnormal panel pair candidate (P1, P4) is lowered to the threshold DI th Since the above is true, the abnormal panels P1 and P4 are paired. The determination index DI (=0.5) of the abnormal panel pair candidate (P1, P5) is equal to or greater than the threshold DI th However, if the abnormal panels P1 and P5 are paired, the result will be contradictory, as the separate abnormal panels P1 and P2 in the captured image img1 are the same panel, and therefore the pairing will be rejected.

[0065] As a result, the candidate abnormal panel pairs (P2, P5) and (P1, P4) are paired and defined as an abnormal panel pair. The abnormal similarity S between the abnormal panel P2 in the photographed image img1 and the abnormal panel P4 in the photographed image img2 is ab (=IoU) is the abnormal similarity S between the abnormal panel P1 of the photographed image img1 and P4 of the photographed image img2, which are originally identical. ab However, since pairing is attempted in descending order of the priority index PI, it is possible to avoid pairing the abnormal panels P2 and P4 after the abnormal panels P1 and P4 have been paired.

[0066] As a result, for example, as shown in the upper part of Figure 11, the abnormal panels detected in each of the captured images img1, img2, and img3 were identified, and it was verified that the same panel identifier (abnormal ID) was assigned to the same abnormal panel. As shown in the lower part of Figure 11, the abnormal panels assigned the panel identifiers globalID "294," "295," "296," "297," "300," "301," "303," "305," and "306" respectively indicate the same panel. For example, globalID5 (from img1), globalID13 (from img4), and globalID129 (from img28) are identified as a single abnormal ID5.

[0067] (Action and effect) According to the name identification method of the present invention, misrecognition is unlikely to occur due to the following properties.

[0068] (1) Abnormal similarity S ab is unlikely to be high by random probability.

[0069] (2) RPI similarity S RPI However, there is a competitive nature between different captured images.

[0070] (3) High-reliability pairings are prioritized, and high-reliability groups are formed early on.

[0071] (4) The formation of subsequent pairings that would cause conflicts between groups is prevented.

[0072] A condition for achieving high accuracy is that different captured images are captured so that the overlapping area is large.

[0073] Situations in which misidentification may occur include situations in which all members of two different groups G1 and G2 are present within a range that is less than a second specified distance (e.g., 7.5 m), members of groups G1 and G2 do not coexist in the same captured image, and / or there is a combination of high DI within groups G1 and G2.

[0074] (Another embodiment of the present invention) In the above embodiment, the distance (= {(latitude deviation)}) of the representative point is calculated from the plurality of captured images imgn. 2 +(longitude deviation) 2} 1 / 2 ) is less than a first specified distance (e.g., 50 m) is extracted as a pair of captured images (see FIG. 2 / STEP 20). In another embodiment, a pair of captured images in which the overlap degree is equal to or greater than a specified degree may be extracted as a pair of captured images. In yet another embodiment, pairs of abnormal panels P in each of all pairs of captured images img may be identified as "primary panel pair candidates" regardless of the distance between the representative points (see FIG. 2 / STEP 21). In this case, to identify all pairs of abnormal panels P as primary panel pair candidates, simply set "all panels P in other images are applicable" or "use B in which all upper triangular components of B (excluding the diagonal components) are 1 (True)". Alternatively, set C=A· t Assuming A, C may be defined as a "non-primary panel pair candidate" (a candidate that will not be paired). This can be applied in situations where the positional relationship between objects is similar. For example, it is possible to match "objects flowing in a river" or "objects on a conveyor belt." In this case, the positional relationship of the objects (RPI similarity) may be defined by their position in the image (x-y coordinates) rather than latitude and longitude.

[0075] According to the above embodiment, a solar panel (abnormal panel P) is extracted as an object having an abnormality as a specified attribute as a matching target, but in other embodiments, an object such as a car having specified attributes such as "having a specified item (or item of a specified shape) attached" or "having a part of a specified color (or hue)" may be extracted as a matching target. [Explanation of symbols]

[0076] 10. Name matching device 20‥Unmanned aircraft 22. Camera P...Solar panel (abnormal panel).

Claims

1. Identifying, among primary pair candidates of a designated object detected as an object having a designated attribute from each of a pair of photographed images among a plurality of photographed images in which a plurality of objects are reflected, secondary pair candidates of the designated object whose distance is equal to or less than a first predetermined distance; calculating a first similarity which is a similarity between a relative arrangement mode of each of the pair of designated objects constituting the second pair candidates in each of the pair of captured images and other designated objects, and a second similarity which is a similarity between the designated attributes of each of the pair of designated objects constituting the second pair candidates, a determination process is executed in which, for a pair of the designated objects constituting the secondary pair candidates in the order according to the first similarity, the pair of designated objects is determined to be the same designated object, provided that the second similarity is equal to or greater than a threshold; and, if a contradiction occurs in which different designated objects extracted from one of the photographed images are determined to be the same designated object even when the requirement is satisfied, the pair of designated objects constituting the secondary pair candidates is determined to be different designated objects. Name matching method.

2. The method of claim 1 , Identifying the second pair candidate from among the first pair candidates of the specified object detected from each of a pair of photographed images constituting a photographed image pair, the distance between which is equal to or less than a second predetermined distance that is longer than the first predetermined distance, among the plurality of photographed images. Name matching method.

3. The method of claim 1 , The threshold is lowered and the determination process is repeated on the condition that the determination process has not been completed for all of the secondary pair candidates. Name matching method.

4. The method of claim 1 , The determination process is performed on the pair of designated objects constituting the second pair candidates in an order according to the second similarity in addition to the first similarity. Name matching method.

5. The method of claim 1 , A norm is defined such that the longer the distance to the other designated object in each of the pair of photographed images constituting the photographed image pair is, the shorter the norm is, and the similarity of one vector or a composite vector of multiple vectors directed from each of the pair of designated objects constituting the secondary pair candidate to the other designated object in each of the pair of photographed images constituting the photographed image pair is calculated as the first similarity. Name matching method.

6. The name identification method according to claim 5 , The number of the other designated objects is adjusted so that it is equal to or less than a designated number selected in order of shortest distance from each of the pair of designated objects constituting the second pair candidate. Name matching method.

7. The method of claim 1 , The specified attribute is the presence of an abnormality, and a similarity of a layout pattern of the abnormality in the specified object is calculated as the second similarity. Name matching method.

8. A name matching program that provides an information processing terminal with a function for implementing the name matching method according to any one of claims 1 to 7.

Images