Information processing device and information processing method and program

Abstract

To reduce the cost of correcting an offset between objects detected in images taken at different times.SOLUTION: The information processing device obtains first positional information that indicates a position of an object detected in a first image obtained by capturing the object and second positional information that indicates a position of the object detected in a second image obtained by capturing the object at a time different from that of the first image. The information processing device estimates the amount of positional deviation of the object based on the first positional information and the second positional information, and corrects at least one of the first positional information and the second positional information based on the amount of positional deviation.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In recent years, in order to improve the efficiency of inspection work for aging social infrastructure structures in Japan, the soundness of structures has been evaluated using inspection images of the inspection target. With this technology, damage on the inspection images is extracted as polylines (vector data) by workers tracing or by image recognition using machine learning, and the damage is evaluated based on its location, shape, size, number, and density, as well as the degree of progression through comparison over time.

[0003] For example, when the structure is a tunnel, the cross section in the direction of travel is often circular, and the lining surface on the curved surface along this circumference is the main inspection target. In order to efficiently obtain high-definition inspection images of a wide area such as the tunnel lining surface, there is a method in which a vehicle with multiple cameras facing the target is driven and the captured image data is synthesized by digital processing. This method is improving year by year, for example, by changing the camera used from a video camera to a higher-definition line sensor camera, and by introducing drones for imaging bridges and the like. Therefore, the imaging conditions such as the type, angle of view, or number of cameras may be different between the most recently acquired inspection image and the past inspection image to be compared over time. Even if these conditions are the same, if the movement path during imaging is different, the angle of view covered by each camera will differ. Therefore, there will be differences in the results of the processing to synthesize the captured images, and even if the same object is captured, there will be a position shift between the inspection images depending on the time of acquisition, making it difficult to compare the damage extracted from them.

[0004] As a method for correcting positional deviation between images of the same subject and enabling comparison, Patent Document 1 discloses a technique for aligning chest X-ray images in the medical field. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 7-37074 Summary of the Invention [Problem to be solved by the invention]

[0006] However, when evaluating the soundness of a structure, if the inspection images used for comparison are high-resolution captured images, the large amount of data will result in high processing and storage costs.

[0007] An object of the present invention is to reduce the cost of correcting misalignment between objects detected in images captured at different times. [Means for solving the problem]

[0008] In order to achieve the object of the present invention, for example, an information processing device according to one embodiment has the following configuration: An acquisition means for acquiring first position information indicating a position of an object detected from a first image capturing an object and second position information indicating a position of the object detected from a second image capturing the object at a time different from that of the first image, an estimation means for estimating a positional shift amount of the object based on the first position information and the second position information, and a correction means for correcting at least one of the first position information and the second position information based on the positional shift amount. Effect of the Invention

[0009] The cost of correcting the misalignment between objects detected in images captured at different times is reduced. [Brief description of the drawings]

[0010] [Figure 1] FIG. 2A is a block diagram showing an example of a hardware configuration of an information processing device, and FIG. 2B is a block diagram showing an example of a functional configuration of the information processing device. [Diagram 2] FIG. 2 is a diagram for explaining a tunnel to be inspected and an inspection image. [Diagram 3] FIG. 2 is a diagram for explaining a situation in which an inspection image of a tunnel is captured. [Figure 4] 13A and 13B are diagrams for explaining polyline data, (c) and (d) are diagrams for explaining the coordinates of vertices of the polyline data, and (e) is a diagram for explaining a display in which the polyline data is superimposed. [Diagram 5] 11 is a flowchart showing an example of a pattern image generation process. [Figure 6] FIG. 4 is a diagram for explaining a template region. [Figure 7] FIG. 4 is a diagram for explaining a process for determining a search region. [Figure 8] FIG. 13 is a diagram for explaining drawing of a polyline. [Figure 9] 11 is a flowchart showing an example of a calculation process of a deviation vector. [Figure 10] 5 is a diagram for explaining template matching executed by the information processing device; [Figure 11] 6A and 6B are diagrams for explaining a calculation process of a deviation vector. [Figure 12] 11 is a flowchart showing an example of a generation process of a conversion function. [Figure 13] A graph plotting the positional relationship between pre-transformation coordinates and deviation vectors. [Figure 14] FIG. 13 is a diagram showing an example of a display in which converted polyline data is superimposed. [Figure 15] 11A and 11B are diagrams for explaining polyline data obtained by dividing a template. [Figure 16] FIG. 13 is a diagram for explaining a case where there are multiple score peaks. [Figure 17] FIG. 11 is a block diagram showing an example of a functional configuration of an information processing device according to a second embodiment. [Figure 18] FIG. 2 is a diagram for explaining a GUI displayed by the information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0012] [Embodiment 1] The information processing device according to this embodiment acquires first position information indicating the position of an object detected from a first image capturing an image of the object, and second position information indicating the position of the object detected from a second image capturing the object at a time different from that of the first image. Next, the information processing device estimates a position shift amount of the object based on the acquired first position information and second position information, and corrects at least one of the first position information and the second position information based on the position shift amount.

[0013] For example, the information processing device according to the present embodiment can obtain, as the first position information and the second position information, a polyline data group including one or more pieces of polyline data each showing the shape of the target by information on each vertex from the first image and the second image, respectively. In this case, the information processing device generates a function for converting the coordinates in the coordinate system of the first polyline data into the coordinates in the coordinate system of the second polyline data based on the positional deviation amount estimated based on the polyline data, and converts the coordinates shown by the first polyline data by the generated function. The first polyline data and the second polyline data will be described later.

[0014] FIG. 1(a) shows an information processing device according to this embodiment. The information processing device 100 includes a CPU 101, a ROM 102, a RAM 103, a HDD 104, a display unit 105, an operation unit 106, and an I / F unit 107. The CPU 101 is a central processing unit (Central Processing Unit) that performs calculations and logical judgments for various processes and controls each component connected to a system bus 110. The ROM (Read-Only Memory) 102 is a program memory that stores programs for control by the CPU 101, including various processes described later. The RAM (Random Access Memory) 103 is used as a main memory of the CPU 101 or a temporary storage area such as a work area. The program memory may be realized by loading a program into the RAM 103 from the HDD 104 or an external storage device connected to the information processing device 100.

[0015] The HDD 104 is a hard disk for storing electronic data such as image data or a program according to the present embodiment. An external storage device may be used to fulfill the same role as the HDD 104. Here, the external storage device may be realized by, for example, a medium (recording medium) and an external storage drive for realizing access to the medium. For example, a flexible disk (FD), a CD-ROM, a DVD, a USB memory, or an SSD (Solid State Drive) may be used as such a medium. The external storage device may also be a device external to the information processing device 100, such as a server device connected via a network.

[0016] The display unit 105 is, for example, a liquid crystal display (LCD) or an organic light emitting display (OLED), and outputs an image to a display screen. The display unit 105 may be an external device connected to the information processing device 100 by wire or wirelessly. The operation unit 106 is, for example, a keyboard and a mouse, and accepts various operations by a user. The operation unit 106 is not particularly limited as long as it can accept input from a user, and may be, for example, a touch panel or a microphone. The I / F unit 107 performs two-way communication by wire or wirelessly with other information processing devices, communication devices, external storage devices, etc., using known communication technology.

[0017] Fig. 1(b) is a block diagram showing an example of a hardware configuration of the information processing device 100. Each functional unit shown in Fig. 1(b) is realized by a CPU 101 expanding a program stored in a ROM 102 into a RAM 103 and executing processing according to each flowchart described later.

[0018] The acquisition unit 111 acquires position information indicating the position of the subject (target) from each of two or more images capturing an object. Here, the acquisition unit 111 acquires first position information indicating the position of the target detected from the first image, and second position information indicating the position of the target detected from the second image. For example, a first polyline data group and a second polyline data group are acquired as the position information according to this embodiment. Here, the polyline data is data indicating the shape of the subject (target) in the image by information on each vertex, for example, as illustrated in FIG. 4(c) described later, and the polyline data group includes one or more pieces of polyline data. Here, a group of polyline data indicating the shape of the subject in the first image is defined as the first polyline data group, and a group of polyline data indicating the shape of the subject in the second image different from the first image is defined as the second polyline data group. For example, the acquisition unit 111 can acquire the first polyline data group and the second polyline data group from polyline data captured via the I / F unit 107 and stored in the HDD 104. The acquisition unit 111 may acquire the first image and the second image, detect a subject from each image, and generate polyline data. Hereinafter, the polyline data may be referred to as a "polyline." Hereinafter, the term "vertex" refers to a vertex represented by the polyline data. Hereinafter, the term "subject" refers to the above-mentioned "target" that is the processing target and whose position information is acquired by the acquisition unit 111.

[0019] The information processing device 100 according to the present embodiment estimates the amount of positional deviation of the target based on the acquired first position information and the second acquired information. Here, the amount of positional deviation of the target is, for example, the amount of positional deviation between the polyline data included in the first polyline data group and the polyline data included in the second polyline data group. The information processing device 100 according to the present embodiment can evaluate the similarity between the polyline data included in the first polyline data group and the polyline data included in the second polyline data group. Hereinafter, such polyline data included in the first polyline data group is referred to as the first polyline data, and the polyline data compared with the first polyline data for similarity is referred to as the second polyline data.

[0020] The information processing device 100 according to the present embodiment generates pattern images from the first polyline data and the second polyline data, and can evaluate the similarity from these pattern images by template matching. The configuration for evaluating the similarity will be described below.

[0021] The image generating unit 112 can generate a pattern image used for calculating the amount of deviation between the positions of the first polyline data and the second polyline data from the polyline data. The process performed by the image generating unit 112 will be described later.

[0022] The vector calculation unit 113 estimates the amount of positional deviation of the target from the first position information and the second position information acquired by the acquisition unit 111. Here, for example, a deviation vector calculated by a process described later is used as the amount of positional deviation. The calculation process of the deviation vector will be described later.

[0023] The function generating unit 114 generates a function for converting the coordinates in the coordinate system of the first polyline data into the coordinates in the coordinate system of the second polyline data based on the first polyline data group and the second polyline data group. For example, the first polyline data may include one or more polyline data, and the above-mentioned function may be generated based on at least one or more polyline data included in the first polyline data group and the corresponding polyline data included in the second polyline data group. Hereinafter, such a function may be simply called a "conversion function". The function unit 114 according to this embodiment generates a conversion function based on the deviation vector calculated by the vector calculation unit 113 for each polyline included in the first polyline data. The generation process of the conversion function will be described later with reference to FIG. 9.

[0024] In the following description, the coordinate system of the first polyline data is the coordinate system in a search target image (described later), and the coordinate system of the second polyline data is the coordinate system in a template image (described later). However, as long as conversion can be performed in a similar manner, the coordinate systems are not particularly limited to these, and different corresponding coordinates may be used, such as using the coordinate system of the corresponding vertex in the first image as the coordinates indicating the vertices of the first polyline data.

[0025] The coordinate conversion unit 115 corrects at least one of the first position information and the second position information based on the positional deviation amount estimated by the vector calculation unit 113. In this embodiment, for example, the function generation unit 114 generates a conversion function based on the deviation vector calculated by the vector calculation unit 113, and the coordinate conversion unit 115 converts the coordinates indicated by the first polyline data using the generated conversion function. Note that in this embodiment, the following description will be given assuming that the coordinates indicated by the first polyline data are converted into coordinates in the coordinate system of the second polyline data, but the coordinates indicated by the second polyline data may be converted into coordinates in the coordinate system of the first polyline data.

[0026] In this embodiment, a first image and a second image are acquired as inspection images. Here, the inspection images are two or more images of subjects to be compared, which are used to evaluate the soundness of a structure, for example. Here, a method for acquiring the inspection images and a situation in which a positional shift of the images occurs during the acquisition will be described with reference to Figs. 2 and 3.

[0027] Fig. 2 is a schematic diagram for explaining the relationship between the structure of a tunnel and an inspection image to be captured, assuming that an image of the inside of the tunnel is captured as an inspection image for tunnel inspection. Fig. 2 illustrates a cross-sectional view 200 of the tunnel and a development view 201 of the tunnel inner wall, which respectively show the tunnel lining surface 203, lighting 204 attached near the ceiling, and curbstones 205. The development view 201 also illustrates an opening 202 that serves as the entrance and exit of the tunnel, and a section (span) 206 that serves as a unit for pouring concrete on the lining surface. In this embodiment, inspection (capturing an inspection image) is performed in units of this span, which is repeated in the traveling direction.

[0028] FIG. 3 shows the situation in which an inspection image of a tunnel is captured, as viewed from a tunnel cross section 200. In order to capture an inspection image of the lining surface 206 to be inspected, a vehicle 301 equipped with a line sensor camera is driven along a route within the tunnel as shown in FIG. 3(a). The attached cameras 302-304 are line sensor cameras that capture images in different directions, and boundaries 308 and 309 indicate the boundaries of their imaging ranges. FIG. 3(b) shows the situation when the same tunnel is captured at a different time, and the cameras equipped on a vehicle 310 are the same as the cameras 302-304, but the driving position is different from the situation in FIG. 3(a).

[0029] In this case, the angle of view of cameras 302 to 304 is the same in Fig. 3(a) and Fig. 3(b), but the imaging positions are different, so even if the same subject is imaged, the position in the image varies. Also, in Fig. 3(a) and Fig. 3(b), the position of the boundary of the imaging range changes from 308 to 311 and from 309 to 312 due to the difference in the distance to the wall of each camera. Therefore, when the images captured by the three cameras are synthesized after imaging for inspection to create an image for inspection, the size of each image is adjusted, and this difference in imaging range causes a positional shift of the subject between the images for inspection.

[0030] From the inspection image acquired as described above, defects such as cracks are extracted as subjects, and polyline data is generated. As a method of extraction (detection of subject), any known method for setting a subject area in an image can be used. For example, to detect the subject, an operator may manually trace using dedicated software, or the subject may be automatically detected by image recognition using machine learning, or a combination of these methods may be used. In the following description, it is assumed that cracks in the image are detected as subjects, and the positions and number of each vertex are generated as polyline data.

[0031] FIG. 4 is a diagram for explaining polyline data according to this embodiment, which is generated by detecting cracks on the tunnel lining surface. FIG. 4(a) and FIG. 4(b) are schematic diagrams of polyline data groups extracted from inspection images acquired at different times (two periods) for the same span in the tunnel, and correspond to one span of the development 201 in FIG. 2. In the polyline data group 400 in FIG. 4(a), polyline data corresponding to ten cracks Ca001 to Ca010 are shown as a group of straight lines. In FIG. 4(b), which shows a polyline data group 401 acquired at a later time (several years later), some of the cracks Cb001 to Cb010 corresponding to the ten cracks in FIG. 4(a) have become longer due to the progression of deformation, and two new cracks Cb011 and Cb012 have also appeared. In Fig. 4(b), the polylines are shown as dotted lines to distinguish them from the cracks in Fig. 4(a), and the polyline data is not discontinuous. The polyline data shown in Fig. 4(a) and Fig. 4(b) is stored as a file in the HDD 104.

[0032] Fig. 4(c) is a table showing an example of polyline data corresponding to Fig. 4(a), and Fig. 4(d) is a table showing an example of polyline data corresponding to Fig. 4(b). Fig. 4(a) shows three fields: a deformation ID 411 for identifying each crack, a vertex count 412 showing the number of vertices included in the crack, and a vertex coordinate list 413 showing the coordinates of each vertex. The vertex coordinate list 413 is a list of coordinate values ​​of each vertex, with the upper left corner of one span of the inspection image as the origin, the horizontal direction as the X axis, and the vertical direction as the Y axis. 414 in Fig. 4(d) is the vertex coordinate list 414 corresponding to Fig. 4(b), and the origin of the vertex coordinates is the same as 410.

[0033] 420 in Fig. 4(e) is the polyline data of Fig. 4(a) and Fig. 4(b) superimposed with the origins aligned, indicating that there is a difference between the polyline data of the two time periods. The difference between the polyline data of the two time periods includes both the progression of deformation due to aging and the positional deviation caused by the difference in the imaging conditions shown in Fig. 3. Rectangular area 421 is a circumscribed rectangular area that includes the origin and all of the polyline data of the two time periods, and will be referred to again in the explanation of processing later.

[0034] In this embodiment, the coordinates of the polyline data included in the first image are converted from the first image and the second image, which are images for inspection. That is, in order to perform the inspection, the coordinates of the polyline data included in the first image are converted to coordinates in the coordinate system of the polyline data included in the second image, using the polyline data included in the second image as a reference (reference image). Here, for example, the second image is taken at a newer timing than the first image, but the selection of the image to be used as the reference (without converting the coordinates) is not limited to this. For example, the embodiment may be such that the user can select an image to be used as the image (image to be converted) for converting the coordinates of the polyline data from among the two images. For example, the acquisition unit 111 may allow the operator to specify the file names of the polyline data of the two periods to be compared and which image to use as the image to be converted, by a command prompt realized by the display unit 105 and the operation unit 106. For this specification, the acquisition unit 111 may present a file list as a GUI and accept the selection by the operator.

[0035] FIG. 5 is a flowchart showing an example of a pattern image generation process in the image generation unit 112. The image generation unit 112 generates a pattern image of a template used in template matching performed by the vector calculation unit 113 and a pattern image of a search destination. Here, of the two polyline data acquired by the polyline acquisition unit 111, the first polyline data included in the first polyline data group is used as the template, and the second polyline data included in the second polyline data group is used as the search destination, but either one may be used as the template. If the area or number of vertices in the two polyline data varies, the one with fewer vertices is used as the template and the one with more vertices is used as the search destination, thereby suppressing unnecessary searches that do not have a match. When inspecting for deformation of a structure, the number of vertices included in polyline data usually tends to increase over time as shown in FIG. 4, so the polyline data group 400 acquired at an older time may be used as the template, and the polyline 401 acquired at a newer time may be used as the search destination.

[0036] In S501, the image generating unit 112 determines a template region. In this embodiment, as described above, the circumscribing rectangle of each piece of polyline data included in the polyline data group 400 captured earlier is used as the template. Note that the drawing line width of the polyline data, which will be described later in the explanation of S503, may be added to the template. FIG. 6(a) is a schematic diagram showing an example of a template region. In FIG. 6(a), two polylines 601 and 602 are shown, and rectangular regions 603 and 604 are indicated by dotted lines as the template region.

[0037] In S502, the image generating unit 112 determines a search area for matching with the template. FIG. 7 is a schematic diagram for explaining the process of determining the search area. FIG. 7(a) and FIG. 7(b) are polyline data acquired at two different times. In FIG. 7(a), the templates 701 to 710 determined in S501 are shown in a dotted rectangle. The template 711 in FIG. 7(b) is a template 701 shown on the same coordinate system, and the search widths 713 to 716 are added to the left, right, top, and bottom of the template 701 to form a search range 712. The search widths 713 to 716 can be set to desired values ​​taking into consideration the degree of deviation of the polylines at the two times, and may all be the same width, or the horizontal directions 713 and 714, or the vertical directions 713 and 716, may each be the same width, or all may be different widths. The search width may be different for each polyline of the templates 701 to 710, or may all be the same width.

[0038] In S503, the image generating unit 112 generates a template image. The template image may be created by drawing only the polyline in the area determined in S501 as shown in Fig. 6(b), or may be created by drawing all the polylines to be used as the template and then cutting out the template area as shown in Fig. 6(c). In the latter case, parts of other polylines 602 included in the template area will also be included in the template.

[0039] FIG. 8 is a diagram for explaining drawing of a polyline. For a polyline 801 shown in FIG. 8(a), the image generating unit 112 may generate an image as a template in which a background 802 is drawn in black and a polyline 803 is drawn in white with a certain line width as shown in FIG. 8(b). Alternatively, the image generating unit 112 may generate an image as a template in which a pixel value gradient is applied in the line width direction with a polyline 804 at the center as shown in FIG. 8(c). When a pixel value gradient is applied as in the polyline 804, it is expected that in template matching, tolerance for the amount of deviation and matching accuracy can be achieved at a certain level.

[0040] In S504, the image generating unit 112 generates a search target image. The image generating unit 112 draws all polylines 401 acquired at a new time as search targets in the same manner as the template in S503, and cuts out the search target area determined in S502 to create a search target image. In drawing the polylines 401, search widths 713 to 716 are added around the rectangular area 421 in FIG. 4(e) in background color areas on the left, right, top, and bottom to ensure a search area corresponding to the template near the periphery. Note that in the template image and the search target image in which the polylines are drawn, the polylines are white and the background color is black, but these colors are not particularly limited, and a desired color can be set according to conditions.

[0041] 9 is a flowchart for explaining the process of calculating a deviation vector in the vector calculation unit 113. As described above, the deviation vector according to this embodiment indicates the amount of deviation between the first polyline data and the second polyline data. Here, the deviation vector indicates the deviation of the position of the polyline at the timing of the conversion target image specified by the polyline acquisition unit 111 relative to the corresponding polyline at the other timing. Due to the reason for the occurrence of the position deviation described above, the deviation vector varies depending on the position in the coordinate space.

[0042] In S901, the vector calculation unit 113 performs template matching using the template image and search target image generated by the image generation unit 112. Fig. 10(a) is a diagram for explaining the template matching executed in this embodiment. Polylines 1003 and 1004 are drawn in a search target image 1002 for a template image 1001. In Fig. 10(a), the polylines in the search target image 1002 are dotted lines, but this does not mean that the polylines are drawn as dotted lines in the pattern image, and the solid and dotted lines are used merely to distinguish them from the polylines of the template for the sake of explanation.

[0043] The vector calculation unit 113 slides the template image 1001 vertically and horizontally (rightward and downward) in the search target image 1002 from position 1005 to position 1006, and calculates a similarity score for each position of the template image. The similarity score calculated here is assumed to be the similarity between the first polyline data and the second polyline data for each position of the template image. The score calculation method can be any general method such as SSD (Sum of Squared Difference) or NCC (Normalized Cross Correlation), and is not limited to this method. If the sizes (width, height) of the template image 1001 and the search target image 1002 are (w, h) and (W, H), respectively, the matching result is a two-dimensional map with a size of (Ww, Hh). For example, when the template image 1001 and the search target image 1002 are matched by NCC, the matching result is a map 1007 and a map 1008 in the cross sections at the center of the horizontal and vertical directions, respectively.

[0044] In S902, the vector calculation unit 113 obtains, based on the score map, the position where the template image 1001 matches with the search target image 1002. This matching position is the position on the map calculated in S901 where the similarity is maximum (the minimum score in the case of SSD, the maximum score in the case of NCC).

[0045] FIG. 10(b) is a diagram for explaining the match position output in S902. In FIG. 10(b), the relative positions of the two (the position of the template image with respect to the search target image) when the template image 1009 is moved within a search target image 1011 including a polyline 1010 are indicated by a map range 1012. Here, the matching score (similarity) is calculated as the score at the center position (x, y) of the range 1012 with the upper left corner as the origin for the map 1012. In the example of FIG. 10(b), the coordinates of a position 1013 where the template image is superimposed on the search target image without being moved are (x0, y0), and the coordinates of a position 1015 where the template 1009 has moved to an area 1014 and the similarity is maximized are (xmax ,y max ) Note that, if multiple search target polylines run parallel to each other, the similarity may become maximum with a polyline that should not be matched, but such matching noise will be dealt with in later processing.

[0046] In S903, the vector calculation unit 113 calculates a deviation vector between the polylines at two times from the match position obtained in S902. Hereinafter, with reference to FIG. 11, the calculation process of the deviation vector according to this embodiment will be described. In FIG. 11(a), matching occurs when the template 1009 moves to 1014, as in FIG. 10(b). Here, in the coordinate system of the span image in FIG. 4, the coordinates of the center position 1100 of the template 1009 before the movement are (x1, y1), and the coordinates of the center position 1101 of the template 1009 at the time of matching are (x2, y2). If the polyline of the conversion target image specified by the polyline acquisition unit 111 is the polyline at the time used as the search target image (illustrated by a gray dotted line in FIG. 11), the deviation vector at the position 1101 is the vector 1102 extending from the position 1101 to the position 1100. That is, the deviation vector in FIG. 11(a) is (x1-x2, y1-y2). Furthermore, when the conversion target image specified by the polyline acquisition unit 111 is the polyline at the timing when it was used as the template image (shown by a solid black line in FIG. 11), the deviation vector at position 1100 is indicated by vector 1103 as shown in FIG. 11(b). That is, the deviation vector in FIG. 11(b) is (x2-x1, y2-y1). In this way, the vector calculation unit 113 according to this embodiment can calculate, as the deviation vector, the vector between the position of the template before template matching and the position of the template with the maximum similarity.

[0047] The coordinates of all vertices of the first polyline data according to this embodiment may be converted into coordinates in the coordinate system of the second polyline data by shifting them by the same deviation vector (for example, the above-mentioned (x1-x2, y1-y2)). However, the subject does not necessarily move parallel on the first image and the second image, and a simple movement by the deviation vector may not provide appropriate coordinates. From this viewpoint, the vector calculation unit 113 according to this embodiment may calculate deviation vectors from each of the multiple polyline data included in the first polyline data group. In this case, the function generation unit 114 can generate a conversion function based on the deviation vectors calculated by the vector calculation unit 113.

[0048] In this way, by calculating a plurality of deviation vectors and estimating a conversion function from the plurality of deviation vectors, it is possible to obtain a conversion function that can be applied to all vertices to convert the coordinate system of the polyline. Therefore, it is possible to convert the coordinates of each vertex of each polyline data included in the first polyline data group by conversion using the conversion function and perform inspection.

[0049] In this embodiment, a shift vector V indicating a vector to coordinates (transformed coordinates) transformed from the input coordinates U into the coordinate system of the second polyline data is output using a transformation function f, as shown in the following equation (1). V=f(U) Equation (1)

[0050] Here, the vector calculation unit 113 calculates a deviation vector V for the polyline data. Here, for example, the vector calculation unit 113 may calculate the deviation vector V for one of the polyline data included in the first polyline data group, may calculate the deviation vector V for a part of the polyline data, or may calculate the deviation vector V for the entire polyline data. Next, the function generation unit 114 generates a conversion function based on the x-component V of the deviation vector V. x and the y-coordinate V yFor each of and , a conversion function can be generated by modeling x and y as independent variables as follows and estimating each constant (here, A to H). V x =f x (x,y)=Ax+By+C V y =f y (x,y)=Dx 2 +Ey 2 +Fx+Gy+H Equation (2)

[0051] Here, the vector calculation unit 113 calculates the deviation vector V for each polyline data, for example, for each of Ca001 to Ca010 shown in FIG. 4, and the function generation unit 114 can calculate the constants of the formula (2) based on the calculated deviation vector V.

[0052] Note that the displacement vector (V x ,V y The transformation function for finding y is not limited to that shown in formula (2), and other model functions such as affine transformation may be adopted, taking into consideration the cause of the shift or the tendency of the shift. For example, when using an inspection image of the tunnel lining surface, it is assumed that there is almost no change in the shift in the running direction, i.e., the X-axis direction in Figure 4, over a short section of one span, so the number of constants to be estimated can be reduced by making y the only independent variable.

[0053] 12 is a flowchart showing an example of a conversion function generation process performed by the function generation unit 114. In S1201, the function generation unit 114 sets the coordinates of the first polyline data (here, the position of the center of the template) and the deviation vector calculated by the vector calculation unit 113 as actual measurement values ​​of the independent variable and dependent variable, respectively.

[0054] 13(a) is a diagram showing the candidates determined in S1201 using the displacement vectors calculated by processing the polyline groups at two times shown in FIG. 4. In 1300, the above-mentioned actual measurement values ​​are calculated based on the coordinates (x, y) of the polyline data and the y component V of the displacement vector converted using the coordinates. y Using the three-dimensional coordinates (x, y, V y ) in which the X-axis 1301 represents the x-coordinate in the coordinate system of the first image, the Y-axis 1302 represents the y-coordinate in the same coordinate system, and the Z-axis 1303 represents the y-component of the transformation vector V y The area surrounded by the dotted line 1304 shows the actual measurement values ​​(x, y, V y ) are plotted. x ) can also be plotted in a similar manner, but the explanation will be omitted here.

[0055] In S1202, the function generating unit 114 removes noise from the values ​​set as the actual measurement values ​​in S1201. For example, the function generating unit 114 calculates a vector (V x ,V y For one or both scalar values ​​of each component of , the difference from the overall average is greater than a constant multiple of the standard deviation, and other known processing that removes outliers that are considered to be outliers in general statistical processing can be used to remove noise. Note that when a position shift occurs due to a difference in the imaging positions of the first image and the second image, the tendency of conversion by the conversion function changes depending on the position. Therefore, instead of removing the entire measured values, the region shown in FIG. 4 may be divided into several partial regions, and outliers may be found and removed for each of those partial regions.

[0056] In S1203, the function generating unit 114 may correct the conversion function based on the number of actual measured values ​​remaining after excluding outliers. The model function of equation (2) includes eight constants A to H, but if there are only a few actual measured values, these constants may not be calculated correctly. From this perspective, in S1202, the function generating unit 114 can correct the model function so that the number of constants to be calculated is reduced depending on the number of actual measured values ​​remaining without being excluded. For example, when the number of remaining actual measured values ​​becomes less than a predetermined threshold (for example, the number of constants of the model function), the function generating unit 114 corrects f y The model function can be modified by setting either or both of the squared components of (x, y) to 0, lowering the order of the model function, etc.

[0057] In S1204, the function generating unit 114 calculates a weight for each actual measurement value to be used in the conversion function generating process in S1205. The function generating unit 114 may calculate this weight based on, for example, the shape of the template. In this case, the function generating unit 114 may calculate the weight based on the magnitude of the direction perpendicular to each of the X component and the Y component of the deviation vector from the shape of the template. For example, the function generating unit 114 calculates f x For (x,y), the height of the template image is taken as f y For (x, y), the width value of the template image can be used as the weight. Here, the function generating unit 114 may use the height and width values ​​of the template image as the weights as they are, or may use values ​​obtained by performing a predetermined operation, such as multiplying each of the height and width values ​​by a predetermined coefficient, as the weights. The function generating unit 114 may also calculate the weights based on the similarity at the time of matching. In this case, the function generating unit 114 can set the weights so that the weight for a search target image with a higher matching similarity has a larger value.

[0058] In S1205, the function generating unit 114 applies the weight calculated in S1204 to the actual measurement value from which the outliers have been removed in S1202, and calculates the constants of the model function corrected in S1203. Here, the function generating unit 114 can output the model function by, for example, the weighted least squares method. For the generation process of the model function, any known technology that outputs a model function from a similar input can be used, and detailed description thereof will be omitted.

[0059] In S1206, the function generating unit 114 can remove polylines that are determined to be outliers based on the transformed coordinates and the actual measured values ​​output using the transformation function generated in S1205. For example, the function generating unit 114 can determine a polyline in which the difference between the transformed coordinates and the actual measured values ​​is 3.0 times or more their standard deviation as an outlier. Note that the method of determining outliers is not limited to this, and other known methods used in statistical analysis may be applied.

[0060] In S1207, the function generating unit 114 determines whether the number of actual measured values ​​(the number of polylines) remaining after removing the outliers in S1206 is equal to or less than a threshold value, or whether removal of the outliers was not performed in S1206. The threshold value is the number of actual measured values ​​required to obtain the constants of the model function in S1205, for example, the number of constants. If the determination in S1207 is No, the process returns to S1205, and if the determination is Yes, the process in FIG. 12 ends.

[0061] Fig. 13(b) is a three-dimensional graph showing the actual measurement values ​​shown in Fig. 13(a) and the output of the conversion function generated based on those actual measurement values ​​by the process illustrated in Fig. 12. A mesh-like curved surface 1305 is a Y component V y The distribution of the X component of the deviation vector V x The distribution (not shown) of becomes planar due to the order of equation (2).

[0062] The coordinate conversion unit 115 converts the coordinates of all vertices of the polyline of the timing of the conversion target image specified by the acquisition unit 111 using the conversion function generated by the function generation unit 114 to generate converted coordinates. Equation (3) shows the relationship between the coordinates (x, y) before conversion and the coordinates (x', y') after conversion.

number

[0063] FIG. 14 shows the result of coordinate conversion by coordinate conversion unit 115, where 1400 is the polyline in FIG. 4(a) shown by the solid line and the polyline in FIG. 4(b) superimposed on each other after coordinate conversion, shown by the dashed line.

[0064] According to this configuration, a function for converting the coordinates in the coordinate system of a first image to the coordinate system of a second image is generated based on the polyline data of a plurality of images captured at different times, and the coordinates of the vertices of one of the polylines can be converted based on the function. In particular, a template image and a search target image are generated, the deviation vectors of each polyline are calculated by template matching, and the coordinates of each polyline are converted by a conversion function estimated from the deviation vectors. This makes it possible to correct the positional deviation of the subject caused by the difference in the imaging conditions of the images used for extracting the polylines using only the polyline data without using the captured images. This has the effect of reducing the cost required for correcting the positional deviation of the subject in the images.

[0065] Furthermore, by using the one of the two polyline data with the fewer polyline area or vertices as the template, it is possible to suppress the occurrence of unnecessary matching processing, and improve the processing speed and the accuracy of misalignment correction. Also, by applying a pixel value gradient in the line width direction when drawing a polyline in generating a pattern image, it is possible to improve the accuracy of misalignment correction.

[0066] Furthermore, by removing deviation vectors that are outliers through processing such as S1202 and S1206, it is possible to reduce the influence of template matching errors that occur when polylines run parallel to one another, etc. In addition, by adding such outlier removal, it is possible to reduce the influence of errors in polyline data caused by tracing errors made by an operator or false positives in image recognition by machine learning when extracting polylines from an image.

[0067] Furthermore, by modifying the model function so as to reduce the number of constants to be estimated by the process of S1203, etc., it is possible to deal with cases where there is a small amount of polyline data to be used as a template. Furthermore, weighting the actual measurement value in S1203 of Fig. 12 has the effect of improving the accuracy of generating the conversion function. Furthermore, weighting by the size of the template that is orthogonal to the deviation component has the effect of prioritizing deviation correction of polylines over a wide range that is of high importance.

[0068] [Variation 1] In the first embodiment, the template region is a circumscribing rectangle of each polyline in S501 of the pattern image generation process shown in Fig. 5. In this modification, the number of templates is increased by dividing the template region into smaller partial regions.

[0069] Fig. 15 is a diagram for explaining an example of polyline data in which the template area is made smaller and the number of templates is increased. Rectangular area 1500 is a circumscribed rectangular area that includes all the polyline data at two times in Fig. 4(e), and is a drawing area for polylines. This area is divided into partial areas in a vertical and horizontal grid shape as shown by dotted lines, and among them, partial areas in which polylines overlap, such as area 1501 shown by diagonal lines, are used as templates. In Fig. 15, rectangles with diagonal lines in the background indicate those to be used as templates, and diagonal lines are not drawn as the background.

[0070] Alternatively, each polyline may be divided into pieces of appropriate size, and the bounding rectangle for each divided polyline may be used as a template. For example, the longer side or the length of the diagonal of the bounding rectangle of the polyline may be l, and an integer n that satisfies the following formula (4) may be found, and the polyline may be divided into n pieces with approximately the same number of vertices. Ln≦l≦L(n+1) Equation (4)

[0071] According to this process, by reducing the template area, the accuracy of each deviation vector is improved, the number of valid deviation vectors is increased, and the accuracy of generating the conversion function is improved. Also, by reducing the template area, the effect of reducing the influence of the deformation of the polyline shape due to the difference in deviation depending on the position on the matching is reduced.

[0072] [Variation 2] For example, in the case of a polyline extracted from a crack in an inspection image of a tunnel lining surface, the main cause of positional deviation is thought to be the variation in camera movement path and speed, which is thought to be small between adjacent spans. Therefore, it is expected that the tendency of positional deviation of adjacent spans is almost the same, and the same transformation function can be used at the vertices of the spans.

[0073] That is, the function generating unit 114 according to this modification judges whether the imaging conditions are similar between the polyline data, and converts the coordinates of the polyline data having similar imaging conditions using a common conversion function. In this case, the function generating unit 114 may apply the conversion function generated for one polyline data by the same process as in the first embodiment to the polyline data having similar imaging conditions to the polyline data. For example, the function generating unit 114 may generate a conversion function to be used for the polyline data from all the polyline data having similar imaging conditions.

[0074] In this modification, unlike the first embodiment, when the function generating unit 114 generates a conversion function, it uses not only one span but also deviation vectors calculated from a polyline of a plurality of adjacent spans. The range for combining deviation vectors can be divided into a certain number of spans, or a certain distance for the length of the connected spans, and the deviation vectors of the vertices of the polyline for one division can be combined. The function generating unit 114 may combine all the spans in the tunnel into one. Alternatively, when the number of spans to be combined is n, the function generating unit 114 may combine the conversion function for the i-th span (1≦i≦n) by sliding the process so as to perform the generation of the conversion function for each of the i-th to i+n-1-th spans of the polylines.

[0075] In the process of generating the conversion function, the function generating unit 114 combines the spans of the range to be combined and performs each process using a coordinate system with the upper left corner as the origin. In this case, the function generating unit 114 adds the sum of the span widths on the left side to the x calculated in S903 and sets the value as the independent variable in the X-axis direction in S1201.

[0076] Although the case where processing is performed on continuous spans in tunnel inspection has been described here, the application of this modified example is not limited to such an example. For example, when processing inspection images of a bridge deck or a building wall, the present invention can also be applied to a case where imaging settings such as the distance from the target, angle, and lens focal length are kept constant and imaging is performed continuously over multiple inspection units.

[0077] According to this type of processing, the deviation vectors obtained for each successive inspection unit are used together to generate a conversion function, which has the effect of increasing the number of valid deviation vectors and improving the accuracy of the conversion function generation compared to when the generation is performed using only a single inspection unit.

[0078] [Variation 3] In the first embodiment, in the process of calculating the deviation vector shown in Fig. 9, the deviation vector is calculated using the coordinates when the matching similarity is at its maximum. However, depending on the arrangement state of the polylines at the two times, the matching similarity may not show a clear peak, or there may be multiple similarity peaks. From this viewpoint, the vector calculation unit 113 according to this modification considers not only the value of the matching similarity but also the distribution state of the similarity when calculating the deviation vector.

[0079] FIG. 16 is a diagram for explaining polylines in the case where there are multiple matching similarities. 1600 shows a group of polylines from two periods superimposed on each other, with the polylines from the old timing shown as solid lines and the polylines from the new timing shown as dotted lines. The polylines from the new timing shown in a partial region 1601 in 1600 are those corresponding to the old timing, and newly added polylines run in parallel. FIG. 16 also shows an enlarged search region 1602 and template 1603 related to the partial region 1601. A cross section cut in the Y-axis direction at the center of the left and right of the two-dimensional map of the matching score calculated by the vector calculation unit 113 using these is shown as graph 1604.

[0080] 16, as shown in graph 1604, there are two peaks in the score because template 1603 matches both of the two upper and lower polylines that exist in search area 1602. In such a case, if the coordinates of the one with the higher similarity (the relationship between the high and low scores and the similarity follows the matching method) are simply taken as the match position, an erroneous result may occur.

[0081] Therefore, when there are multiple peaks in the distribution of these similarities, the vector calculation unit 113 according to this modification calculates a deviation vector for each of these peaks by the same process as that described in the first embodiment, and performs subsequent processes without any change. Note that, here, the region where the similarity exceeds a certain value is set as the match position (the position where the peak occurs), but for example, the region where the similarity falls within a predetermined range from the calculated maximum similarity may be set as the peak position, and can be set according to the user's desired conditions.

[0082] Of the deviation vectors calculated from each of the multiple peaks, one is considered to be noise, but it is expected that it will be removed as an outlier by the process of removing outliers such as S1202 or S1206. Note that when there are multiple peaks, the vector calculation unit 113 may not calculate a deviation vector by regarding both as noise, but may calculate a deviation vector only when there is a single peak. Also, the vector calculation unit 113 may not calculate a deviation vector when there is no clear peak.

[0083] According to this processing, the noise caused by matching errors can be reduced by calculating the deviation vector taking into account the distribution of matching similarity. In particular, even in the case where polylines run in parallel, the processing considering multiple peaks has the effect of improving the accuracy of position deviation correction.

[0084] In this embodiment, the subject of imaging is a structure, the subject of processing is a deformation, and each processing is performed, but this is not limited to this as long as similar processing can be performed based on the image detection results.

[0085] [Embodiment 2] The information processing device 1700 according to the second embodiment can perform the same processing as the information processing device 100 according to the first embodiment. Furthermore, the information processing device 1700 according to this embodiment has a function for setting various parameters required for coordinate conversion processing and a function for displaying the position deviation correction result, and provides an interactive GUI to the operator. The hardware configuration of the information processing device 1700 is the same as that shown in FIG. 1(a) of the first embodiment.

[0086] FIG. 17 is an example of a block diagram showing a functional configuration of an information processing device 1700 in this embodiment. Each of these functional units is realized by the CPU 101 expanding a program stored in the ROM 102 into the RAM 103 and executing processing according to each flowchart described later. The information processing device 1700 according to this embodiment has the same functional units as those shown in FIG. 1(b) of the first embodiment except that it additionally has a control unit 1701 and a display control unit 1702, and therefore duplicated explanations will be omitted. The control unit 1701 accepts an operation on a GUI component displayed on the display unit 105 through the operation unit 106. The display control unit 1702 displays the polylines of the specified two periods and the polylines whose position deviations have been corrected on the display unit 105.

[0087] Fig. 18 shows an example of a GUI screen displayed by the information processing device 1700 according to this embodiment. This GUI screen is realized using a window system provided by the OS running on the information processing device 100, and as shown in Fig. 18(a), GUI components 1801 to 1809 are arranged in a main window 1800.

[0088] 18(a), when the operator clicks on an area of ​​the file list 1801 using the operation unit 106 (for example, a mouse), the control unit 1701 displays a file selection dialog (not shown). Using the file selection dialog, the control unit 1701 allows the operator to specify two polyline data files and the order of their acquisition times (imaging timing), and stores the results. Note that the imaging timing may be stored in association with the polyline data, and in that case, the order of the imaging timing may be selected based on the associated information regardless of the user operation.

[0089] When the polyline data files have been specified, the control unit 1701 displays the file names in the specified order of acquisition time in a file list 1801. Next, the display control unit 1702 displays the polylines contained in each file in a superimposed manner in a polyline display area 1809. Figure 18(b) shows the screen after file selection, where the names of the two files are displayed in a field 1810, and the polylines of each file are displayed in a different color (represented by different line types in the figure) in a polyline display area 1812.

[0090] The control unit 1701 holds, as a variable, which of the two selected files is to be converted, and highlights the name of the file to be converted as shown in 1811. Here, the conversion target is switched by having the operator click on the file name in the file list 1801, but the conversion target image may be set as in the first embodiment.

[0091] When the operator clicks to select the polyline with the oldest acquisition date (shown by a solid line in Fig. 18) among the polylines displayed in display area 1812, display control unit 1702 changes the polyline display area as shown in Fig. 18(c). When the operator slides the knob of line width slider 1802, control unit 1701 determines the line width for drawing the polyline when generating the template image and the search target image in S503 and S504 in Fig. 5, respectively, according to the slide position. An example of the display of the polyline changed to the line width determined in this way is shown in display 1813.

[0092] Furthermore, the display control unit 1702 displays the template region 1814 determined in S501 and the search target image region 1815 determined in S502 for the selected polyline. The control unit 1701 determines and holds the search range values ​​of the search widths 713 to 716 in FIG. 7(b) in response to an input by the operator to move or enlarge / reduce the search target image region 1815. The line width and search range are specified by the operator by selecting one of the polylines, but this does not need to be done for each polyline, and the values ​​specified by the user are applied to all polylines. Note that the line width and search range may be set for each individual polyline.

[0093] 18 is used to specify the template division settings described in Modification 1 of the first embodiment. For example, when the slider 1803 is set to "Large", template division is not performed, and when set to "Small", the polyline is divided into smaller ones. For example, when the slider 1803 is changed from "Large" to "Small", the constant L in equation (4) is changed from the maximum value of the polyline length to 20% of that value.

[0094] A first slider 1804 for noise removal and a second slider 1805 for noise removal shown in Fig. 18 are used to specify thresholds used in the outlier removal processing in S1202 and S1206 in Fig. 12. For example, a constant to be multiplied by the standard deviation to be used as each threshold is specified within the range of 0.5 to 4.0.

[0095] Although not included in the GUI components shown in FIG. 18, GUI components may be provided for other parameters such as the constants used for weighting in S1204, and inputs relating to these may be received.

[0096] 18, default values ​​are used unless the operator specifies otherwise. When the operator clicks on a parameter save button 1808, the values ​​held by the various GUIs at that point in time are saved as a file in the HDD 104.

[0097] In Fig. 18(c), when the operator clicks on the correction start button 1807, the processing shown in Fig. 5, Fig. 9 and Fig. 12 and the coordinate conversion of the polyline of the conversion target image are performed using the parameter values ​​held at that time, and the result is displayed in the polyline display area 1809. Fig. 18(d) shows the screen after the coordinate conversion, in which the polyline (solid line) of the reference image and the polyline (dotted chain line) after the coordinate conversion are superimposed and displayed, as in the display screen 1816. In the GUI shown in Fig. 18(d), when the operator clicks on the return button 1806, the screen returns to Fig. 18(c), and the operator can adjust the parameters by inputting them.

[0098] According to this configuration, in addition to the processes described in the first embodiment, it is possible to provide an information processing device having a means for designating parameters to be used in the coordinate conversion process and displaying the results of the coordinate conversion. Therefore, it is possible to provide an interactive GUI to the operator and efficiently encourage the operator to set appropriate parameters. Furthermore, by storing the determined parameters in an external storage device, it is possible to reuse the parameters even for polyline data acquired under similar conditions.

[0099] The disclosure of this specification includes the following information processing device, information processing method, and program.

[0100] (Item 1) an acquisition means for acquiring first position information indicating a position of an object detected from a first image capturing an image of the object, and second position information indicating a position of the object detected from a second image capturing the object at a time different from that of the first image; an estimation means for estimating a positional deviation amount of the object based on the first position information and the second position information; and a correction unit that corrects at least one of the first position information and the second position information based on the amount of positional deviation. 23. An information processing apparatus comprising: (Item 2) The information processing device described in item 1, characterized in that the acquisition means acquires, as the first position information, a first polyline data group including one or more polyline data indicating the shape of the object in the first image with information on each vertex, and, as the second position information, a second polyline data group including one or more polyline data indicating the shape of the object in the second image with information on each vertex. (Item 3) a first generating means for generating a function for converting coordinates in a coordinate system of the first polyline data included in the first polyline data group into coordinates in a coordinate system of the second polyline data included in the second polyline data group based on the amount of positional deviation; 3. The information processing device according to item 2, wherein the correction means corrects coordinates indicated by the first polyline data based on the function. (Item 4) the estimation means estimates, as the positional deviation amount, a similarity between the polyline data included in the first polyline data group and the polyline data included in the second polyline data group; 4. The information processing device according to item 3, wherein the first generating means generates the function based on the similarity. (Item 5) a second generating means for generating a search target image corresponding to the first polyline data based on the first polyline data included in the first polyline data group, and for generating a template image corresponding to the second polyline data based on the second polyline data included in the second polyline data group, 5. The information processing device according to item 4, wherein the estimation means estimates a similarity of polyline data for each position of the template image by template matching using the template image and the search target image. (Item 6) 6. The information processing device according to item 5, wherein the second generating means generates a circumscribed rectangular area of ​​the first polyline data as the search target image. (Item 7) 7. The information processing device according to item 5 or 6, wherein the second generating means generates a circumscribed rectangular area of ​​the second polyline data as the template image. (Item 8) 8. The information processing device according to item 7, characterized in that the template image is an image having a gradient of pixel values ​​in the line width direction with the first polyline data as the center within the circumscribed rectangular region. (Item 9) 6. The information processing device according to item 5, wherein the second generating means generates a partial area obtained by dividing an area including the first polyline data as the template image. (Item 10) The information processing device described in item 5, characterized in that the second generation means generates a circumscribed rectangular area of ​​the third polyline data, which includes third polyline data obtained by dividing the first polyline data, as the template image. (Item 11) a calculation unit for calculating a vector between a position of the template image before the template matching and a position at which the similarity is maximized, The information processing device described in any one of items 5 to 10, characterized in that the first generation means generates the function based on the vector calculated by the calculation means from a plurality of polyline data included in the first polyline data group. (Item 12) The information processing device described in item 11, characterized in that the first polyline data is included in the polyline data included in the first polyline data group, with polyline data in which the vector is an outlier removed. (Item 13) Item 13. The information processing device according to item 12, further comprising a modification means for modifying the function when the number of polyline data from among the polyline data included in the first polyline data group, after removing polyline data whose vector is an outlier, is less than a predetermined threshold value. (Item 14) The information processing device according to any one of items 5 to 11, characterized in that the first generating means generates, as the function, a model function that inputs the position of the first polyline data in the search target image and outputs an X component or a Y component of the vector. (Item 15) a setting unit for setting a weight for each vector calculated by the calculation unit, Item 15. The information processing device according to item 14, wherein the first generating means generates the model function further based on a weight for each of the vectors. (Item 16) The method further includes a determination unit that determines whether or not the imaging conditions of the first image and the third image are similar, The information processing device described in any one of items 3 to 15, characterized in that, for a third image determined to have imaging conditions similar to those of the first image, the first generation means generates a function that converts coordinates in the coordinate system of fourth polyline data included in a third polyline data group including one or more polyline data indicating the shape of an object in the third image by information on each vertex into coordinates in the coordinate system of the second polyline data, as the same function as the function that converts coordinates in the coordinate system of the first polyline data into coordinates in the coordinate system of the second polyline data. (Item 17) 17. The information processing device according to any one of items 5 to 16, characterized in that the number of polyline data included in the first polyline data group is smaller than the number of polyline data included in the second polyline data group. (Item 18) A second acquisition means for acquiring an input from an operator; a setting means for setting a parameter to be used in a correction process using the function based on the input acquired by the second acquisition means; a presentation means for presenting a result of the coordinate correction performed by the correction means to the operator; 18. The information processing device according to any one of items 3 to 17, further comprising: (Item 19) The information processing device described in any one of items 1 to 18, further comprising a display means for displaying polyline data indicating the shape of the target by information on each vertex on a display device based on at least one of the first position information and the second position information corrected by the correction means. (Item 20) 20. The information processing device according to any one of items 1 to 19, wherein the object is a structure, and the target is a deformation of the structure. (Item 21) acquiring first position information indicating a position of an object detected from a first image capturing an image of the object, and second position information indicating a position of the object detected from a second image capturing the object at a time different from that of the first image; estimating a positional deviation amount of the object based on the first position information and the second position information; correcting at least one of the first position information and the second position information based on the amount of positional deviation; An information processing method comprising: (Item 22) A program for causing a computer to function as each of the means of the information processing device according to any one of items 1 to 20.

[0101] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0102] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0103] 111: Acquisition unit, 112: Image generation unit, 113: Vector calculation unit, 114: Function generation unit, 115: Coordinate conversion unit

Claims

1. an acquisition means for acquiring first position information indicating a position of an object detected from a first image capturing an object, and second position information indicating a position of the object detected from a second image capturing the object at a time different from that of the first image; an estimation means for estimating a positional deviation amount of the object based on the first position information and the second position information; a correction unit that corrects at least one of the first position information and the second position information based on the amount of positional deviation.

23. An information processing apparatus comprising:

2. The information processing device according to claim 1, characterized in that the acquisition means acquires, as the first position information, a first polyline data group including one or more polyline data indicating the shape of an object in the first image by information on each vertex, and, as the second position information, a second polyline data group including one or more polyline data indicating the shape of an object in the second image by information on each vertex.

3. a first generating means for generating a function for converting coordinates in a coordinate system of the first polyline data included in the first polyline data group into coordinates in a coordinate system of the second polyline data included in the second polyline data group based on the amount of positional deviation; 3. The information processing apparatus according to claim 2, wherein said correction means corrects the coordinates indicated by said first polyline data on the basis of said function.

4. the estimation means estimates, as the positional deviation amount, a similarity between the polyline data included in the first polyline data group and the polyline data included in the second polyline data group; The information processing apparatus according to claim 3 , wherein said first generating means generates said function based on said similarity.

5. a second generating means for generating a search target image corresponding to the first polyline data based on the first polyline data included in the first polyline data group, and for generating a template image corresponding to the second polyline data based on the second polyline data included in the second polyline data group, 5. The information processing apparatus according to claim 4, wherein said estimating means estimates the similarity of the polyline data for each position of said template image by template matching using said template image and said search target image.

6. 6. The information processing apparatus according to claim 5, wherein said second generating means generates a circumscribing rectangular area of ​​said first polyline data as said search target image.

7. 6. The information processing apparatus according to claim 5, wherein said second generating means generates a circumscribing rectangular area of ​​said second polyline data as said template image.

8. 8. The information processing apparatus according to claim 7, wherein said template image is an image having a gradient of pixel values ​​in said circumscribing rectangular area in a line width direction with said first polyline data as a center.

9. 6. The information processing apparatus according to claim 5, wherein said second generating means generates, as said template image, a partial area obtained by dividing an area including said first polyline data.

10. 6. The information processing apparatus according to claim 5, wherein the second generating means generates, as the template image, a circumscribing rectangular area of ​​the third polyline data, the third polyline data including the third polyline data obtained by dividing the first polyline data.

11. a calculation unit for calculating a vector between a position of the template image before the template matching and a position where the similarity is maximized, 6. The information processing apparatus according to claim 5, wherein said first generating means generates said function based on said vector calculated by said calculating means from a plurality of polyline data included in said first polyline data group.

12. 12. The information processing apparatus according to claim 11, wherein the first polyline data is included in the first polyline data group after removing polyline data in which the vector is an outlier.

13. 13. The information processing apparatus according to claim 12, further comprising a modification means for modifying the function when the number of polyline data from which polyline data in the first polyline data group, from which polyline data in which the vector is an outlier has been removed, is less than a predetermined threshold value.

14. 12. The information processing apparatus according to claim 11, wherein the first generating means generates, as the function, a model function that receives as input the position of the first polyline data in the search target image and outputs an X component or a Y component of the vector.

15. a setting unit for setting a weight for each vector calculated by the calculation unit, The information processing apparatus according to claim 14 , wherein the first generating means generates the model function further based on a weight for each of the vectors.

16. The method further includes a determination unit that determines whether or not the imaging conditions of the first image and the third image are similar, 4. The information processing device according to claim 3, wherein the first generation means generates a function for converting coordinates in a coordinate system of fourth polyline data included in a third polyline data group including one or more polyline data indicating the shape of an object in the third image by information on each vertex into coordinates in the coordinate system of the second polyline data for a third image determined to have imaging conditions similar to those of the first image, as a function identical to a function for converting coordinates in the coordinate system of the first polyline data into coordinates in the coordinate system of the second polyline data.

17. 6. The information processing apparatus according to claim 5, wherein the number of polyline data included in said first polyline data group is smaller than the number of polyline data included in said second polyline data group.

18. A second acquisition means for acquiring an input from an operator; a setting means for setting a parameter to be used in a correction process using the function based on the input acquired by the second acquisition means; a presentation means for presenting a result of the coordinate correction performed by the correction means to the operator; The information processing device according to claim 3 , further comprising:

19. 2. The information processing device according to claim 1, further comprising a display means for displaying, on a display device, polyline data indicating the shape of the target by information on each vertex, based on at least one of the first position information and the second position information corrected by the correction means.

20. The information processing apparatus according to claim 1 , wherein the object is a structure, and the object is a deformation of the structure.

21. acquiring first position information indicating a position of an object detected from a first image capturing an image of the object, and second position information indicating a position of the object detected from a second image capturing the object at a time different from that of the first image; estimating a positional deviation amount of the object based on the first position information and the second position information; correcting at least one of the first position information and the second position information based on the positional deviation amount; An information processing method comprising:

22. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 20.

Images