Information processing apparatus, information processing method, storage medium, and computer program product

Abstract

The invention relates to an information processing apparatus, an information processing method, a storage medium, and a computer program product. The obtaining unit obtains first position information for indicating a position of an object detected in a first image in which an object is captured, and second position information for indicating a position of an object detected in a second image in which the object is captured at a different timing from the first image. An estimation unit estimates a position offset amount of the object based on the first position information and the second position information. A correction unit corrects at least one of the first position information and the second position information based on the position offset amount. The first position information includes information of each vertex of an object in the first image, and the second position information includes information of each vertex of an object in the second image.

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing method and a storage medium. Background Art

[0002] In recent years, in order to optimize the inspection tasks of aging social infrastructure structures in Japan, the soundness of the structures is evaluated using inspection images taken with the inspection objects as the subject. With this technology, damage in the inspection image is extracted as polylines (vector data) by tracking the operator or by image inspection using machine learning, and the damage in the inspection image is evaluated by considering the damage status such as position, shape, size, number and density, and the progress of comparison over time.

[0003] For example, in the case where the structure is a tunnel, the cross section relative to the advancing direction is generally in the shape of a circle, and the lining surface as a curved surface along the circle corresponds to the main inspection object. A method for effectively obtaining a high-definition inspection image of a wide area such as a tunnel lining surface includes driving a car with multiple cameras attached to the object and synthesizing the captured image data via digital processing. This method is being improved with the change of the camera used from a video camera to a line sensor camera with higher definition, and the introduction of drones for capturing images of bridges, etc. Thus, in the case of a recently acquired inspection image and a previous inspection image as an object of comparison that changes over time, there is a possibility that the shooting conditions such as the camera type, field of view, or number of cameras are different. In addition, even if these conditions are the same, there may be differences in the field of view covered by each camera when the moving path is different during shooting. This may lead to differences in the processing results when synthesizing the captured images, and even for the same object, this may lead to positional shifts between the inspection images due to the acquisition time period, making it difficult to extract damage.

[0004] An example of a method for enabling comparison by correcting positional shifts between images captured of the same subject is described in Japanese Patent Laid-Open No. H7-37074. This technology, which is related to the medical field, includes aligning chest X-ray images. Summary of the invention

[0005] According to one embodiment of the present invention, an information processing device includes: an obtaining unit, which is configured to obtain first position information indicating the position of an object detected in a first image taken of the object, and second position information indicating the position of the object detected in a second image taken of the object at a time different from that of the first image; an estimating unit, which is configured to estimate the position offset of the object based on the first position information and the second position information; and a correcting unit, which is configured to correct at least one of the first position information and the second position information based on the position offset, wherein the first position information includes information of each vertex of the object in the first image, and the second position information includes information of each vertex of the object in the second image.

[0006] According to another embodiment of the present invention, an information processing method includes: obtaining first position information for indicating the position of an object detected in a first image taken of the object, and second position information for indicating the position of the object detected in a second image taken of the object at a time different from that of the first image; estimating a position offset of the object based on the first position information and the second position information; and correcting at least one of the first position information and the second position information based on the position offset, wherein the first position information includes information of each vertex of the object in the first image, and the second position information includes information of each vertex of the object in the second image.

[0007] According to another embodiment of the present invention, a non-transitory computer-readable storage medium stores a program, which, when executed by a computer, causes the computer to perform an information processing method, the information processing method comprising: obtaining first position information for indicating the position of an object detected in a first image taken of the object, and second position information for indicating the position of the object detected in a second image taken of the object at a time different from that of the first image; estimating a position offset of the object based on the first position information and the second position information; and correcting at least one of the first position information and the second position information based on the position offset, wherein the first position information includes information of each vertex of the object in the first image, and the second position information includes information of each vertex of the object in the second image.

[0008] Further features of the present invention will become apparent from the following description of exemplary embodiments (with reference to the attached drawings). BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1A is a block diagram showing an example of the hardware configuration of an information processing device.

[0010] Figure 1B is a block diagram showing an example of a functional configuration of an information processing device.

[0011] Figure 2 A diagram for explaining a tunnel to be inspected and an inspection image.

[0012] Figure 3A and Figure 3B It is a diagram for explaining a state in which an inspection image of a tunnel is captured.

[0013] Figure 4A and Figure 4B This is a graph for explaining line data.

[0014] Figure 4C and Figure 4D This is a diagram for explaining the coordinates of the vertices of the polyline data.

[0015] Figure 4E This is a diagram for explaining the superimposed display of line data.

[0016] Figure 5 is a flowchart illustrating an example of pattern image generation processing.

[0017] FIG. 6A to FIG. 6C It is a diagram for explaining the template area.

[0018] Fig. 7A and Figure 7B It is a diagram for explaining the process for specifying the target area.

[0019] FIG. 8A to FIG. 8C This is a diagram for explaining line drawing.

[0020] Fig. 9 is a flowchart illustrating an example of offset vector calculation processing.

[0021] Fig. 10A and Fig. 10B is a diagram for explaining template matching performed by the information processing device.

[0022] Fig.11A and Fig. 11B This is a diagram used to illustrate the offset vector calculation process.

[0023] Fig.12 is a flowchart illustrating an example of transformation function generation processing.

[0024] Fig.13A and Fig. 13B It is a graph that plots the positional relationship between the coordinates before transformation and the offset vector.

[0025] Fig.14 : is a diagram showing an example of transformed polyline data displayed in a superimposed manner.

[0026] Fig.15 It is a diagram for explaining the polyline data of the divided template.

[0027] Fig.16 This is a diagram for explaining a case where there are multiple score peaks.

[0028] Fig.17 is a block diagram showing an example of a functional configuration of an information processing apparatus according to the second embodiment.

[0029] 18A to 18D It is a diagram for explaining a GUI displayed by an information processing device. DETAILED DESCRIPTION

[0030] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. A plurality of features are described in the embodiments, but are not limited to inventions requiring all such features, and a plurality of such features may be appropriately combined. In addition, in the accompanying drawings, the same reference numerals are given to the same or similar configurations, and their redundant descriptions are omitted.

[0031] In the evaluation of the solidity of a structure, etc., a high-definition captured image is used as an inspection image for comparison, but this increases the processing and storage costs due to the large amount of data.

[0032] An embodiment of the present invention provides an information processing device that reduces the cost of correcting the offset between detection objects in images captured at different times.

[0033] First embodiment

[0034] The information processing device according to the present embodiment obtains first position information indicating the position of an object detected in a first image captured of the object, and second position information indicating the position of the object detected in a second image captured of the object at a different time from the first image. Next, the information processing device estimates a positional shift amount of the object based on the obtained first position information and second position information, and corrects at least one of the first position information and the second position information based on the positional shift amount.

[0035] For example, the information processing device according to the present embodiment can obtain a polyline data group including one or more polyline data indicating the shape of an object by information of each vertex from both the first image and the second image as the first position information and the second position information. In this case, the information processing device generates a function for transforming the coordinates in the coordinate system of the first polyline data into the coordinates in the coordinate system of the second polyline data based on the position offset estimated from the polyline data, and uses the generated function to transform the coordinates indicated by the first polyline data. The first polyline data and the second polyline data will be described below.

[0036] Figure 1A An information processing device according to the present embodiment is shown. The information processing device 100 includes a CPU 101, a ROM 102, a RAM 103, a HDD 104, a display unit 105, an operation unit 106, and an I / F unit 107. The CPU 101 is a central processing unit for performing arithmetic and logic decisions for processing and controlling components connected to a system bus 110. A read-only memory (ROM) 102 is a program memory for storing programs including various types of processing described below controlled by the CPU 101. A random access memory (RAM) 103 is used as a main memory of the CPU 101 or as a temporary storage area such as a work area. Note that the program memory can be implemented by a program loaded onto the RAM 103 from the HDD 104, or an external storage device connected to the information processing device 100.

[0037] HDD 104 is a hard disk for storing electronic data or programs such as image data related to the present embodiment. Note that an external storage device can be used to implement a function similar to that of HDD 104. Here, the external storage device can be implemented by, for example, a medium (storage medium) and an external storage drive for implementing access to the medium. A floppy disk (FD), CD-ROM, DVD, USB memory, or solid-state drive (SSD) or the like can be used as the medium. In addition, the external storage device can be a server device connected to a network or a similar device outside the information processing device 100.

[0038] The display unit 105 is, for example, a liquid crystal display (LCD) or an organic EL display (OLED) for outputting an image on a display screen. Note that the display unit 105 may be an external device connected to the information processing device 100 via a wired or wireless connection. The operation unit 106 is, for example, a keyboard and a mouse, and accepts various types of operations from a user. Note that the operation unit 106 is not particularly limited, and any device capable of accepting input from a user may be used, examples of which include a touch panel and a microphone. The I / F unit 107 performs bidirectional communication with other information processing devices, communication devices, or external storage devices via a wired or wireless connection using known communication technologies.

[0039] Figure 1B is a block diagram showing an example of the functional configuration of the information processing apparatus 100 . Figure 1B The illustrated functional units are realized by the CPU 101 loading a program stored in the ROM 102 onto the RAM 103 and executing processing according to each flowchart described below.

[0040] The acquisition unit 111 acquires position information indicating the position of the subject (object) from each of two or more images captured of the object. Here, the acquisition unit 111 acquires first position information indicating the position of the object detected in the first image, and second position information indicating the position of the object detected in the second image. As the position information according to this embodiment, for example, a first foldline data group and a second foldline data group are acquired. As described below Figure 4C As shown, the polyline data refers to data in which the shape of the subject (object) in the image is indicated by the information of each vertex, and the polyline data group includes one or more polyline data. Here, the polyline data group for indicating the shape of the subject in the first image is defined as the first polyline data group, and the polyline data group for indicating the shape of the subject in the second image different from the first image is defined as the second polyline data group. For example, the obtaining unit 111 can obtain the first polyline data group and the second polyline data group from the polyline data obtained via the I / F unit 107 and stored in the HDD 104. Note that the obtaining unit 111 can obtain the first image and the second image, and generate the polyline data by detecting the subject in these images. In the following, the polyline data may be referred to as "polyline". In addition, in the following, in the case of simply referring to "vertex", this means the vertex represented by the polyline data. Note that in the following, in the case of simply referring to "subject", this means the above-mentioned "object" corresponding to the processing object whose position information is obtained by the obtaining unit 111.

[0041] The information processing device 100 according to the present embodiment estimates the position offset of the object based on the obtained first position information and the second position information. Here, the position offset of the object refers to, for example, the offset of the position of the foldline data included in the first foldline data group and the foldline data included in the second foldline data group. The information processing device 100 according to the present embodiment can evaluate the similarity between the foldline data included in the first foldline data group and the foldline data included in the second foldline data group. In the following, the foldline data included in the first foldline data group is referred to as the first foldline data, and the foldline data compared for similarity with the first foldline data is referred to as the second foldline data.

[0042] The information processing device 100 according to the present embodiment can generate pattern images from both the first foldline data and the second foldline data, and can use these pattern images to perform similarity evaluation via template matching. A configuration for performing similarity evaluation will be described below.

[0043] The image generation unit 112 can generate a pattern image for use in calculating the positional shift amount of the first foldline data and the second foldline data based on the foldline data. The processing performed by the image generation unit 112 will be described below.

[0044] The vector calculation unit 113 estimates the positional offset of the object from the first position information and the second position information obtained by the obtaining unit 111. Here, for example, as the positional offset, an offset vector calculated by the process described below is used. The offset vector calculation process will be described below.

[0045] The function generating unit 114 generates a function for transforming the coordinates in the coordinate system of the first foldline data into the coordinates in the coordinate system of the second foldline data based on the first foldline data group and the second foldline data group. For example, the first foldline data may include one or more foldline data, and the above function may be generated based on the one or more foldline data included in the first foldline data group and the corresponding foldline data included in the second foldline data group. Hereinafter, the function may be referred to as a "transformation function" for short. The function generating unit 114 according to the present embodiment generates a transformation function based on the offset vector calculated by the vector calculating unit 113 for each foldline included in the first foldline data. The following will refer to Fig. 9 The transformation function generation process is described.

[0046] Here, the coordinate system of the first foldline data is the coordinate system in the object image described below, and the coordinate system of the second foldline data is the coordinate system in the template image described below. These will be described below. However, as long as the transformation can be performed in a similar manner, there is no particular limitation on the coordinate system. Corresponding different coordinates may be used, such as a coordinate system of vertices corresponding to the first image having coordinates indicating vertices of the first foldline data, etc.

[0047] The coordinate transformation unit 115 corrects at least one of the first position information and the second position information based on the position offset estimated by the vector calculation unit 113. In the present embodiment, for example, the function generation unit 114 generates a transformation function based on the offset vector calculated by the vector calculation unit 113, and the coordinate transformation unit 115 uses the generated transformation function to transform the coordinates indicated by the first polyline data. Note that in the present embodiment described below, the coordinates indicated by the first polyline data are transformed into coordinates in the coordinate system of the second polyline data. However, the coordinates indicated by the second polyline data may also be transformed into coordinates in the coordinate system of the first polyline data.

[0048] In this embodiment, the first image and the second image are obtained as the inspection image. Here, the inspection image refers to two or more images taken of a subject corresponding to the comparison object, which are used in evaluating the firmness of the structure, for example. Figure 2 as well as Figure 3A and Figure 3B A method for obtaining an inspection image and a situation where an image position shift occurs during the acquisition will be described.

[0049] Figure 2 1 is a schematic diagram for explaining the relationship between the tunnel structure and the captured inspection image when assuming that a captured image of the interior of the tunnel is obtained as an inspection image. Figure 2 , a cross-sectional view 200 of a tunnel and an expanded view 201 of the inner wall of the tunnel are shown. A tunnel lining surface 203, lighting 204 attached at or near the ceiling, and a curb 205 are also shown. In addition, in the expanded view 201, an opening 202 corresponding to the entrance / exit of the tunnel and a span 206 corresponding to a unit of concrete for installing the lining surface are shown. In this embodiment, inspection (capturing of inspection images) is repeated in the advancing direction in units of spans.

[0050] Figure 3A and Figure 3B 2 is a diagram showing a state of capturing an inspection image of a tunnel as viewed from a tunnel cross-sectional view 200. In order to capture an inspection image of a lining surface 203 (inspection object), as shown in FIG. Figure 3AA vehicle 301 with attached line sensor cameras is shown driving along a path in a tunnel. The attached cameras 302 to 304 are line sensor cameras for capturing images in different directions, and boundaries 308 and 309 indicate boundaries of the field of view. Figure 3B 302 to 304 are the same cameras, but the driving position is different from the vehicle 310. Figure 3A The driving position in the illustrated states is different.

[0051] In this case, cameras 302 to 304 are Figure 3A and Figure 3B The two have the same field of view, but because the camera positions are different, even if the same subject is being photographed, the position in the image will change. Figure 3A and Figure 3B In the example, the position of the boundary of the field of view changes from 308 to 311 and from 309 to 312 due to the difference in the distance from the camera to the wall. Therefore, when the inspection image is produced by synthesizing the images captured by the three cameras after the inspection imaging is completed, the size of each image is adjusted, so the difference in the field of view causes the position of the object to shift between the inspection images.

[0052] From the inspection image obtained in this way, a deformation such as a crack is extracted as a subject, and polyline data is generated. As an extraction (subject detection) method, any known method (such as setting a subject area in an image, etc.) can be used. For example, in order to detect the subject, an operator can use dedicated software and track it manually, machine learning can be used to perform automatic detection via image recognition, or a combination thereof can be used. Note that in the example described below, a crack in an image is detected as a subject, and the position and number of each vertex of the crack are generated as polyline data.

[0053] FIG. 4A to FIG. 4E 1 is a diagram for explaining the broken line data according to the present embodiment generated after a crack in a tunnel lining surface is detected. Figure 4A and Figure 4B is a schematic diagram of a polygonal data set extracted from inspection images obtained at different timings (two time periods) for the same span in a tunnel. Figure 4A and Figure 4B Each with Figure 2 The expansion diagram 201 corresponds to a span. Figure 4A In the broken line data group 400, the broken line data corresponding to the ten cracks Ca001 to Ca010 are represented as a straight line group. Figure 4B middle, Figure 4BA broken line 401 obtained at a later timing (in this example, several years later) is shown, Figure 4A One or more of the ten cracks Cb001 to Cb010 corresponding to the above have become longer and their deformation is in progress. In addition, two new cracks Cb011 and Cb012 have appeared. Figure 4B In the example, the broken lines are shown as dotted lines. This is to Figure 4A The crack areas are separated, and this does not mean that the broken line data are discontinuous. Figure 4A and Figure 4B The shown polyline data is stored in the HDD 104 as a file.

[0054] Figure 4C is shown with Figure 4A Table of examples of corresponding line data. Figure 4D is shown with Figure 4B Table of examples of corresponding line data. Figure 4C Three fields are indicated: a deformation ID 411 for identifying each crack, a vertex number 412 for indicating the number of vertices included in the crack, and a vertex coordinate list 413 for indicating the coordinates of each vertex. The vertex coordinate list 413 is a list of coordinate values ​​of each vertex using the upper left corner of the inspection image of one span as the origin, the horizontal direction as the X-axis, and the vertical direction as the Y-axis. Figure 4D The 414 is related to Figure 4B The corresponding vertex coordinate list 414 has the same origin as that in 410 .

[0055] Figure 4E The 420 overlaps when the origin is aligned. Figure 4A and Figure 4B The difference between the two time periods is shown. The difference between the two time periods includes the result of deformation progression due to aging and the difference between the two time periods. Figure 3A and Figure 3B The positional offset caused by the different camera states shown. Rectangular area 421 is a bounding box including the origin and all the polyline data of the two time periods. This will be mentioned again in the description of the following processing.

[0056] In the present embodiment, in the first image and the second image as the inspection image, the coordinates of the broken line data included in the first image are transformed. In other words, for inspection, the broken line data included in the second image is used as a reference (reference image) to transform the coordinates of the broken line data included in the first image into the coordinates in the broken line data coordinate system included in the second image. Here, the second image is, for example, an image taken at a timing closer than the first image. However, the selection of an image used as a reference (but not for coordinate transformation) is not limited to this. For example, in another embodiment, the user can select an image (image for transformation) to be subjected to broken line data coordinate transformation from two images. For example, the acquisition unit 111 can prompt the operator to specify which of the file names of the broken line data of the two time periods to be compared will be used as the image for transformation via a command prompt implemented by the display unit 105 and the operation unit 106. Note that for the designation, the acquisition unit 111 can present the file list as a GUI and accept the selection from the user.

[0057] Figure 5 : is a flowchart showing an example of pattern image generation processing by the image generation unit 112. At the image generation unit 112, a pattern image of a template used in the template matching performed by the vector calculation unit 113 and a pattern image of an object are generated. Here, in the two fold line data obtained by the acquisition unit 111, the first fold line data included in the first fold line data group is set as a template, and the second fold line data included in the second fold line data group is set as an object, but either one may be set as a template. In the case where the area or number of vertices in the two fold line data varies, the smaller one of the two is used as a template and the larger one of the two is used as an object, thereby making it possible to suppress unnecessary searches that are performed in the absence of a matching party. In the case of inspecting deformation in a structure, usually as FIG. 4A to FIG. 4E As shown, the number of vertices included in the polyline data tends to increase over time. Thus, the polyline data set 400 obtained at an earlier time period can be used as a template, while the polyline 401 obtained at a newer time period can be used as an object.

[0058] In step S501, the image generation unit 112 determines a template area. In the present embodiment, as described above, the bounding box of the polyline data included in the polyline data group 400 of the earlier imaging time period is used as a template. Note that in the template, the drawing line width of the polyline data described below in relation to step S503 may be added. Fig. 6A is a schematic diagram showing an example of a template area. Fig. 6A In FIG. 6 , two fold lines 601 and 602 are shown, and rectangular areas 603 and 604 representing template areas are indicated by dotted lines.

[0059] In step S502 , the image generation unit 112 determines a template and an object region for matching. Fig. 7A and Figure 7B It is a schematic diagram for explaining the object area determination processing. Fig. 7A and Figure 7B The figure shows the line data obtained in two different time periods. Fig. 7A , the templates 701 to 710 determined in step S501 are indicated by dashed rectangles. Figure 7B The template 711 of the embodiment of the present invention corresponds to the template 701 at the same coordinates. The object area 712 includes object widths 713 to 716 added in the upward, downward, left and right directions around the template 711. The object widths 713 to 716 can be set to the expected value taking into account the offset of the fold line in two time periods, or can all be the same width. The object widths 713 and 714 in the horizontal direction or the object widths 715 and 716 in the vertical direction can each have the same width, or they can all have different widths. In addition, the object widths of each fold line of templates 701 to 710 can be different, or they can all be the same width.

[0060] In step S503, the image generation unit 112 generates a template image. Figure 6B As shown, the template image may include only the polyline drawn in the area determined in step S501, or as shown in FIG. Figure 6C As shown, all fold lines used as one side of the template can be drawn, and the template area can be obtained via clipping. In the latter case, a portion of other fold lines 602 included in the template area can be included in the template.

[0061] FIG. 8A to FIG. 8C is a diagram for explaining the drawing of a broken line. The image generating unit 112 generates Figure 8B The image shown in FIG. 8 is a white polyline 803 having a constant line width drawn on a black background 802, as an image based on Fig. 8A Alternatively, the image generation unit 112 may generate a template of the fold line 801 as shown. Figure 8C An image is shown with a drawn polyline 804 and a pixel value gradient added in the line width direction centered on the polyline 804. When a pixel value gradient is added like the polyline 804, a certain degree of balance can be expected between tolerance for offset and matching accuracy in template matching.

[0062] In step S504, the image generation unit 112 generates an object image. The image generation unit 112 draws all the polylines 401 obtained in the newer time period as objects in a similar manner to the template of step S503, and cuts out the object region determined in step S502 to obtain an object image. Figure 4E A rectangular area 421 is formed, a background color area is added in the upper, lower, left, and right directions using object widths 713 to 716, and an object area corresponding to the template near the periphery is ensured. Note that in the template image and the object image with the drawn polyline, the polyline is white and the background color is black. However, the color is not particularly limited and can be set to a desired color according to the conditions.

[0063] Fig. 9 1 is a flowchart for explaining the offset vector calculation processing of the vector calculation unit 113. The offset vector according to the present embodiment refers to the offset between the first foldline data and the second foldline data as described above. Here, the offset vector is the offset of the position of the foldline corresponding to the different timing relative to the foldline of the timing of the image for transformation specified by the acquisition unit 111. Due to the occurrence of the above-mentioned position offset, the offset vector changes according to the position in the coordinate space.

[0064] In step S901 , the vector calculation unit 113 performs template matching using the template image generated by the image generation unit 112 and the object image. Fig. 10A 1 is a diagram for explaining template matching performed according to the present embodiment. In an object image 1002 for a template image 1001, polylines 1003 and 1004 are drawn. Fig. 10A The fold lines of the object image 1002 in FIG. 1 are dotted lines. However, the fold lines are not drawn in dotted lines in the pattern image, and the solid lines and dotted lines are used here to distinguish between the fold lines of the template only for the convenience of explanation.

[0065] The vector calculation unit 113 calculates the similarity score for each position of the template image by sliding the template image 1001 from position 1005 to position 1006 vertically and horizontally (in the right direction and downward direction) within the object image 1002. The similarity score calculated here is the similarity between the first polyline data and the second polyline data at each position of the template image. As a score calculation method, any method including general methods such as the sum of squared differences (SSD) and normalized cross correlation (NCC) can be used, and the method is not particularly limited. The sizes (width, height) of the template image 1001 and the object image 1002 are defined as (w, h) and (W, H), respectively, and the matching result is a two-dimensional graph of size (wW, hH). For example, in the case of matching the template image 1001 and the object image 1002 using NCC, the matching result is a cross-section of the horizontal center and the vertical center corresponding to Figures 1007 and 1008.

[0066] In step S902, the vector calculation unit 113 obtains the position where the template image 1001 matches the object image 1002 based on the score map. The matching position is the position on the map where the similarity in the map calculated in step S901 is the largest (the minimum score in the case of SSD and the maximum score in the case of NCC).

[0067] Fig. 10B It is a diagram for explaining the matching position output in step S902. Fig. 10B The relative positions of the two in the region 1012 (the position of the template image relative to the object image) are shown when the template 1009 moves within the object image 1010 including the fold line 1011. Here, for the region 1012 in the figure, the matching score (similarity) is calculated as the score at the center position (x, y) of the region 1012 with the upper left as the origin. Fig. 10B In the example of FIG. 1 , the coordinates of the position 1013 where the template image overlaps with the object image that has not been moved and has not been changed are defined as (x 0 ,y 0 ), and the template 1009 is moved to the region 1014, resulting in the coordinates of the position 1015 with the greatest similarity being defined as (x max ,y max ). Note that when multiple object polylines run side by side, there is a possibility that the similarity with the unmatched polyline is the highest. However, such noise during matching is dealt with by subsequent processing.

[0068] In step S903, the vector calculation unit 113 calculates the offset vector between the polylines of the two time periods according to the matching position obtained in step S902. Fig.11A and Fig. 11BTo describe the offset vector calculation processing according to this embodiment. Fig.11A Shown with Fig. 10B The same example moves template 1009 to the match of 1014. Here, in FIG. 4A to FIG. 4E In the coordinate system of the span image, the coordinates of the position 1100 of the center of the template 1009 before the movement are defined as (x 1 ,y 1 ), and the coordinates of the center position 1101 of the template 1009 during matching are defined as (x 2 ,y 2 The polyline of the image for conversion specified by the acquisition unit 111 is a polyline of the time period when the image of the object is used (in Fig.11A and Fig. 11B In the case of (shown by the gray dashed line in FIG. 1 ), the offset vector of position 1101 corresponds to vector 1102 extending from position 1101 to position 1100. In other words, Fig.11A The offset vector in (x 1 -x 2 ,y 1 -y 2 ). In addition, Fig. 11B As shown, the image for transformation specified by the acquisition unit 111 is a timing polyline used as a template image (in Fig.11A and Fig. 11B In the case of (shown by a black solid line in FIG), the offset vector at position 1100 is indicated by vector 1103. In other words, Fig. 11B The offset vector in (x 2 -x 1 ,y 2 -y 1 ) corresponds. In this way, the vector calculation unit 113 according to the present embodiment can calculate the vector between the position of the template before template matching and the position of the template with the maximum similarity as an offset vector.

[0069] The coordinates of each vertex of the first polyline data according to this embodiment are offset by the same offset vector (for example, the above-mentioned (x 1 -x 2 ,y 1 -y 2)) to transform these coordinates into coordinates in the coordinate system of the second polyline data. However, between the first image and the second image, this is not limited to the subject merely being translated in the image, but there is a possibility that appropriate coordinates cannot be obtained through simple translation using an offset vector. From this point of view, the vector calculation unit 113 according to the present embodiment can calculate the offset vector for each polyline data included in the first polyline data group. In this case, the function generation unit 114 can generate a transformation function based on each offset vector calculated by the vector calculation unit 113.

[0070] In this way, by calculating a plurality of offset vectors and estimating a transformation function based on the plurality of offset vectors, a transformation function that can be applied to each vertex to transform the coordinate system of the polyline can be obtained. Thus, for each vertex of each polyline data included in the first polyline data group, the coordinates can be transformed through transformation using the transformation function, and can be checked.

[0071] According to this embodiment, using the transformation function f, as shown in the following equation (1), an offset vector V is output, which is used to indicate a vector from the input coordinates U to the coordinates after being transformed into the coordinate system of the second polyline data (transformed coordinates).

[0072] V = f(U) Formula (1)

[0073] Note that the vector calculation unit 113 calculates the offset vector V for the polyline data. Here, for example, the vector calculation unit 113 may calculate the offset vector V for one of the polyline data included in the first polyline data group, may calculate the offset vector V for one or more polyline data, or may calculate the offset vector V for all polyline data. Next, the function generation unit 114 generates the offset vector V for the x component V of the offset vector V using x and y as arguments. x and the y component V y The transformation function is modeled by the components in , and the constants (A to H in this example) are estimated to generate the transformation function.

[0074] V x =f x (x,y)=Ax+By+C

[0075] V y =f y (x,y)=Dx 2 +Ey 2 +Fx+Gy+H Formula (2)

[0076] Here, the vector calculation unit 113 calculates the vector of each polyline data (for example FIG. 4A to FIG. 4ECa001 to Ca010 as shown in the figure calculate the offset vector V, and the function generating unit 114 can calculate the constants of the formula (2) based on the calculated offset vector V.

[0077] Note that the offset vector (V x ,V y ) is not limited to the transformation function shown in equation (2), and other models such as affine transformation can be used in consideration of the cause of the offset or the trend of the offset. For example, in the case of using an inspection image of a tunnel lining surface, it is expected that in a short interval of one span, the driving direction (i.e. FIG. 4A to FIG. 4E There will be almost no offset change in the X-axis direction. Therefore, by using only the independent variable y, the constants to be estimated can be reduced.

[0078] Fig.12 12 is a flowchart showing an example of transformation function generation processing performed by the function generation unit 114. In step S1201, the function generation unit 114 sets the coordinates of the first polyline data (the position of the center of the template in this example) and the offset vector calculated by the vector calculation unit 113 as actual measured values ​​of the independent variable and the dependent variable.

[0079] Fig.13A This is a diagram showing that in step S1201, FIG. 4A to FIG. 4E The two time periods of the polyline group shown are candidates for the offset vector calculated as the processing object. In 1300, the polyline data coordinates (x, y) and the y component V of the offset vector transformed using these coordinates are used. y , plot the above actual measured values ​​as three-dimensional coordinates (x, y, V y ). The X-axis 1301 represents the x-coordinate in the coordinate system of the first image, the Y-axis 1302 represents the y-coordinate in the same coordinate system, and the Z-axis 1303 represents the y-component of the transformation vector V y The units used are all pixel numbers. Fig. 7A The actual measured values ​​(x, y, V) corresponding to the templates 701 to 710 y ). In a similar way, you can also generate a graph with three-dimensional coordinates (x, y, V x ), but this description will be omitted in this article.

[0080] In step S1202, the function generation unit 114 excludes the value corresponding to the noise from the value set as the actual measurement value in step S1201. The function generation unit 114 may exclude the value corresponding to the noise from the vector (V ) for one or both of the scalar values ​​of the components through a known process for excluding abnormal values ​​using general statistical processing, such as excluding values ​​whose difference from the overall mean value is greater than a constant multiple of the standard deviation. x ,V y ) excludes the values ​​corresponding to the noise. Note that in the case of a positional shift caused by the difference in the imaging position between the first image and the second image, the tendency of the transformation via the transformation function varies depending on the position. Thus, instead of the overall actual measurement value, FIG. 4A to FIG. 4E The shown area is segmented into partial areas, and outliers in each partial area can be obtained and excluded.

[0081] In step S1203, the function generation unit 114 may perform a transformation function correction based on the number of actual measurement values ​​remaining after excluding the outliers. The model function of formula (2) includes eight constants from A to H. However, in the case where there are few actual measurement values, correct calculation may not be feasible. From this perspective, in step S1202, the function generation unit 114 may use the number of remaining actual measurement values ​​that have not been excluded to correct the model function to reduce the number of constants that need to be calculated. For example, in the case where the number of remaining actual measurement values ​​is less than a predetermined threshold (e.g., the number of constants of the model function), the function generation unit 114 may correct the model function by setting f y The model function is corrected by setting one or both of the constants of the square components of (x, y) to 0, thereby lowering the order of the model function, etc.

[0082] In step S1204, the function generation unit 114 calculates the weight of each actual measurement value to be used in the transformation function generation process in step S1205. For example, the function generation unit 114 may calculate the weight based on the shape of the template. In this case, the function generation unit 114 may calculate the weight according to the shape of the template based on the size in the direction orthogonal to the X component and the Y component of the offset vector. For example, the function generation unit 114 may calculate the weight for f x (x,y) Set the height value of the template image as the weight, and for f y(x, y) sets the width value of the template image as the weight. Here, the function generation unit 114 may set the height value and width value of the template image as the weight without changing, or may set the value obtained by performing a predetermined operation (such as multiplying by a predetermined coefficient, etc.) on each template image as the weight. In addition, the function generation unit 114 may calculate the weight based on the similarity at the time of matching. In this case, the function generation unit 114 may set the weight so that the value of the weight for the object image with a high matching similarity increases.

[0083] In step S1205, the function generation unit 114 adds the weight calculated in step S1204 to the actual measurement value excluding the abnormal value in step S1202, and calculates the constant of the model function corrected in step S1203. Here, the function generation unit 114 can output the model function via a weighted least square method, for example. For the model function generation process, any known technology for outputting a model function based on similar inputs can be used, and its detailed description will be omitted.

[0084] In step S1206, the function generation unit 114 can remove the broken line determined as an abnormal value based on the transformed coordinates and the actual measurement value output using the transformation function generated in step S1205. For example, the function generation unit 114 can set the broken line whose difference between the transformed coordinates and the actual measurement value is 3.0 times or more of the standard deviation as an abnormal value. Note that the abnormal value obtaining method is not limited to this, and other known methods such as using statistical analysis can be used.

[0085] In step S1207, the function generation unit 114 determines whether the number of actual measurement values ​​(the number of broken lines) remaining after the outliers are removed in step S1206 is equal to or less than a threshold value, or determines whether outlier removal has been performed in step S1206. The threshold value is the number of actual measurement values ​​required to obtain the constants of the model function in step S1205, and is, for example, the number of constants. In the case of "No" in step S1207, the process returns to step S1205. In the case of "Yes", Fig.12 The processing is finished.

[0086] Fig. 13B Show Fig.13A The actual measured values ​​shown are based on actual measured values. Fig.12 The three-dimensional graph of the output of the transformation function generated by the process shown. The mesh surface 1305 indicates the Y component V of the offset vector as the output of the transformation function. y Note that according to the order of equation (2), the X component of the offset vector V x The distribution of (not shown) corresponds to the plane.

[0087] The coordinate conversion unit 115 converts the coordinates of each vertex of the polyline of the timing of the image for conversion specified in the acquisition unit 111 using the conversion function generated by the function generation unit 114, and generates the conversion coordinates. Formula (3) indicates the relationship between the coordinates (x, y) before conversion and the coordinates (x', y') after conversion.

[0088]

[0089] Fig.14 1400 shows the result of the coordinate transformation by the coordinate transformation unit 115, and 1400 shows the coordinates indicated by the solid line. Figure 4A The broken line and the Figure 4B The coordinate transformation of the polyline obtained by the polyline overlaps the polyline indicated by the dot-dash line.

[0090] According to this configuration, based on the polyline of multiple images with different shooting timings, a function for transforming the coordinates in the coordinate system of the first image into the coordinate system of the second image can be generated, and the coordinates of the vertices of one polyline can be transformed based on the function. In particular, a template image and an object image are generated, the offset vector of each polyline is calculated using template matching, and the coordinate transformation of each polyline is performed using a transformation function estimated from the offset vector. Therefore, the positional offset of the subject caused by the difference in the shooting state of the image used in polyline extraction can be corrected using only the polyline data without using the shot image. This has the effect of reducing the cost required to correct the positional offset of the subject in the image.

[0091] By using the polyline with a smaller polyline area or fewer vertices of the two polyline data as a template, this also has the effect of suppressing unnecessary matching processing and improving processing speed and correction accuracy. Another effect includes: when drawing a polyline to generate a pattern image, the offset correction accuracy is improved by adding a pixel value gradient in the line width direction.

[0092] Another effect includes reducing the influence of errors in template matching caused by polylines running side by side, etc., by removing offset vectors corresponding to outliers through the processing of steps S1202 and S1206, etc. Another effect includes reducing the influence of polyline data errors caused by tracking errors made by operators when extracting polylines from images and detection errors in image recognition using machine learning, by adding such outlier removal.

[0093] Another effect includes: by correcting the model function through the processing of step S1203 and the like to reduce the constant to be estimated, it is possible to cope with a case where the amount of polyline data corresponding to the template is small. Fig.12In step S1203, weights are added to the actual measured values ​​to improve the accuracy of the transformation function generation. Another effect includes: by adding weights based on the size of the template running orthogonally to the offset component, offset correction of polylines in a wide area with high importance is prioritized.

[0094] First Modification

[0095] In the first embodiment, Figure 5 In step S501 of the pattern image generation process shown in FIG. 1 , the template region is a bounding box including each fold line. In this variation, the number of templates is increased by dividing the template region into smaller partial regions.

[0096] Fig.15 15 is a diagram for explaining an example of line data with a smaller template area and an increased number of templates. Figure 4E The bounding box area of ​​all the polyline data of the two time periods is the polyline drawing area. As shown by the dotted lines, the area is divided vertically and horizontally into grid-like partial areas, and the partial areas of these partial areas, such as the area 1501 indicated by the shadow, on which the polyline is superimposed are used as templates. Fig.15 In the example above, a rectangle with a shaded background is used as a template, but this does not mean that the shaded background is drawn.

[0097] In addition, each polyline can be divided into a certain size, and the bounding box of each divided polyline can be used as a template. For example, when the length of the longest side or diagonal of the bounding box of the polyline is defined as l and the reference length is defined as L, an integer n that satisfies the following formula (4) is obtained, and the polyline is divided into n parts so that the number of vertices in each part is approximately the same.

[0098] Ln≤l≤L(n+1) Formula (4)

[0099] According to this processing, by reducing the size of the template area, the following effects are achieved: improving the accuracy of each offset vector, increasing the number of valid offset vectors, and improving the accuracy of transformation function generation. Another effect includes reducing the size of the template area to reduce the influence of the transformation of the polyline shape caused by the offset difference depending on the position on the matching.

[0100] Second Modification

[0101] For example, in the case of a crack polyline extracted from an inspection image of a tunnel lining surface, it is reasonable that the camera movement path and speed (the main cause of positional offset) hardly change between adjacent spans. Therefore, the trend of positional offsets of adjacent spans is roughly the same, and it can be expected that the same transformation function can be used for the vertices of the span.

[0102] In other words, the function generation unit 114 according to the present modification determines whether the imaging conditions are similar between the polyline data, and uses a common transformation function to perform coordinate transformation for the polyline data having similar imaging conditions. In this case, the function generation unit 114 can apply the transformation function generated for one polyline data to the polyline data having similar imaging conditions as the one polyline data through processing similar to that in the first embodiment. In addition, for example, the function generation unit 114 can generate a transformation function to be used for the polyline data based on all polyline data having similar imaging conditions.

[0103] This variation differs from the first embodiment in that when the transformation function is generated at the function generation unit 114, the offset vector calculated based on the broken line of multiple adjacent spans is used as a whole, as opposed to the case of only one span. The overall area of ​​the offset vector can be divided into a certain number of spans or in a manner that makes the length of the connected spans correspond to a certain distance, and the offset vectors of the vertices of the broken line divided once can be dealt with as a whole. In addition, the function generation unit 114 can deal with all spans in the tunnel as a single span. Alternatively, in a case where the overall number of spans is defined as n, the function generation unit 114 performs the transformation function generation processing related to the i-th span as a whole for the broken lines of each span sliding from the i-th (1≤i≤n) span to the (i+n-1)-th span.

[0104] In the transformation function generation process, the function generation unit 114 combines the entire span area and performs each process using a coordinate system with the upper left as the origin. In this case, the function generation unit 114 sets the value obtained by adding the total span width on the left side to the x obtained in step S903 as the independent variable in the X-axis direction in step S1201.

[0105] Note that in the example described here, processing is performed on continuous spans in tunnel inspection. However, the application of this modification is not limited to such an example. For example, this can be applied to a case where an inspection image related to a bridge deck of a bridge or a wall surface of a building is a processing object, imaging settings such as the distance to the object, angle, and lens focal length are constant, and images are continuously captured over a plurality of inspection units.

[0106] According to this process, by using the offset vector obtained for each continuous inspection unit as a whole in generating the transformation function, the accuracy of the transformation function generation is improved compared to using only a single inspection unit in the generation.

[0107] Third Modification

[0108] In the first embodiment, Fig. 9In the offset vector calculation process shown, the offset vector is calculated using the coordinates when the matching similarity is maximum. However, depending on the configuration state of the polyline of the two time periods, the peak of the matching similarity may not be clearly indicated, or there may be multiple similarity peaks. From this point of view, the vector calculation unit 113 according to this modification considers the matching similarity value and the similarity distribution state when calculating the offset vector.

[0109] Fig.16 is a diagram for explaining a broken line in the case where there are multiple matching similarities. In 1600, a broken line group of two time periods is superimposed. The broken line of the earlier timing is indicated by a solid line, and the broken line of the newer timing is indicated by a dotted line. The broken line of the newer timing shown in the partial area 1601 in 1600 includes a broken line corresponding to the broken line of the earlier timing and a new broken line running side by side. In addition, in Fig.16 , an object region 1602 and a template 1603 are shown in an enlarged manner in relation to a partial region 1601. Using these, a cross section taken along the Y-axis direction at the left and right center positions of the two-dimensional graph of the matching score calculated by the vector calculation unit 113 is shown as a graph 1604.

[0110] As shown in graph 1604, Fig.16 In the example of , template 1603 matches both the polylines above and below the template 1603 in object region 1602. This results in two peaks in the score. In this case, simply using the coordinates with the highest similarity (according to the relationship between the high / low score and the similarity of the matching method) as the matching position has the possibility of producing incorrect results.

[0111] Thus, in the case where there are multiple peaks in the similarity distribution, the vector calculation unit 113 according to the present modification calculates the offset vector for each of these peaks using a process similar to that described in the first embodiment, and performs subsequent processing as it is. Note that here, an area where the similarity is greater than a certain value is set as a matching position (a position corresponding to a peak). However, for example, an area where the similarity is within a predetermined range from the calculated maximum similarity may be set as a position corresponding to a peak, or this may be set according to a condition desired by the user.

[0112] It is reasonable that any of the offset vectors calculated from the multiple peaks corresponds to noise. However, it is expected that these offset vectors are removed as outliers using the process for removing outliers in step S1202 or S1206, etc. Note that the vector calculation unit 113 may calculate the offset vector only in the case of a single peak, and regard the two peaks as noise without calculating the offset vector in the case of multiple peaks. In addition, the vector calculation unit 113 may calculate the offset vector in the absence of a clear peak.

[0113] According to this processing, by performing offset vector calculation taking into account the matching similarity distribution, noise caused by mismatch can be reduced. In particular, in the case where the broken lines run side by side, using processing taking into account multiple peaks can have the effect of improving the accuracy of position offset correction.

[0114] Note that in the present embodiment described above, each process is performed using a structure as an imaging target object and using deformation as a processing target. However, such a limitation is not intended, and it is sufficient that similar processes can be performed based on image detection results.

[0115] Second embodiment

[0116] The information processing device 1700 according to the second embodiment can perform processing similar to that of the information processing device 100 according to the first embodiment. In addition, the information processing device 1700 according to the present embodiment is provided with a function for setting various types of parameters required for coordinate transformation processing and a function for displaying position shift correction results, and provides an interactive GUI for the operator. The hardware configuration of the information processing device 1700 is similar to that of the first embodiment. Figure 1A The hardware configuration shown is similar.

[0117] Fig.17 1700 according to the present embodiment. The functional units are realized by CPU 101 developing the program stored in ROM 102 on RAM 103 and executing the processing according to each flowchart described below. The information processing device 1700 according to the present embodiment includes the same components as those of the first embodiment except that a control unit 1706 and a display control unit 1707 are additionally included. Figure 1B The functional units 1701-1705 are similar to those functional units 111-115 shown, and thus redundant description will be omitted. The control unit 1706 accepts operation of the GUI element displayed on the display unit 105 via the operation unit 106. The display control unit 1707 displays the polyline of the specified two time periods and the polyline after positional offset correction on the display unit 105.

[0118] 18A to 18D1700 according to the present embodiment. These GUI screens are implemented using a window system provided by an OS running on the information processing device 1700. Fig.18A As shown, GUI elements 1801 to 1809 are arranged in a main window 1800 .

[0119] exist Fig.18A , in the case where the operator clicks an area of ​​the file list 1801 using the operation unit 106 (e.g., a mouse), the control unit 1706 displays a file selection dialog box (not shown). Using the file selection dialog box, the operator specifies two polyline data files and the order of obtaining time periods (imaging timings), and the control unit 1706 stores the result. Note that the imaging timings may be stored in association with the polyline data, and in this case, the order of imaging timings may be selected based on the associated information regardless of the user operation.

[0120] When the designation of the polyline data files has been completed, the control unit 1706 displays the file names in the designated acquisition time period order in the file list 1801. Next, the display control unit 1707 displays the polylines included in each file in a superimposed manner in the polyline display area 1809. Fig.18B The screen after file selection is shown, in which two file names are displayed in the field 1810, and the line of each file is displayed in a different color (indicated by a different line type in the figure) in the line display area 1812.

[0121] The control unit 1706 stores the file to be used as the conversion object of the two selected files as a variable, and highlights the file name of the conversion object as shown in 1811. Here, the operator can click the file name in the file list 1801 to switch the conversion object, but the image for conversion can be set as in the first embodiment.

[0122] When the operator clicks and selects the broken line with the earlier obtained time period among the broken lines displayed in the display area 1812 (in 18A to 18D In the case of (indicated by solid lines in FIG. 1 ), the display control unit 1707 Fig.18C When the operator slides the thumb of the line width slider 1802, the control unit 1706 determines the area to be displayed in the line width slider 1802 according to the sliding position. Figure 5 The line width of the polyline drawn when the template image and the object image are generated in steps S503 and S504 is displayed. An example of such a display of the polyline changed to the determined line width is display 1813.

[0123] Furthermore, the display control unit 1707 displays the template area 1814 determined in step S501 and the area 1815 of the object image determined in step S502 in relation to the selected fold line. The control unit 1706 determines and stores the area 1815 of the object image according to the operator's input of moving and enlarging / reducing the area 1815 of the object image. Figure 7B The line width and object area are specified by the operator selecting any foldline. However, this does not necessarily have to be done for each of these foldlines, and the user-specified value can be applied to all foldlines. Note that the line width and object area can be set for each of these foldlines.

[0124] 18A to 18D The template segmentation slider 1803 shown is used to specify the settings for template segmentation described in the first variant of the first embodiment. For example, when the slider 1803 is in "large", no template segmentation is performed, and when the slider 1803 is in "small", the fold line is segmented into smaller parts. For example, between "large" and "small", the constant L of equation (4) changes to 20% of the maximum value of the fold line length.

[0125] 18A to 18D The first slider 1804 for noise removal and the second slider 1805 for noise removal are shown to be used to specify the Fig.12 The threshold used in the outlier removal process of steps S1202 and S1206 is specified. For example, a constant multiplied by the standard deviation used as each threshold is specified in the range of 0.5 to 4.0.

[0126] Note that although not included 18A to 18D In the GUI elements shown, however, other GUI elements related to parameters such as constants used in the weighting in step S1204 may be provided, and inputs related to these parameters may be accepted.

[0127] In addition, if the operator does not specify a value, it can be 18A to 18D The parameters designated by 1801 to 1805 and 1814 shown may use default values. In addition, when the operator clicks a parameter storage button 1808, the values ​​held in various GUIs at that time are stored in the HDD 104 as a file.

[0128] like Fig.18C As shown, when the operator clicks the start calibration button 1807, the calibration is performed using the parameter values ​​held at that time. Figure 5 , Fig. 9 and Fig.12 The processing shown is used to transform the coordinates of the polyline of the transformed image, and the result is displayed in the polyline display area 1809. Fig.18DThe coordinate transformation screen is shown, and as shown in the display screen 1816, the polyline (solid line) of the reference image and the polyline (dotted line) after the coordinate transformation are superimposed. Fig.18D In the case of the return button 1806 in the GUI shown, the Fig.18C The screen displays and the parameters can be adjusted through operator input.

[0129] According to this configuration, an information processing device can be provided, which is configured to specify parameters used in coordinate transformation and display the result of coordinate transformation in addition to executing the various processes described in the first embodiment. This has the effect of providing an interactive GUI to the operator and being able to effectively prompt the operator to set appropriate parameters. Another effect includes: by storing the determined parameters in an external storage device, it is possible to reuse the broken line data obtained under similar conditions.

[0130] Other embodiments

[0131] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0132] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments.The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An information processing device, comprising: an obtaining unit configured to obtain first position information indicating a position of an object detected in a first image captured of the object, and second position information indicating a position of the object detected in a second image captured of the object at a different time from the first image; an estimating unit, configured to estimate a position offset of the object based on the first position information and the second position information; as well as a correction unit configured to correct at least one of the first position information and the second position information based on the position offset, The first position information includes information of each vertex of the object in the first image, and the second position information includes information of each vertex of the object in the second image.

2. The information processing device according to claim 1, wherein: The obtaining unit obtains a first polyline data group including one or more polyline data indicating the shape of the object in the first image by information of each vertex as the first position information, and obtains a second polyline data group including one or more polyline data indicating the shape of the object in the second image by information of each vertex as the second position information.

3. The information processing device according to claim 2, further comprising: a first generating unit configured to generate, based on the position offset, a function for transforming coordinates in a coordinate system of first foldline data included in the first foldline data group into coordinates in a coordinate system of second foldline data included in the second foldline data group; The correction unit corrects the coordinates indicated by the first polyline data based on the function.

4. The information processing device according to claim 3, wherein: the estimating unit estimates, as the positional offset, a similarity between the foldline data included in the first foldline data group and the foldline data included in the second foldline data group, and The first generation unit generates the function based on the similarity.

5. The information processing device according to claim 4, further comprising: a second generating unit configured to generate an object image corresponding to the first foldline data based on the first foldline data included in the first foldline data group, and to generate a template image corresponding to the second foldline data based on the second foldline data included in the second foldline data group, The estimation unit estimates the similarity of the polyline data at each position of the template image through template matching using the template image and the object image.

6. The information processing device according to claim 5, wherein: The second generating unit generates a bounding box region of the first polyline data as the object image.

7. The information processing device according to claim 5, wherein: The second generating unit generates a bounding box area of ​​the second polyline data as the template image.

8. The information processing device according to claim 7, wherein: The template image is an image including a pixel value gradient in a line width direction with the first polyline data as the center in the bounding box region.

9. The information processing device according to claim 5, wherein: The second generating unit generates a partial region including a segmented region of the first polyline data as the template image.

10. The information processing device according to claim 5, wherein: The second generating unit generates, as the template image, a bounding box region of the third polyline data including the third polyline data obtained by segmenting the first polyline data.

11. The information processing device according to claim 5, further comprising: a calculation unit configured to calculate a vector between the position of the template image before template matching and the position with the greatest similarity, The first generating unit generates the function based on the vector calculated by the calculating unit according to the plurality of broken line data included in the first broken line data group.

12. The information processing device according to claim 11, wherein: The first broken-line data is included in the broken-line data included in the first broken-line data group from which the broken-line data whose vector is an outlier is removed.

13. The information processing device according to claim 12, further comprising: A correction unit is configured to correct the function when the number of broken line data after removing the broken line data whose vector is an abnormal value from the broken line data included in the first broken line data group is less than a predetermined threshold.

14. The information processing device according to claim 11, wherein: The first generation unit generates a model function for outputting an X component or a Y component of the vector as the function, wherein the position of the first polyline data in the object image is used as an input.

15. The information processing device according to claim 14, further comprising: A setting unit is configured to set a weight for each vector calculated by the calculation unit, wherein the first generation unit further generates the model function based on the weight for each vector.

16. The information processing device according to claim 3, further comprising: a determination unit configured to determine whether the shooting conditions of the first image and the third image are similar, Wherein, the first generating unit generates, for the third image judged to have similar photographic conditions to those of the first image, a function for transforming the coordinates in the coordinate system of the fourth polyline data included in the third polyline data group including one or more polyline data indicating the shape of the object in the third image by information of each vertex into the coordinates in the coordinate system of the second polyline data, as the same function as the function for transforming the coordinates in the coordinate system of the first polyline data into the coordinate system of the second polyline data.

17. The information processing device according to claim 5, wherein: The number of foldline data included in the first foldline data group is smaller than the number of foldline data included in the second foldline data group.

18. The information processing device according to claim 3, further comprising: a second obtaining unit configured to obtain an input from an operator; a setting unit configured to set a parameter to be used in a correction process using the function based on the input obtained by the second obtaining unit; as well as A presentation unit is configured to present the coordinate correction result from the correction unit to the operator.

19. The information processing device according to claim 1, further comprising: A display unit is configured to display, on a display device, polyline data indicating a shape of the object by information of each vertex based on at least one of the first position information and the second position information corrected by the correction unit.

20. The information processing device according to claim 1, wherein: The object is a structure, and the object is a deformation of the structure.

21. An information processing method, comprising: obtaining first position information indicating a position of an object detected in a first image captured of the object, and second position information indicating a position of the object detected in a second image captured of the object at a different time than the first image; estimating a position offset of the object based on the first position information and the second position information; as well as correcting at least one of the first position information and the second position information based on the position offset, The first position information includes information of each vertex of the object in the first image, and the second position information includes information of each vertex of the object in the second image.

22. A non-transitory computer-readable storage medium storing a program, wherein when the program is executed by a computer, the computer performs an information processing method, the information processing method comprising: obtaining first position information indicating a position of an object detected in a first image captured of the object, and second position information indicating a position of the object detected in a second image captured of the object at a different time than the first image; estimating a position offset of the object based on the first position information and the second position information; as well as correcting at least one of the first position information and the second position information based on the position offset, The first position information includes information of each vertex of the object in the first image, and the second position information includes information of each vertex of the object in the second image.

23. A computer program product comprising a program, which, when executed by a computer, causes the computer to perform an information processing method, the information processing method comprising: obtaining first position information indicating a position of an object detected in a first image captured of the object, and second position information indicating a position of the object detected in a second image captured of the object at a different time than the first image; estimating a position offset of the object based on the first position information and the second position information; as well as correcting at least one of the first position information and the second position information based on the position offset, The first position information includes information of each vertex of the object in the first image, and the second position information includes information of each vertex of the object in the second image.

Images