Digital image correction apparatus, digital image correction method, and program

Abstract

To provide a high-resolution DIC configured to reduce measurement error.SOLUTION: A digital image correction apparatus includes: an image acquisition unit which acquires a digital image obtained by imaging a surface of a measurement object in an imaging range of an angle of view including a random pattern and a correction figure of a predetermined shape; an image correction unit which corrects distortion of the digital image based on a matrix which is determined when converting the correction figure with distortion into a correction figure of the predetermined shape without distortion, in the digital image; and an image analysis unit which estimates displacement or distortion of the measurement object based on the image analysis using the corrected digital image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a digital image correction device, a digital image correction method, and a program. [Background technology]

[0002] Digital image correlation (DIC) is a well-known technology for measuring displacement and strain in structures, etc. In DIC, a random pattern of irregular spots is applied to the surface of the area of ​​interest to be measured, a load is applied to the object to deform it, the shape of the random pattern that changes before and after the deformation is photographed, and the relative movement is analyzed using images to estimate the distribution of displacement and strain.

[0003] Incidentally, Patent Document 1 discloses technology related to DIC. Specifically, Patent Document 1 states that "This displacement measurement sheet is used to acquire images in a digital image correlation method. This displacement measurement sheet is made up of a flexible sheet of a finite shape, and the measurement field of view range of the displacement measurement sheet is divided vertically into n areas and horizontally into m areas, and in each divided area, a pattern of an arbitrary shape and a color different from the background color is formed using digital data for measurement by a digital image correlation method, and the pattern is arranged within the sheet without line symmetry or point symmetry." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2023-71193 Summary of the Invention [Problem to be solved by the invention]

[0005] Because DIC uses image analysis, the measurement resolution of displacement and strain depends on the resolution of the digital image. In other words, the measurement resolution improves if the measurement area can be analyzed using a digital image with a larger number of pixels. Therefore, the upper limit of measurement resolution is limited by the upper limit of the resolution of the camera used. On the other hand, the wider the measurement area in a single image, the fewer the number of pixels per unit area, and therefore the lower the measurement resolution.

[0006] Therefore, when performing wide-area, high-resolution measurements using DIC, it is effective to divide the measurement area into multiple images rather than capturing a single image. In this case, for example, one possible method is to move the measurement object or the camera and capture images while changing (scanning) the measurement area. However, this method requires moving either the camera or the measurement object to capture images before and after deformation of the measurement object, which makes it easy for the positional relationship and orientation of the camera relative to the measurement object to change.

[0007] When the positional relationship and orientation of the camera and the object being measured changes, there is a concern that distortion in the digital image that occurs when the object is not directly facing the camera during DIC image analysis may be mistakenly recognized as being due to distortion or displacement of the object being measured, which could result in measurement errors.

[0008] Patent Document 1 discloses a technique for creating a random pattern as digital data so that the random pattern used in DIC can be reproduced even if the random pattern is damaged. However, the technique in this document does not take into consideration the reduction of measurement errors due to distortion of the digital image. Therefore, even if the technique in this document is applied, it is difficult to solve the above-mentioned problem.

[0009] The present invention has been made in view of the above problems, and has an object to provide a high-resolution DIC that can further reduce measurement errors. [Means for solving the problem]

[0010] The present application includes a plurality of means for solving at least part of the above-described problems, examples of which are as follows: A digital image correction device according to one aspect of the present invention for solving the above-described problems includes an image acquisition unit that acquires a digital image obtained by capturing an image of the surface of a measurement object within a shooting range having an angle of view that includes a random pattern and a correction figure of a predetermined shape, an image correction unit that corrects distortion of the digital image based on a matrix obtained when converting the distortion-containing correction figure in the digital image into a distortion-free correction figure of the predetermined shape, and an image analysis unit that estimates displacement or distortion of the measurement object based on image analysis using the corrected digital image. [Effects of the Invention]

[0011] According to the present invention, it is possible to provide a high-resolution DIC that can further reduce measurement errors.

[0012] Problems, configurations, effects, and the like other than those described above will become clear from the following description of the embodiments. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of a digital image correction system and a functional configuration of a digital image correction device. [Figure 2] 1A and 1B are diagrams showing an example of a process for estimating displacement and strain of a measurement object using DIC. [Figure 3] FIG. 10 is a diagram schematically illustrating a measurement area after a patterning process. [Figure 4] 4A and 4B are diagrams showing an example of a correction graphic. [Figure 5] 5A and 5B are diagrams showing an example of a correction graphic. [Figure 6] FIG. 10 is a diagram showing an example of a correction graphic; [Figure 7] FIG. 10 is a schematic diagram showing an example of a photographing process. [Figure 8] FIG. 2 is a cross-sectional view of a camera and a measurement object in a photographing process. [Figure 9] FIG. 10 is a plan view of a measurement area in which a plurality of correction figures are arranged. [Figure 10] FIG. 10 is a diagram showing an example of the positional relationship between a plurality of correction figures and a shooting range. [Figure 11] FIG. 10 is an explanatory diagram of a correction process. [Figure 12] FIG. 10 is a diagram showing an example in which one correction figure is formed by a plurality of correction figures. [Figure 13] FIG. 10 is a diagram showing another example in which one correction figure is formed by a plurality of correction figures. [Figure 14] FIG. 1 is a diagram illustrating an example of a hardware configuration of a digital image correction device. DETAILED DESCRIPTION OF THE INVENTION

[0014] The following embodiments are examples for explaining the present invention, and some omissions and simplifications have been made as appropriate for clarity of explanation. The present invention can be implemented in various other forms. Furthermore, unless otherwise specified, each component may be singular or plural.

[0015] Furthermore, in order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0016] Furthermore, various types of information may be described using expressions such as "table," "list," and "queue," but the various types of information may be expressed using data structures other than these. For example, various types of information such as "XX table," "XX list," and "XX queue" may be expressed as "XX information." When describing identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are interchangeable.

[0017] In addition, when there are multiple components having the same or similar functions, they may be described by using the same reference numeral with different subscripts, or when there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0018] In addition, in the embodiments, there may be cases where processing performed by executing a program is described. Here, a computer executes the program using a processor (e.g., a CPU or a GPU), and performs processing defined by the program while using storage resources (e.g., memory) and interface devices (e.g., communication ports). Therefore, the processor may be the entity that executes the program and performs the processing.

[0019] Similarly, the entity that executes the program and performs the processing may be a controller, device, system, computer, or node having a processor. The entity that executes the program and performs the processing may be any computing unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit is, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0020] A program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in an embodiment, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0021] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0022] 1 is a diagram showing an example of a schematic configuration of a digital image correction system 1000 according to this embodiment and a functional configuration of a digital image correction device 100. As shown in the figure, the digital image correction system 1000 includes a digital image correction device 100 and a camera 200 that photographs an object to be measured, which are interconnected so as to be able to communicate with each other via a predetermined network N. The network N may be, for example, the Internet, an intranet, a wide area network (WAN), a mobile phone network, or the like.

[0023] The digital image correction system 1000 is a system that uses a digital image of the object to be measured to measure the distribution of displacement and strain of the object to be measured by digital image correlation (hereinafter sometimes referred to as "DIC").

[0024] Camera 200 takes digital images of a measurement object having a random pattern on its surface before and after applying a load to deform it (hereinafter sometimes referred to as "before and after deformation") and sends these images to digital image correction device 100. If the entire measurement area of ​​the measurement object is wider than the imaging range (angle of view) of camera 200, the measurement area is virtually divided, and either camera 200 or the measurement object is moved to take a digital image of each divided measurement area, thereby taking a digital image of the entire measurement area.

[0025] The digital image correction device 100 performs image analysis by DIC using digital images before and after deformation, measures the displacement and strain of the measurement object, and outputs output information that displays the distribution of the displacement and strain.

[0026] A correction figure is attached to the surface of the object to be measured together with a random pattern, and the digital image correction device 100 corrects the distortion of the correction figure photographed together with the random pattern, thereby correcting the distortion of the digital image caused by changes in the positional relationship and posture of the camera 200 relative to the object to be measured, and then performs image analysis using DIC using the corrected digital image.

[0027] This allows the digital image correction device 100 to provide a high-resolution DIC that can further reduce measurement errors.

[0028] <Camera 200> The camera 200 is an imaging device that captures a digital image of the measurement object. The camera 200 captures a digital image of the measurement object (measurement area) including the random pattern and correction figures attached to the surface of the measurement object, and transmits the image to the digital image correction device 100.

[0029] The type of camera 200 is not particularly limited, and any known camera capable of capturing digital images used in image analysis by DIC (e.g., a high-resolution camera capable of fully recognizing random patterns and correction figures in the captured digital images) may be used.

[0030] <Digital image correction device 100> The digital image correction device 100 is a device that measures the displacement and strain of a measurement object based on image analysis by DIC using a digital image of the measurement object. Specifically, the digital image correction device 100 estimates the distribution of displacement and strain by performing image analysis of the relative movement amount of a random pattern before and after deformation of the measurement object.

[0031] Furthermore, the digital image correction device 100 corrects the digital image by correcting distortion of the correction figure photographed together with the random pattern prior to image analysis by DIC.

[0032] 1, the digital image correction device 100 has various functional units. Specifically, the digital image correction device 100 has a processing unit 110, a storage unit 120, and a communication unit 130.

[0033] The processing unit 110 is a functional unit that performs processing executed by the digital image correction device 100. Specifically, the processing unit 110 has an image acquisition unit 111, an image correction unit 112, an image analysis unit 113, and an output information generation unit 114 as individual functional units that perform various processes.

[0034] The image acquisition unit 111 is a functional unit that acquires digital images of the measurement object from the camera 200. Specifically, the image acquisition unit 111 acquires digital images of the measurement object before and after deformation via the communication unit 130, and stores these in the storage unit 120 (captured image DB).

[0035] The image correction unit 112 is a functional unit that corrects distortion of the digital image. Specifically, the image correction unit 112 performs homography transformation on the coordinates of the correction figure for each digital image obtained by photographing each measurement area of ​​the measurement object before and after deformation, thereby correcting distortion of the digital image caused by changes in the positional relationship and posture of the camera 200 with respect to the measurement object. Details of the correction method using the correction figure will be described later.

[0036] The image analysis unit 113 performs image analysis by DIC using the corrected digital image and calculates the distribution of displacement and strain of the measurement object. Note that DIC is a well-known technology and will not be described in detail here, but the image analysis unit 113 identifies random patterns within a subset (a rectangular area of ​​about several tens of pixels) identified from a digital image capturing the state of the measurement object before deformation from a digital image capturing the state after deformation, and calculates the displacement and strain of the measurement object based on the amount of displacement (amount of relative movement) and the coordinate changes of the surrounding random patterns.

[0037] The output information generation unit 114 is a functional unit that generates predetermined types of output information visualizing the displacement and strain of the measurement object. For example, the output information generation unit 114 generates output information such as a contour map or a color map indicating the displacement and strain of the measurement object. Further, the output information generation unit 114 causes the output information to be displayed on a display device (e.g., a display, etc.) included in the digital image correction device 100 or an external display device connected via the communication unit 130.

[0038] Next, the storage unit 120 will be described. The storage unit 120 is a functional unit that stores various information. Specifically, the storage unit 120 has a photographed image DB 121. Note that the photographed image DB 121 is a database that stores digital images of the measurement object before and after deformation photographed by the camera 200.

[0039] Next, the communication unit 130 will be described. The communication unit 130 is a functional unit that performs information communication with an external device. Specifically, the communication unit 130 acquires a digital image of the measurement object photographed by the camera 200. Further, the communication unit 130 outputs, for example, the output information generated by the output information generation unit 114 to an external display device.

[0040] The functional configuration of the digital image correction device 100 has been described above.

[0041] <Displacement and Strain Estimation Processing by DIC> FIG. 2 is a diagram showing an example of displacement and strain estimation processing by DIC. As shown in the figure, the processing for estimating (calculating) the displacement and strain of the measurement region is roughly classified into a patterning step P1, a photographing step P2, an analysis step P3, and a display step P4. Note that the photographing step P2 is executed by the camera 200, and the processing related to the analysis step P3 and the display step P4 is executed by the digital image correction device 100.

[0042] <<Patterning Step P1>> In the patterning process P1, a random pattern of irregular spots is applied to the measurement area of ​​the object to be measured. Also, in the patterning process P1, unlike the random pattern, a correction figure having a predetermined shape is applied to the measurement area of ​​the object to be measured.

[0043] 3 is a diagram showing a schematic diagram of a measurement area after the patterning step P1 is performed. In the example shown, a random pattern 1 is applied to a measurement area 4 that is wider than the imaging range (angle of view) 2 of the camera 200. Furthermore, a correction figure 3 is applied in a size such that its outline (outline) fits within the imaging range 2.

[0044] The random pattern 1 is used to estimate the amount of movement of each coordinate on the surface of the object to be measured before and after deformation using DIC. As long as the random pattern 1 is an irregular spotted pattern, it may be the pattern of the material itself of the object to be measured, or may be a pattern applied to the surface of the object to be measured for another purpose, such as for design or surface treatment of the object to be measured.

[0045] Other methods for applying the random pattern 1 include spraying, brushing, stamping with a random pattern, sealing, and laser marking. Typically, a white or black paint is applied as a base, and then a different color from the base, either white or black, is applied on top of that as the random pattern 1, and image analysis is performed using DIC based on the black and white luminance distribution.

[0046] Note that the random pattern 1 is not limited to black and white or other visible colors, and may be displayed in, for example, the infrared or ultraviolet region, as long as image analysis by DIC is possible. In this case, there is an advantage that it is not necessary to remove the random pattern 1 from the measurement object after processing.

[0047] As shown in the figure, the correction figure 3 is a figure of a predetermined shape, such as a polygon (for example, a square or a triangle). Unlike the random pattern 1, the correction figure 3 preferably has a shape that makes it easy to perform image analysis of each individual shape. Furthermore, a plurality of correction figures 3 may be arranged within the measurement area 4.

[0048] The correction figure 3 may also be applied to the surface of the object to be measured for other purposes, such as design, surface treatment of the object to be measured, wiring for an electric circuit, etc. Other methods for applying the correction figure 3 include, for example, stamping, roller, printer, and laser marking.

[0049] Furthermore, the correction figure 3 is not limited to visible colors, and may be displayed in, for example, the infrared or ultraviolet region, as long as its shape can be recognized by the camera 200. In this case, there is an advantage that it is not necessary to remove the correction figure 3 after processing.

[0050] 4 to 6 are diagrams showing examples of correction figures. For example, as shown in Fig. 4A, a single correction figure 3 may be a figure in which multiple squares are arranged alternately within a rectangular frame. Alternatively, as shown in Fig. 4B, a single correction figure 3 may be a figure in which smaller squares are arranged inside a larger outer square.

[0051] 5A and 5B, the correction figure 3 may be formed as a set of a plurality of correction figures 3 arranged so as to fit within the shooting range 2. Also, as shown in Fig. 6, the correction figure 3 may be formed as a set of a plurality of correction figures 3 having different shapes.

[0052] Although not shown, the correction graphic 3 may be a two-dimensional barcode such as a QR code (registered trademark) or a two-dimensional digital matrix code such as a DM code (registered trademark).

[0053] <<Photography process P2>> In the photographing step P2, digital images including the random pattern 1 and the correction figure 3 before and after the deformation of the measurement object are photographed by the camera 200.

[0054] 7 is a schematic diagram showing an example of the photographing process. As shown in the figure, in the photographing process P2, a camera 200 is placed in a position directly facing a measurement area 4 of the measurement object. For convenience, a flat measurement object 5 is shown in the figure, but the measurement object 5 may have a more complex shape. In the photographing process P2, the camera 200 or the measurement object 5 is moved so that the correction figure 3 falls within the photographing range 2 of the camera 200, and then the camera 200 photographs a digital image of the measurement area 4 included in the photographing range 2.

[0055] Fig. 8 is a cross-sectional schematic diagram of the camera 200 and the measurement object in the photographing process. Although Fig. 7 shows the state in which the measurement object 5 is photographed by one camera 200, as shown in Fig. 8, the digital image correction system 1000 may be configured using a pair of stereo cameras.

[0056] When the distance between the camera 200 and the measurement area 4 is unlikely to change before and after deformation of the measurement object, using a single camera 200 has the advantage of being easier to measure. On the other hand, when it is better to consider changes in the distance between the camera 200 and the measurement area 4, a configuration using a stereo camera as shown in FIG. 8 is effective. When a stereo camera is used, the parallax between the two cameras 200 can be used to calculate the distance between the camera 200 and the measurement area 4. This allows the digital image correction device 100 to perform highly accurate correction (e.g., enlargement or reduction) that takes the distance into consideration in the analysis process.

[0057] As shown in the figure, a light 6 may be installed to illuminate the measurement area 4. When moving the camera 200 to change the imaging range 2, it is preferable that the light 6 be fixed to the same jig as the camera 200 so that the camera 200 and the light can move simultaneously in unison. This keeps the angle of the light 6 relative to the imaging range 2 constant, making it less likely that the way the light hits the area will change, and it is possible to prevent the brightness distribution of the digital image acquired for DIC from changing due to the position of the light.

[0058] Furthermore, the photographing step P2 is preferably performed in a darkroom or the like, or alternatively, a camera 200, a light 6, and a cover 7 covering at least the photographing area 2 above the measurement area 4 are installed, as shown in the figure. Such a cover 7 reduces the influence of light from the sun or lighting fixtures, thereby suppressing changes in the brightness distribution of the digital image due to differences in location and time. When moving the camera 200, the cover 7 is preferably fixed to the same jig as the camera 200 so that the camera 200 and cover 7 can move simultaneously in unison. Furthermore, it is preferable that the cover 7 and the measurement object 5 are in contact with each other. This reduces the gap between the cover 7 and the measurement object 5 and suppresses light penetration. Furthermore, it is preferable that the contact surface between the cover 7 and the measurement object 5 be made of a cushioning material such as a soft brush or felt. This prevents the random pattern 1 and the correction figure 3 from being damaged by contact with the cover 7.

[0059] 9 is a plan view of a measurement area in which multiple correction figures are arranged. In the photographing step P2, the camera 200 or the measurement object 5 is moved to scan the measurement area 4. The camera 200 determines whether the correction figure 3 falls within a specific position in the photographing range 2, for example, based on image analysis such as template matching or contour detection, and photographs the measurement area 4 included in the photographing range 2 when it is determined that the correction figure 3 falls within the position. As a result, even if the positional relationship changes due to the movement of the camera 200 or the measurement object 5, digital images of the measurement area 4 including the random pattern 1 and the correction figure 3 before and after deformation can be photographed again at the same position.

[0060] 10 is a diagram showing an example of the positional relationship between a plurality of correction figures and the imaging range. As shown in the figure, a plurality of correction figures 3 are added to the measurement area 4 at intervals narrower than the width of the imaging range 2. By arranging the correction figures 3 at such intervals, when the camera 200 or the measurement object 5 is moved to change the imaging range 2, parts of the imaging range 2 overlap. This prevents gaps from occurring between the imaging ranges 2, which is effective when comprehensively measuring the measurement area 4.

[0061] The camera 200 transmits the captured digital image to the digital image correction device 100 via the network N. The image acquisition unit 111 of the digital image correction device 100 acquires the digital image from the camera 200 via the communication unit 130 and stores the acquired digital image in the captured image DB 121.

[0062] <<Analysis process P3>> In the analysis step P3, the digital image correction device 100 corrects distortion of the digital image based on the correction figure 3. Furthermore, the digital image correction device 100 estimates the displacement and distortion of the measurement object 5 by calculating the relative movement amount of the random pattern 1 based on image analysis by DIC using the corrected digital image.

[0063] Fig. 11 is an explanatory diagram of correction processing based on a correction figure. Fig. 11A shows a shooting range 21 and a correction figure 31 of a digital image without distortion, i.e., a digital image captured with camera 200 facing approximately directly toward measurement object 5. Fig. 11B shows a shooting range 22 and a correction figure 32 of a digital image with distortion, i.e., a digital image captured with camera 200 not facing directly toward measurement object 5.

[0064] In the distortion-free digital image (FIG. 11A), two axes parallel to sides 10 and 11 defining shooting range 21 are defined in advance as x and y. Also, assume that the contour of distortion-free correction figure 31 is rectangular (FIG. 11A), and that camera 200 is not directly facing measurement area 4, resulting in a trapezoidal correction figure 32 being captured (FIG. 11B).

[0065] In this case, if the trapezoidal correction figure 32 is transformed so that it matches the contour (rectangle) of the distortion-free correction figure 31, it is possible to correct the influence of the orientation of the camera 200. An example of the transformation method is homography transformation.

[0066] When using homography transformation, the transformation from the coordinates (x, y) of an arbitrary distorted rectangle (corresponding to correction figure 32) to the coordinates (x', y') of an arbitrary undistorted rectangle (corresponding to correction figure 31) can be calculated based on the following equations (1) and (2). x′=(ax+by+c) / (gx+hy+1)···(1) y′=(dx+ey+f) / (gx+hy+1)···(2)

[0067] Here, a, b, c, d, e, f, g, and h represent the components of the homography transformation matrix. For these eight unknowns, a, b, c, d, e, f, g, and h, the x and y coordinates of the four vertices of the quadrangle (the trapezoidal correction figure 32 before transformation) and the x and y coordinates of the four vertices of the quadrangle (the rectangular correction figure 31 after transformation) can be determined based on the pixel positions of the digital image. Therefore, eight equations can be formulated to determine the homography transformation matrix. In particular, since the dimensions of the distortion-free quadrangle (correction figure 31) are predetermined and known, using this known dimensional information assigned to the surface of the measurement object 5 as coordinate information after the homography transformation can further improve the accuracy of the transformation, i.e., the correction accuracy of the digital image itself. Note that this homography transformation can be applied to quadrangles of any shape, not just trapezoids and rectangles. Furthermore, the above explanation of the homography transformation has been based on the assumption that a quadrilateral correction figure is used, but the homography transformation can also be applied to polygons of any shape (for example, triangles) in a similar manner.

[0068] Using the homography transformation matrix thus obtained, the coordinates of each pixel in the distorted capture range 22 (FIG. 11B) captured as a trapezoidal shape can be converted into coordinates in the undistorted rectangular capture range 21, thereby correcting the digital image to reduce the influence of changes in the posture of the camera 200. Note that this correction process is performed by the image correction unit 112. Specifically, the image correction unit 112 performs the correction process on all digital images stored in the captured image DB 121 that are captured of the measurement area 4 before and after deformation of the measurement object 5.

[0069] Next, a correction method when a single correction figure is formed by combining a plurality of correction figures 3 will be described with reference to FIGS.

[0070] 12 is a diagram showing an example in which one correction figure is formed from a plurality of correction figures. As shown in the figure, one rectangle and three triangles, which are individual correction figures 3, are located at the four corners of the shooting range 2, and one virtual correction figure 52 is formed by connecting the vertices 51 of each correction figure 3. In this case, the image correction unit 112 applies homography transformation to the four vertices 51 of the virtual correction figure 52, thereby correcting the distorted digital image in the same manner as described above.

[0071] 13 is a diagram showing another example in which one correction figure is formed by a plurality of correction figures. As shown in the figure, correction figures 3 having the shape shown in FIG. 4B are located at the four corners of the imaging range 2. In this case, correction vertices 51 are defined in advance for the rectangles inside each correction figure 3, and one virtual correction figure 52 is formed by connecting the correction vertices 51. In this case, the image correction unit 112 also applies homography transformation to the four vertices 51 of the virtual correction figure 52, thereby correcting the distorted digital image in the same manner as described above.

[0072] In this way, the digital image correction device 100 can achieve higher-precision correction by correcting a digital image using one virtual correction figure 52 that is a combination of multiple correction figures 3. This is because the greater the distance from the center of the imaging range 2 to the correction figure 3, the more error there will be if the same homography transformation matrix is ​​applied to both the correction figure 3 and the imaging range 2.

[0073] 11, one correction figure 32 is positioned away from the center of the imaging range 22, and applying a homography transformation matrix calculated using this correction figure 32 to the imaging range 22 results in a large error. In contrast, if a single virtual correction figure 52 formed by combining multiple correction figures 3 is used, the center of the imaging range 22 can be positioned inside the virtual correction figure 52, thereby narrowing the distance between them. Therefore, the digital image correction device 100 can improve the accuracy of digital image correction by calculating a homography transformation matrix using this virtual correction figure 52.

[0074] The image analysis unit 113 uses the corrected digital image to estimate (calculate) the displacement and strain of the measurement object 5 based on the image analysis by DIC.

[0075] <<Display process P4>> In the display step P4, output information for displaying the displacement and strain of the measurement object 5 estimated in the analysis step P3 is generated and displayed on a predetermined device. Specifically, the output information generation unit 114 generates output information that visualizes the displacement and strain of the measurement object 5, such as a contour diagram or a color map diagram. The output information generation unit 114 also displays the output information on, for example, a display device included in the digital image correction device 100 or a display device of an external device connected via the communication unit 130.

[0076] The above has described the process of estimating the displacement and strain of the measurement object 5 using DIC.

[0077] This digital image correction device can provide high-resolution DIC that can further reduce measurement errors. In particular, the digital image correction device uses a correction figure attached to the measurement object along with a random pattern to pre-correct the digital image to be used for DIC image analysis. This allows the digital image correction device to perform DIC image analysis using a corrected digital image in which distortions caused by changes in the positional relationship between the camera and the measurement object or changes in posture have been corrected. Therefore, the digital image correction device can estimate the displacement and strain of the measurement object with high resolution and precision.

[0078] In image analysis using DIC (digital image correlation), calibration is sometimes performed during the imaging process. For example, it is known that in an imaging process using two or more cameras, an image of a checkerboard with a black and white checkerboard pattern is captured, and the position and orientation of the measurement target relative to the camera can be estimated based on calibration using the image information. However, measurement errors can occur during calibration, or due to changes in the measurement environment after calibration. Another issue is that the above-mentioned general calibration alone is prone to errors, such as distance in the depth direction.

[0079] For this reason, in the present invention, more accurate correction of the digital image is achieved by using known dimensional information of the measurement object 5. This known dimensional information of the measurement object 5 refers to dimensional information of the correction figure 3 attached to the surface of the measurement area 4, and since the dimensions of the correction figure 3 are known (because a figure of a predetermined shape and size is attached to the surface of the measurement object 5), this dimensional information is used for correction (transformation matrix during correction), thereby improving the correction accuracy of the digital image.

[0080] DIC is a technology for measuring displacement and strain of a measurement object, and when the camera is moved, more accurate estimation is required than ever before. The digital image correction device 100 can meet this requirement by achieving more accurate correction of digital images based on the correction described above. Furthermore, the technology related to digital image correction of the present invention does not exclude the combination with conventional general calibration, and the digital image correction device 100 may perform both the correction process described above and conventional calibration.

[0081] <Hardware Configuration of Digital Image Correction Device 100>

[0082] 14 is a diagram showing an example of the hardware configuration of a digital image correction device 100. As shown in the figure, the digital image correction device 100 has an input device 310, a display device 320, a processing device 330, a main memory device 340, an auxiliary memory device 350, a communication device 360, and a bus 370 that electrically interconnects these devices.

[0083] The input device 310 is, for example, a touch panel, a keyboard, a mouse, etc. The display device 320 is a display device such as a liquid crystal display or an organic display.

[0084] The processing device 330 is, for example, a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The main storage device 340 is a memory device (memory resource) such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The digital image correction device 100 has at least one processor and one or more memory resources.

[0085] The auxiliary storage device 350 is a non-volatile storage device capable of storing digital information, such as a so-called hard disk drive, a solid state drive (SSD), or a flash memory.

[0086] The communication device 360 ​​is a wired communication device that performs wired communication via a network cable, or a wireless communication device that performs wireless communication via an antenna.

[0087] An example of the hardware configuration of the digital image correction device 100 has been described above.

[0088] The processing unit 110 of the digital image correction device 100 is realized by a program that causes the processing device 330 to perform processing. This program is stored in the main memory device 340 or the auxiliary memory device 350, and is loaded onto the main memory device 340 and executed by the processing device 330 when the program is executed.

[0089] The storage unit 120 is realized by a main storage device 340, an auxiliary storage device 350, or a combination of these. The communication unit 130 is realized by a communication device 360.

[0090] Furthermore, the above-described configurations, functions, processing units, and processing means of the digital image correction device may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The above-described configurations and functions may also be implemented in software, with a processor interpreting and executing programs that implement the respective functions. Information such as programs, tables, and files that implement the respective functions may be stored in storage devices such as memory, hard disks, and SSDs, or in recording media such as IC cards, SD cards, and DVDs.

[0091] Furthermore, the present invention is not limited to the above-described embodiments and modifications, and includes various modifications within the scope of the same technical concept. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0092] In addition, in the above explanation, the control lines and information lines are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be considered that almost all components are interconnected. [Explanation of symbols]

[0093] 1000: Digital image correction system, 100: Digital image correction device, 110: Processing unit, 111: Image acquisition unit, 112: Image correction unit, 113: Image analysis unit, 114: Output information generation unit, 120: Memory unit, 121: Captured image DB, 130: Communication unit, 310: Input device, 320: Display device, 330: Processing device, 340: Main memory device, 350: Auxiliary memory device, 360: Communication device, 370: Bus, 200: Camera, N: Network

Claims

1. an image acquisition unit that acquires a digital image of the surface of the measurement object within a photographing range having an angle of view that includes the random pattern and the correction figure of a predetermined shape; an image correction unit that corrects distortion of the digital image based on a matrix obtained when converting the distortion-containing correction figure into a distortion-free correction figure of the predetermined shape in the digital image; and an image analysis unit that estimates the displacement or strain of the measurement object based on image analysis using the corrected digital image. A digital image correction device characterized by:

2. 2. The digital image correction device according to claim 1, The matrix is ​​a homography transformation matrix, The image correction unit Correcting distortion of the digital image based on the homography transformation matrix. A digital image correction device characterized by:

3. 2. The digital image correction device according to claim 1, The image analysis unit estimates the displacement or strain based on image analysis using a digital image correlation method. A digital image correction device characterized by:

4. 2. The digital image correction device according to claim 1, The photographing range includes at least one of the correction figures. A digital image correction device characterized by:

5. 2. The digital image correction device according to claim 1, The photographing range includes a plurality of correction figures having the same shape or different shapes. A digital image correction device characterized by:

6. 5. The digital image correction device according to claim 4, When the imaging range includes a plurality of the correction figures, the plurality of correction figures are arranged at intervals narrower than the imaging range. A digital image correction device characterized by:

7. 2. The digital image correction device according to claim 1, The imaging range includes a plurality of correction figures positioned at the four corners of the imaging range. A digital image correction device characterized by:

8. 8. The digital image correction device according to claim 7, The image correction unit Regarding a virtual correction figure formed by connecting the vertices of the correction figures located at the four corners of the photographing range, distortion of the digital image is corrected based on a matrix obtained when converting the virtual correction figure having distortion into a virtual correction figure having a predetermined shape without distortion. A digital image correction device characterized by:

9. 2. The digital image correction device according to claim 1, The distortion of the digital image is caused by changes in the positional relationship and posture between the camera that captured the digital image and the object to be measured when the imaging range moves. A digital image correction device characterized by:

10. 2. The digital image correction device according to claim 1, At least one of the random pattern and the correction figure is displayed in a visible color or in the infrared or ultraviolet region. A digital image correction device characterized by:

11. 2. The digital image correction device according to claim 1, The image processing device further includes an output information generating unit that generates output information indicating the displacement or strain of the measurement object estimated based on the image analysis and outputs the output information to a predetermined display device. A digital image correction device characterized by:

12. A digital image correction method performed by a digital image correction device, comprising: The digital image correction device includes: an image acquisition step of acquiring a digital image of the surface of the measurement object within a photographing range having an angle of view that includes the random pattern and the correction figure of a predetermined shape; an image correction step of correcting distortion of the digital image based on a matrix obtained when converting the distortion-containing correction figure into a distortion-free correction figure of the predetermined shape in the digital image; and an image analysis step of estimating the displacement or strain of the measurement object based on image analysis using the corrected digital image. A digital image correction method comprising:

13. A program that causes a computer to function as a digital image correction device, The computer an image acquisition unit that acquires a digital image of the surface of the measurement object within a photographing range having an angle of view that includes the random pattern and the correction figure of a predetermined shape; an image correction unit that corrects distortion of the digital image based on a matrix obtained when converting the distortion-containing correction figure into a distortion-free correction figure of the predetermined shape in the digital image; and functioning as an image analysis unit that estimates the displacement or strain of the measurement object based on image analysis using the corrected digital image. A program characterized by:

Images