Inspection system and inspection method

Abstract

It allows for easy identification of areas with significant strain. [Solution] The smoothing unit spatially smooths the brightness of an image of an object represented using multiple pixels. The displacement calculation unit searches for candidate regions of the target image to be inspected, which have had their brightness smoothed, based on the similarity to the brightness distribution of each inspection region of the reference image whose brightness has been smoothed, and determines the displacement up to the searched candidate region for each inspection region. The characteristic analysis unit calculates the strain of the object based on the displacement.

Description

Technical Field

[0001] Embodiments of the present application relate to an image processing apparatus, an image processing system, an image processing method, and a program.

Background Art

[0002] In the design and construction of the pile - ground system of a structure, it is important to grasp the ground deformation region with respect to the design external force. Generally, the ground deformation of an actual structure has confinement pressure dependence of the ground material. Therefore, it is common for the ground deformation to be measured by conducting a full - scale experiment or a reduced - scale experiment in a centrifuge field. In the measurement of ground deformation, an in - ground displacement meter may be used. Considerable labor and cost are required for the preparation such as the setting or installation of the in - ground displacement meter. Also, in a reduced - scale experiment, although a reduced - scale model is used, which is different from a full - scale experiment, the existing in - ground displacement meters are too large in size. Therefore, precise measurement may not be possible.

[0003] On the other hand, as a method for grasping the behavior of a fluid, an image processing method called Particle Image Velocimetry (PIV method) may be used. The PIV method is a method for measuring the amount of movement of an inspection region in a target image to be observed by comparing the brightness between two images. According to PIV, the amount of movement (displacement) of the inspection region is obtained with a pixel, which is the minimum unit constituting the image, as the basic unit (pixel element unit) (see Non - Patent Document 1).

[0004] The PIV method has primarily been used to track the dynamic behavior of fluids, but it has also been applied to the property analysis of solid materials (e.g., Non-Patent Document 2). Generally, solid materials have the property that their material properties change in response to deformation. Such material properties include, for example, temperature, strain, and stress. Of these, strain corresponds to the deformation gradient, and can therefore be detected using the PIV method. Detecting areas of significant strain is useful in various fields. Taking construction materials as an example, it can be used to determine areas of members that require reinforcement or repair. The PIV method is also sometimes applied to measuring strain in the ground. Thus, it is important to analyze the overall distribution of strain and strain-concentrated areas where significant strain appears locally. Generally, strain distribution can only be theoretically understood under limited and simple conditions. Under complex real-world conditions, there is little numerical analysis experience with strain distribution, making accurate and efficient understanding difficult. Therefore, by applying the PIV method, it is expected that the labor and costs required for setting up and configuring equipment such as strain gauges and ground displacement gauges can be significantly reduced. For example, by applying the PIV method based on image processing to an experimental specimen, the strain distribution for any given ground can be obtained. By referring to the obtained strain distribution, it is possible to understand in advance the strain distribution that may occur due to construction work, etc., before actually carrying out pile construction work. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Visualization Society of Japan, PIV Handbook, 2nd Edition, May 31, 2018. [Non-Patent Document 2] Takatoshi Kiriyama, Takumi Hirai, Toshiyuki Iwai, "Basic Research on the Pull-Out Resistance Mechanism of Enlarged-Base Piles (Part 1: Centrifugal Experiment and Image Analysis)," Japanese Geotechnical Society, Proceedings of the 55th Annual Meeting of the Japanese Geotechnical Society, 21-9-2-07, July 21, 2020. [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] The PIV method sometimes uses high-resolution images (typically over 10 million pixels per frame). Using high-resolution images allows for the visualization of detailed deformation of the subject. The spatial derivative of displacement is calculated as strain. However, it has sometimes been difficult to identify areas of significant strain from a strain distribution containing a large amount of information.

[0007] This embodiment aims to easily identify areas where strain is significant. [Means for solving the problem]

[0008] The image processing apparatus according to the first embodiment includes: a smoothing unit that spatially smooths the brightness of an image of an object represented using a plurality of pixels; a displacement calculation unit that searches for candidate regions of a target image to be inspected, which has had its brightness smoothed, based on the similarity to the brightness distribution of each inspection region of a reference image whose brightness has been smoothed, and determines the displacement to the searched candidate region for each inspection region; and a characteristic analysis unit that calculates the strain of the object based on the displacement.

[0009] The second aspect of the image processing method is an image processing method in an image processing apparatus, comprising: a smoothing step of spatially smoothing the brightness of an image of an object represented using a plurality of pixels; a displacement calculation step of searching for candidate regions of a target image to be inspected, which has had its brightness smoothed, based on the similarity with the brightness distribution of each inspection region of a reference image used as a reference for inspection, and determining the displacement to the searched candidate region for each inspection region; and a characteristic analysis step of calculating the strain of the object based on the displacement. [Effects of the Invention]

[0010] According to this embodiment, areas with significant strain can be easily identified. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic block diagram showing an example of the functional configuration of the image processing system according to this embodiment. [Figure 2] This flowchart shows an example of displacement analysis according to this embodiment. [Figure 3] This figure shows the first example of the spatial distribution of displacement and strain. [Figure 4] This figure shows a second example of the spatial distribution of displacement and strain. [Figure 5] This figure shows a third example of the spatial distribution of displacement and strain. [Figure 6] This figure shows the first example of a reference image and a target image. [Figure 7] This figure shows a second example of a reference image and a target image. [Figure 8] This figure shows a third example of a reference image and a target image. [Figure 9] This figure shows an example of a strain distribution display. [Figure 10] This is a schematic block diagram showing an example of the hardware configuration according to this embodiment. [Modes for carrying out the invention]

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Figure 1 is a schematic block diagram illustrating an example of the functional configuration of the image processing system 1 according to this embodiment. The image processing system 1 comprises an image processing device 10, an imaging unit 20, and a display unit 30.

[0013] The image processing device 10 acquires image data from the imaging unit 20 that represents an inspection image to be used as a reference for inspection, and image data that represents a target image to be inspected. The image processing device 10 spatially smooths the brightness of each image, and for each inspection area of ​​the brightness-smoothed reference image, it searches for candidate areas of the brightness-smoothed target image based on the similarity to the inspection area. The image processing device 10 determines the displacement to the searched candidate area for each inspection area, and calculates the strain of the object based on the determined displacement. The image processing device 10 generates a strain distribution showing the strain for each inspection area and configures a display screen including the generated strain distribution. The image processing device 10 outputs display data showing the configured display screen to the display unit 30. In other words, the image processing device 10 is characterized by performing a smoothing process as a preprocessing step for both the inspection image representing the test object and the target image, and then applying the PIV method to the smoothed brightness images of the brightness-smoothed inspection image and target image to determine the displacement of the test object.

[0014] The imaging unit 20 includes an imaging system that captures an image representing an image of an object serving as a subject to be inspected, and generates image data indicating the captured image as a test object image. The object serving as the subject includes, for example, structures such as buildings, ground, or reduced models thereof. The reduced model may be loaded on a centrifugal loading device. Thereby, deformation of the model occurring in the centrifugal field, and as a result, strain distribution can be observed. The imaging system has an image sensor in which a plurality of pixels are arranged at regular intervals on a two-dimensional plane, and an optical system that guides incident light within the field of view onto an imaging region where the image sensor is arranged. The optical system is set such that its optical axis is directed in a direction (z direction) perpendicular to the two-dimensional plane of the image sensor. In the following description, the horizontal direction of the imaging region or the captured image may be referred to as the "x direction", and the vertical direction may be referred to as the "y direction". The image sensor may be capable of capturing high-definition images. The number of pixels arranged in each individual image sensor can typically exceed 10 million. The imaging unit 20 outputs the generated image data to the image processing device 10. The imaging unit 20 may be a digital still camera that captures a still image for each frame, or a video camera that captures a moving image. The imaging unit 20 does not have to be a dedicated camera as long as it is a device having an imaging function. The imaging unit 20 may be a device that does not have imaging or displacement measurement as its main function, such as a multifunctional mobile phone (smartphone) or a tablet terminal device, for example.

[0015] The imaging unit 20 may detect the distance from its own part to each observation point (corresponding to each pixel) on the surface of the subject, or the depth (z-direction coordinate based on the origin of the optical system) from its own part to each point. The imaging unit 20 may include a known distance meter in order to detect the distance or the depth (which may be collectively referred to as "distance or the like" in the following description), or may be configured to include a stereo camera having a plurality of imaging systems. The imaging unit 20 may include information such as the distance or the like to the observation point corresponding to each pixel in addition to the luminance for each pixel in the image data, and output it to the image processing device 10.

[0016] The display unit 30 displays a display screen shown in the display data input from the image processing apparatus 10. The display unit 30 is configured to include a display device such as, for example, a liquid crystal display or an organic electroluminescence display.

[0017] Next, a functional configuration example of the image processing apparatus 10 according to the present embodiment will be described. The image processing apparatus 10 includes a control unit 110, a storage unit 120, and an input / output unit 130. The control unit 110 performs processes for realizing various functions of the image processing apparatus 10 and controlling those functions. The control unit 110 includes a smoothing unit 112, a displacement calculation unit 114, a characteristic analysis unit 116, and an image processing unit 118.

[0018] The smoothing unit 112 acquires image data from the imaging unit 20 via the input / output unit 130. The smoothing unit 112 spatially smooths the luminance for each pixel shown in the image data representing the image of each frame, and calculates the smoothed luminance for each pixel. An image of one frame is represented by a luminance distribution composed of the luminance for each pixel arranged on a two-dimensional plane forming the image data. In the following description, the luminance smoothed for each pixel may be referred to as "smoothed luminance", and an image having the smoothed luminance for each pixel may be referred to as a "smoothed luminance image". The smoothing unit 112 outputs smoothed luminance image data indicating a smoothed luminance image to the displacement calculation unit 114 for each frame.

[0019] The smoothing unit 112 calculates the smoothed brightness of a reference pixel by spatially smoothing the brightness value of each target pixel to be processed within a predetermined window region that includes the reference pixel, for each reference pixel of interest as the target pixel to be processed (smoothing). The calculation related to smoothing may be a simple average or a weighted average. The weight coefficient for each target pixel used in the weighted average may be a positive real number less than 1. The weight coefficient may be determined based on a smoothing function that gives smaller absolute values ​​for target pixels that are farther from the reference pixel. A Gaussian distribution (two-dimensional normal distribution) may be used as the smoothing function. In general, an image of one frame is subdivided into pixel regions that are discretized for each pixel. The weight coefficient is calculated, for example, by normalizing the integral values ​​obtained by integrating the function value of the smoothing function for each pixel region so that the sum of the integral values ​​that span pixels within the window region is 1. The window area may be a square area that includes a reference pixel and at least one spatially adjacent pixel in the horizontal and vertical directions from the reference pixel as target pixels, or it may be an area that includes pixels located within a predetermined distance from the reference pixel as target pixels. Some or all of the setting items related to the smoothing process in the smoothing unit 112, such as the averaging method of the brightness values ​​(simple average or weighted average), the smoothing function (Gaussian distribution, uniform distribution, etc.), and the window size (number of pixels in the horizontal and vertical directions), may be predetermined or selected according to the operation signal input from the input unit 158 ​​(described later).

[0020] The displacement calculation unit 114 determines the relative displacement from the inspection area of ​​the reference image to the corresponding area of ​​the target image using the PIV method. The displacement calculation unit 114 determines a smoothed brightness image of one frame shown in the smoothed brightness image data input from the smoothing unit 112 as the reference image to be used as the reference for inspection. The displacement calculation unit 114 determines at least one frame of a smoothed brightness image separate from the reference image as the target image to be inspected. Typically, the reference image may be an image representing the target object before the occurrence of a predetermined event that may cause deformation of the object (e.g., application of external force, vibration, construction, natural disaster, deterioration over time, etc.). The target image may be an image representing the target object after the occurrence of that event. The displacement calculation unit 114 determines the reference image and the target image, for example, based on an operation signal input from the input unit 158 ​​(Figure 10) that accepts user operations.

[0021] The displacement calculation unit 114 searches for candidate regions corresponding to each inspection region (block) that includes individual pixels within the observation region of the reference image as reference pixels. The observation region may be the entire reference image or a portion of it. The observation region may be set in advance by the unit itself, or it may be set based on an operation signal input from the input unit 158 ​​(Figure 10).

[0022] The displacement calculation unit 114 searches for corresponding regions based on the similarity between the luminance distribution in the inspection region and the luminance distribution in individual candidate regions within the target image. The process of searching for corresponding regions is also called pattern matching. The inspection region is, for example, a square region in the reference image that includes a reference pixel as a measurement point at a predetermined position (e.g., the center) within that region, and contains multiple pixels as a whole. Candidate regions that are candidates for corresponding regions are regions that are the same size (number of pixels contained in that region) as the inspection region and form part of the inspection image. As a search region for searching for corresponding regions from the target image, a region centered on a reference pixel in the reference image and consisting of a portion of the target pixels may be set. The luminance distribution is represented by the luminance of each pixel contained in each region. The displacement calculation unit 114 can use, for example, SAD (Sum of Absolute Differences), SSD (Sum of Squared Differences), or dot product as a measure of similarity. For both SAD and SSD, a smaller value indicates a higher degree of similarity. The dot product indicates a higher degree of similarity as the value increases. Note that the displacement calculation unit 114 may use a recursive method in searching for corresponding regions. The recursive method is a technique that repeats the following steps S1 and S2: (Step S1) Search for corresponding regions within the exploration region that correspond to the inspection region; (Step S2) Define the searched corresponding region as the new exploration region and reduce the size of the inspection region. The repetition continues until the size of the inspection region reaches a predetermined minimum size, or until the number of executions of step S1 reaches a predetermined number. The ratio of the size of the inspection region to the initial corresponding region can be an integer of 2 or more. In contrast, a technique that simply searches for corresponding regions within the exploration region that correspond to the inspection region without step S2 is called the cross-correlation method. In the recursive method, since the exploration region and inspection region are gradually reduced, it is possible to avoid incorrect correspondence between the inspection region and corresponding regions by considering the global brightness distribution.

[0023] The displacement calculation unit 114 calculates the relative displacement amount from a reference pixel to the reference pixel of the corresponding region for each inspection region within the observation region. The reference pixel of the corresponding region corresponds to the pixel that has the same phase (for example, the center) as the reference pixel of the inspection region in the corresponding region. The displacement calculation unit 114 can calculate a relative direction based on the direction of the optical axis, using, for example, the two-dimensional coordinates of the respective image coordinate systems of the reference pixels in each inspection area represented in the image coordinate system and the reference pixels in the corresponding corresponding areas, based on the field of view, optical axis direction, and pixel count (resolution) of the optical system of the imaging unit 20, which are set in advance. The displacement calculation unit 114 can calculate the three-dimensional coordinates of each measurement point and inspection point that make up the surface of the object in the inspection system which forms the three-dimensional space to be observed, for each reference pixel, using the distance of the reference pixels to each inspection area and corresponding area included in the displacement data, and the calculated relative direction corresponding to each reference pixel. The displacement calculation unit 114 can calculate the displacement from the three-dimensional coordinate of the measurement point corresponding to the reference pixel in each inspection area in the observation area to the three-dimensional coordinate of the inspection point corresponding to the reference pixel in the corresponding area corresponding to that inspection area as the relative displacement amount. The displacement calculation unit 114 outputs data showing the relative displacement amount of the reference pixels for each corresponding area as displacement data to the characteristic analysis unit 116. In three-dimensional space, the relative displacement amount is represented by a three-dimensional vector. The set of relative displacement amounts for each reference pixel in the observation area, that is, the set of relative displacement amounts for each pixel shown in the displacement data, represents the displacement distribution on the surface of the object being inspected in the inspection system.

[0024] The characteristic analysis unit 116 calculates strain by spatially differentiating the relative displacement amount for each pixel shown in the displacement data input from the displacement calculation unit 114. The set of strains calculated for each pixel represents the strain distribution on the surface of the object in the inspection system which forms a three-dimensional space. The characteristic analysis unit 116 outputs the strain data showing the calculated strain distribution to the image processing unit 118. The characteristic analysis unit 116 may also output object position information indicating the corresponding three-dimensional coordinates for each reference pixel to the image processing unit 118 along with the strain data. The set of positions on the surface of the object shown by the object position information represents the three-dimensional shape of its surface.

[0025] The characteristic analysis unit 116 can calculate strain by partially differentiating the components of the relative displacement amount for each pixel in the observation region in each base direction in each base direction that makes up the space. That is, in a three-dimensional Cartesian coordinate system, the components in the mutually orthogonal x, y, and z directions are obtained by partially differentiating in the x, y, and z directions, respectively. In three-dimensional space, the strain corresponding to each pixel is represented by a 3x3 matrix (tensor). In the following explanation, the components of strain will be called "strain components," the direction of strain will be called "strain direction," the direction of partial differentiation will be called "partial differentiation direction," and the tensor that shows strain will be called the "strain tensor."

[0026] The characteristic analysis unit 116 determines an index value (scalar) indicating the magnitude of strain for each pixel, and identifies pixels or regions where the absolute value of the determined index value is greater than the absolute values ​​of the surrounding index values ​​as strain concentration points. For example, the characteristic analysis unit 116 can determine a maximum point, which is the location where the absolute value of the determined index value is spatially maximum, or a region within a predetermined range from the maximum point as a strain concentration point. The characteristic analysis unit 116 may also determine a threshold value that is a predetermined width smaller than the absolute value of the index value (maximum value) at the maximum point as a strain concentration reference value, and determine regions where the absolute value of the index value is greater than that strain concentration reference value as strain concentration points.

[0027] The characteristic analysis unit 116 may use any of the following (i)-(iv) as an index value indicating the magnitude of strain: (i) the determinant of the strain tensor, (ii) the element value with the largest absolute value among the nine element values ​​that make up the strain tensor (hereinafter sometimes referred to as the "maximum element value"), (iii) the maximum principal strain calculated from the strain tensor, and (iv) the maximum shear strain calculated from the strain tensor. The maximum principal strain refers to the maximum value of the principal strains, which are eigenvalues ​​of the strain tensor (the number of principal strains is 3 in three-dimensional space and 2 in two-dimensional space). The maximum shear strain refers to the difference between the maximum principal strain and the minimum principal strain. The minimum principal strain refers to the minimum value of the principal strain. The characteristic analysis unit 116 may output to the image processing unit 118 information indicating whether or not each pixel corresponds to a strain concentration area (hereinafter sometimes referred to as "strain concentration information") in the strain data. The set of information indicating whether or not each pixel corresponds to a strain concentration area represents the distribution of strain concentration areas.

[0028] The image processing unit 118 generates a strain distribution image representing the strain distribution based on the strain data input from the characteristic analysis unit 116. The strain distribution is represented by a set of values ​​indicating the magnitude of strain for each pixel (hereinafter sometimes referred to as "strain values"). As strain values, for example, element values ​​of a strain tensor, index values ​​indicating the magnitude of strain, etc., may be used. The type of element values ​​of the strain tensor can be specified in the strain direction and the partial derivative direction. As the type of index value indicating the magnitude of strain, any of the above (i)-(iv) may be specified. The type of strain value may be set in advance, or it may be selected according to the operation signal input from the input unit 158 ​​(Figure 10). The distribution of strain components is represented using a display mode (e.g., color tone, brightness, fill pattern, etc.) corresponding to the magnitude of the strain value.

[0029] If the strain data includes strain concentration information, the image processing unit 118 may refer to the strain concentration information and generate a strain distribution image showing the distribution of strain concentration locations. The distribution of strain concentration locations is represented using a display method (e.g., color tone, brightness, fill pattern, etc.) that corresponds to whether or not an area is a strain concentration location. The image processing unit 118 generates a display screen including the generated strain distribution image and generates display data showing the generated display screen. The image processing unit 118 outputs the generated display data to the display unit 30. Note that a single frame of the display screen may include strain distribution images based on multiple types of strain values, or it may include a strain distribution image showing the distribution of strain concentration points in addition to the strain distribution image based on strain values.

[0030] The storage unit 120 temporarily or permanently stores various data used in processing by the control unit 110, and various data acquired by the control unit 110 (which may include data at any stage before, during, or after processing). For example, the storage unit 120 stores image data input from the imaging unit 20. The input / output unit 130 is connected wirelessly or via wire to other devices to enable input and output of various types of data. The imaging unit 20 and the display unit 30 can be connected to the input / output unit 130. The input / output unit 130 is composed of, for example, an input / output interface, a communication interface, and the like.

[0031] Next, an example of displacement analysis using the PIV method will be described. Figure 2 is a flowchart showing an example of displacement analysis according to this embodiment. However, the following example shows the process up to determining the relative displacement for one inspection area between the reference image and the target image. (Step S102) The displacement calculation unit 114 selects one of the smoothed luminance images, in which the luminance values ​​have been smoothed, as a reference image, defines a portion of the selected reference image as the target area, and identifies the luminance of each pixel within the target area as a luminance value distribution. (Step S104) The displacement calculation unit 114 selects a smoothed luminance image separate from the reference image as the target image, defines multiple candidate regions from a portion of the selected target image, and identifies the luminance value distribution for each pixel within each candidate region. The size of each candidate region is set in advance to be equal to the size of the target region. The position of the reference pixel in each candidate region may be limited to pixels within a predetermined distance from the reference pixel of the target region.

[0032] (Step S106) The displacement calculation unit 114 compares the brightness value distribution within the inspection area of ​​the reference image with the brightness value distribution of the candidate areas of the target area for each candidate area. Here, the displacement calculation unit 114 calculates an index value of the similarity between the two. (Step S108) Based on the calculated similarity index value, the displacement calculation unit 114 identifies candidate regions for the target region that give the brightness value distribution most similar to the brightness value distribution of the inspection region of the reference image as corresponding regions, and sets the position of the reference pixel to be the representative position of the identified corresponding region as, for example, the center point of the corresponding region. (Step S110) The displacement calculation unit 114 calculates the relative displacement as the position of the reference pixel in the inspection area, for example, the displacement from the center point (measurement point) of the inspection area to the center point of the corresponding area. The displacement calculation unit 114 changes the inspection area to be processed to any of the unprocessed areas in the predetermined observation area and repeats the process shown in Figure 2.

[0033] Next, an example of the spatial distribution of strain will be described. The various images exemplified below represent a scaled-down model subjected to centrifugal loading as the test specimen. The scaled-down model simulates the situation in which a pile is embedded in the ground. Another feature of this embodiment is the use of a centrifugal field in image processing such as the PIV method. By using a centrifugal field, clear images can be obtained. It is known that the centrifugal field in model experiments has a clear mechanical correspondence with the real-world situation being simulated. A Gaussian distribution was used as the smoothing function. A recursive correlation method was used in the search for the corresponding region. The size of the exploration region in the target image (image after deformation), the minimum size of the inspection region in the reference image (image before deformation), and the minimum size of the candidate region were set to 8×8 pixels, 64×64 pixels, and 8×8 pixels, respectively, and the initial size of the inspection region was set to 32×32 pixels. Figure 3(a) shows the spatial distribution of displacement in a one-dimensional observation region obtained from a reference image and an inspection image using the conventional PIV method. Figure 3(b) shows the strain distribution obtained by spatially differentiating the displacement exemplified in Figure 3(a). The displacement exemplified in Figure 3(a) tends to change smoothly across the entire observation area. As exemplified in Figure 3(b), strain components with low dependence on position (hereinafter sometimes referred to as "overall components") are prominent across the entire observation area, and the contribution of local strain components with high dependence on position (hereinafter sometimes referred to as "local components") is relatively small. Therefore, a significant increase in strain at an observation point (arrow) where the occurrence of more pronounced strain than in its surroundings is expected tends to be overlooked.

[0034] Figure 4(a) shows another example of the spatial distribution of displacement. Figure 4(b) illustrates the spatial distribution of strain exemplified in Figure 4(a). In this example, the strain distribution mainly contains local components, and no significant overall components are present. Therefore, a significant increase in strain is observed at the observation point. In particular, a rapid spatial change in displacement is observed at and around the observation point, indicating an ideal spatial distribution of strain for detecting areas of strain concentration. From this, it can be expected that areas of strain concentration can be detected more easily if the overall component can be removed or reduced.

[0035] Figure 5(a) shows the spatial distribution of displacement obtained from a reference image and an inspection image, respectively, after spatially smoothing the brightness, according to this embodiment. Figure 5(b) shows the strain distribution obtained by spatially differentiating the displacement exemplified in Figure 5(a). In the example shown in Figure 5(a), although not as pronounced as in Figure 4(a), a more abrupt spatial change in displacement at the observation point is observed than in Figure 3(a). In the example shown in Figure 5(b), although not as pronounced as in Figure 4(b), the overall component is reduced and the local component is significantly emphasized compared to Figure 3(b). The reduction in the overall component is due to the cancellation of common components between adjacent pixels by smoothing the brightness between adjacent pixels. Therefore, visualization becomes possible using a strain distribution image in which the local component is relatively emphasized.

[0036] Figures 6(a) and 6(b) show examples of a reference image and a target image, respectively, before smoothing (unprocessed). The reference image is an image of the object taken before deformation. The target image is an image of the object taken after deformation. Figure 6(c) shows a magnified image of a portion of the target image, specifically region P. In the example shown in Figure 7(c), a rapid spatial change in the brightness of the pattern representing the object is observed. Figures 7(a) and 7(b) show examples of a reference image and a target image obtained by smoothing with an adjacent pixel (referring to the luminance values ​​of a 3x3 pixel area centered on the pixel of interest as a window region) in this embodiment (adjacent pixel). Figure 7(c) shows an enlarged image of region P in Figure 7(b). In the example shown in Figure 7(c), the spatial variation of luminance is more mitigated than in the example shown in Figure 6(c). Figures 8(a) and 8(b) show examples of a reference image and a target image obtained by smoothing between two adjacent pixels (referring to the luminance values ​​of a 3x3 pixel area centered on the pixel of interest as a window region) in this embodiment (two adjacent pixels). Figure 8(c) shows an enlarged image of region P in Figure 8(b). In the example shown in Figure 8(c), the spatial variation in luminance is more mitigated than in the example shown in Figure 7(c). Processing was also attempted for the cases of three adjacent pixels and four adjacent pixels, but the clearest image was obtained in the case of two adjacent pixels.

[0037] Figures 9(a), (b), and (c) illustrate the strain distribution (maximum shear strain) obtained for the unprocessed image, one adjacent pixel, and two adjacent pixels, respectively. The intensity of the strain distribution represents the magnitude of the strain. Brighter areas indicate greater strain, while darker areas indicate less strain. However, the blacked-out area in the center indicates the region where the object is located. Figures 9(a)-(c) all show that the strain is significant in the area in contact with the object or in the area adjacent to it. However, of Figures 9(a)-(c), Figure 9(c) most clearly shows the strain concentration, followed by Figure 9(b). This indicates that the strain concentration is clarified by smoothing the brightness.

[0038] In the above explanation, the case where the displacement calculation unit 114 calculates the displacement of an object in three-dimensional space and the characteristic analysis unit 116 analyzes the strain of an object in three-dimensional space was mainly described, but it is not limited to this. The displacement calculation unit 114 may also calculate the displacement of an object that appears in a two-dimensional plane and perform an analysis of the strain of an object that appears in a two-dimensional plane. In this case, the displacement calculation unit 114 can calculate the displacement from the reference pixel of each inspection area represented in the image coordinate system to the reference pixel of the corresponding target area as a relative displacement amount in a two-dimensional plane. The characteristic analysis unit 116 calculates the strain by differentiating this relative displacement amount in the two-dimensional plane. Therefore, the strain related to each target area is represented by a 2x2 strain tensor. The characteristic analysis unit 116 can calculate the above index value from the obtained strain tensor. Furthermore, the imaging unit 20 only needs to be provided with image data showing a two-dimensional image. In other words, the imaging unit 20 only needs to be equipped with one imaging system and does not require a rangefinder or multiple imaging systems.

[0039] (Hardware configuration) Next, an example of the hardware configuration of the image processing device 10 according to this embodiment will be described. The image processing device 10 may be configured to include dedicated components (for example, integrated circuits) that form sets of one or more functional parts as shown in Figure 2, but it may also be configured as a computer in part or in whole.

[0040] Figure 10 is a schematic block diagram showing an example of the hardware configuration of the image processing device 10 according to this embodiment. The image processing device 10 is composed of a processor 152, a drive unit 156, an input unit 158, an output unit 160, a ROM (Read Only Memory) 162, a RAM (Random Access Memory) 164, an auxiliary storage unit 166, and an interface unit 168. The processor 152, drive unit 156, input unit 158, output unit 160, ROM 162, RAM 164, auxiliary storage unit 166, and interface unit 168 are interconnected using a bus BS (baseline).

[0041] The processor 152, for example, reads programs and various data stored in the ROM 162, executes the programs, and controls the operation of the image processing device 10. The processor 152 may include, for example, one or more CPUs (Central Processing Units). The processor 152 may also include one or more GPUs (Graphic Processing Units) or coprocessors.

[0042] The processor 152 may execute a predetermined program to implement all or some of the functions of the above-mentioned functional units, for example, the smoothing unit 112, displacement calculation unit 114, characteristic analysis unit 116 of the control unit 110, and the functions of the image processing unit 118. The processor 152 may also work in cooperation with any or any combination of the ROM 162, RAM 164, and auxiliary storage unit 166 to implement the functions of the storage unit 120. The processor 152 may also work in cooperation with any or any combination of the input unit 158, output unit 160, and interface unit 168 to implement the functions of the input / output unit 130. Note that "executing a program" means executing the processing instructed by various commands written in the program.

[0043] The storage medium 154 stores various types of data. The storage medium 154 is a portable storage medium such as a magneto-optical disk, a flexible disk, or flash memory. The drive unit 156 is a device that performs, for example, either reading various data from the storage medium 154 or writing various data to the storage medium 154, or both.

[0044] The input unit 158 ​​is an input device that receives user input, generates an operation signal according to the received input, and outputs the generated operation signal to the processor 152. The input unit 158 ​​can be, for example, a pointing device such as a mouse, keyboard, or touch sensor. The output unit 160 is composed of, for example, a display unit such as a screen and a playback unit such as a speaker. The display unit 30 described above may correspond to the output unit 160.

[0045] ROM 162 stores, for example, programs for execution by processor 152. RAM 164 functions as a temporary workspace for storing various data and programs used by processor 152. The auxiliary storage unit 166 is a storage medium such as an HDD (Hard Disk Drive) or flash memory.

[0046] The interface unit 168 connects to other devices and allows for the input and output of various types of data. The interface unit 168 includes, for example, a communication module that connects to a network via wired or wireless connection.

[0047] The image processing device 10 may be configured as a single device that integrates either or both of the imaging unit 20 and the display unit 30, or they may not be integrated. The display unit 30 may be integrated with a touch sensor which is the input unit 158 ​​and configured as a touch panel. The displacement calculation unit 114 may define multiple target images for a single reference image. Each of the multiple frames is used to calculate the displacement amount relative to the reference image. The acquisition times of the reference image and target image may be set in advance in the displacement calculation unit 114, and the smoothed luminance images acquired at the set acquisition times may be defined as the reference image and target image, respectively. This allows for the analysis of the time change in the displacement amount. When smoothed luminance images based on still images of individual frames constituting the video are input at regular time intervals, the displacement calculation unit 114 may define the smoothed luminance image of the latest frame as the target image and the smoothed luminance image of the previous frame as the reference image. This allows for the dynamic change in the displacement amount, and consequently the strain, for each frame. Furthermore, the image to be smoothed is not limited to monochrome images showing the distribution of brightness, but may also be a color image showing color signal values ​​for each pixel. The color signal values ​​can be considered as values ​​representing the brightness of each of several primary colors. The image processing device 10 and the imaging unit 20 may be installed on the vibration table of a centrifugal loading device. This is convenient for capturing clear images of a scaled-down model in a centrifugal field as a test specimen.

[0048] As described above, the image processing apparatus 10 according to the above embodiment comprises a smoothing unit 112, a displacement calculation unit 114, and a characteristic analysis unit 116. The smoothing unit 112 spatially smooths the brightness of an image of an object represented using a plurality of pixels. The displacement calculation unit 114 searches for candidate regions of the target image to be inspected, which have had their brightness smoothed, based on the similarity to the brightness distribution of each inspection region of the reference image used as the basis for inspection, and determines the displacement up to the searched candidate region for each inspection region. The characteristic analysis unit 116 calculates the strain of the object based on the determined displacement. In this configuration, candidate regions corresponding to the inspection region are searched based on the smoothed brightness between adjacent pixels, and strain is calculated based on the displacement from the inspection region to the candidate region. Because components common to the entire region being analyzed are suppressed by smoothing, local components are relatively emphasized. Therefore, areas with significant strain can be easily identified.

[0049] The image processing device 10 may include an image processing unit 118 that generates a strain distribution image showing the strain distribution of an object. This configuration yields a strain distribution image showing the strain distribution of the captured object. The strain distribution image can then be visualized by displaying it.

[0050] The characteristic analysis unit 116 may identify strain concentration points where the strain is greater than the surrounding area. With this configuration, strain concentration points with significant strain can be quantitatively identified.

[0051] The image processing unit 118 may represent areas of strain concentration in the strain distribution image in a manner different from other regions. With this configuration, users can easily identify areas of strain concentration by visualizing the strain distribution image.

[0052] Although this embodiment has been described in detail above with reference to the drawings, the specific configuration is not limited to the configurations described above, and includes designs and the like that do not depart from the gist of this embodiment. The configurations described above can be combined arbitrarily, and some of them may be omitted. [Explanation of symbols]

[0053] 1…Image processing system, 10…Image processing device, 20…Shooting unit, 30…Display unit, 110…Control unit, 112…Smoothing unit, 114…Displacement calculation unit, 116…Characteristic analysis unit, 118…Image processing unit, 120…Storage unit, 130…Input / output unit, 152…Processor, 156…Drive unit, 158…Input unit, 160…Output unit, 162…ROM, 164…RAM, 166…Auxiliary storage unit, 168…Interface unit

Claims

1. A smoothing unit that spatially smooths the brightness of an image of an object represented using multiple pixels, For each inspection area of ​​the reference image, which is used as the basis for inspection, candidate areas of the target image, which is also smoothed in brightness, are searched based on the similarity to the brightness distribution of that inspection area. A displacement calculation unit that determines the displacement up to the explored candidate region for each inspection region, The system includes a characteristic analysis unit that calculates the strain of the object based on the displacement. Image processing device.

2. The system includes an image processing unit that generates a strain distribution image showing the strain distribution of the object. The image processing apparatus according to claim 1.

3. The characteristic analysis unit, Identify areas of strain concentration where the strain is greater than the surrounding area. The image processing apparatus according to claim 2.

4. The aforementioned image processing unit, The strain distribution image is generated in which the strain concentration areas are represented in a manner different from other regions. The image processing apparatus according to claim 3.

5. A program for causing a computer to function as an image processing device according to any one of claims 1 to 4.

6. An image processing apparatus according to any one of claims 2 to 4, The system includes a display unit that displays the strain distribution image. Image processing system.

7. It includes a camera unit for capturing images of the aforementioned object. The image processing system according to claim 6.

8. An image processing method in an image processing device, A smoothing step that spatially smooths the brightness of an image of an object represented using multiple pixels, For each inspection area of ​​the reference image, which is used as the basis for inspection, candidate areas of the target image, which is also smoothed in brightness, are searched based on the similarity to the brightness distribution of that inspection area. The method comprises: a displacement calculation step that determines the displacement up to the explored candidate region for each inspection region; and a characteristic analysis step that calculates the strain of the object based on the displacement. Image processing methods.

Images