Image analysis method and image analysis device

Abstract

To determine motions of an imaging target.SOLUTION: A controller generates a first image including a first speckle pattern on the basis of a first imaging signal, generates a second image including a second speckle pattern on the basis of a second imaging signal generated in such a manner that an imaging device takes an image of the imaging target in a second time, approximates the displacement of a first feature vector in a first pixel aggregate in the first image, approximates the displacement of a second feature vector in a second pixel aggregate in a second image, and calculates the distance between pixels on a pixel-by-pixel basis between the first feature vector and the second feature vector, thereby determining the change between the first speckle pattern and the second speckle pattern.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to an image analysis method and an image analysis device, and more particularly to an image analysis method and an image analysis device that determine slight movements of an object based on speckles resulting from interference of light reflected from the object. [Background technology]

[0002] When the surface of an object is irradiated with light such as laser light, the light reflected by the unevenness of the irradiated surface of the object scatters and interferes, resulting in the appearance of a spotted pattern on the irradiated surface. This pattern is called speckle. When observing an image generated from an object irradiated with coherent light, speckles are observed in the image. When the object moves, the speckle pattern also changes in response to the movement. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2019-519269 Summary of the Invention [Problem to be solved by the invention]

[0004] Techniques have been proposed for determining the motion of an object illuminated by light by analyzing speckle patterns in an image. Patent Document 1 discloses a technique for monitoring mechanical and / or acoustic signals, such as mechanical and / or acoustic vibrations associated with a biological specimen, by collecting image data pieces associated with secondary speckle patterns.

[0005] The technique disclosed in Patent Document 1 determines a correlation function that indicates the spatial correlation between speckle patterns, and applies filtering to the determined correlation function to select changes in correlation that occur with frequency.

[0006] For example, when the distance between the camera and the object is short, the speckle pattern may change suddenly even if the object moves slightly. In such a case, when the speckle pattern is continuously observed, the patterns before and after the change become uncorrelated, and the continuous slight movement of the object cannot be determined. Patent Document 1 does not propose any solution for determining the above-mentioned slight movement of the object. [Means for solving the problem]

[0007] A computing device according to one embodiment is a computing device for determining motion of an object, the computing device including a control device configured to: generate a first image including a first speckle pattern based on a first imaging signal generated by an imaging device imaging the object at a first time; generate a second image including a second speckle pattern based on a second imaging signal generated by the imaging device imaging the object at a second time; calculate a first feature vector from luminance at a first set of pixels in the first image; approximate a displacement of the first feature vector between the first set of pixels; calculate a second feature vector from luminance at a second set of pixels in the second image; approximate a displacement of the second feature vector between the second set of pixels; and determine a change between the first speckle pattern and the second speckle pattern by calculating a distance between pixels for each pixel between the approximated first feature vector and the approximated second feature vector.

[0008] In another embodiment, a method is executed by a computing device to determine motion of an object, the method including: generating a first image including a first speckle pattern based on a first imaging signal generated by an imaging device imaging the object at a first time; generating a second image including a second speckle pattern based on a second imaging signal generated by the imaging device imaging the object at a second time; calculating a first feature vector from luminance at a first set of pixels in the first image and approximating a displacement of the first feature vector between the first set of pixels; calculating a second feature vector from luminance at a second set of pixels in the second image and approximating a displacement of the second feature vector between the second set of pixels; and determining a change between the first speckle pattern and the second speckle pattern by calculating a distance between pixels for each pixel between the approximated first feature vector and the approximated second feature vector. Effect of the Invention

[0009] According to the image analysis system of the embodiment, it is possible to maintain correlation before and after a change in the speckle pattern, and it is possible to accurately determine even slight movements of an object. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an image analysis system. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a computer device. [Diagram 3] FIG. 1 is a diagram showing an example of photographing a subject; [Figure 4] FIG. 13 is a diagram illustrating an example of a local plane. [Diagram 5] 11 is a flowchart illustrating an example of a process executed by the image analysis system. [Figure 6] FIG. 13 is a diagram illustrating an example of a local plane. [Figure 7]FIG. 13 is a diagram showing an example of a state in which the cloth is pressed with a finger. [Figure 8] 11 is a graph showing an example of a change in vibration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An image analysis method and an image analysis device according to an embodiment will be described in detail below with reference to the attached drawings. In this embodiment, the image analysis method and the image analysis device are implemented in an image analysis system. In this embodiment, any object that is irradiated with light and whose movement, etc. is determined is referred to as a "photographed object."

[0012] First, an example of the configuration of an image analysis system 100 will be described with reference to Fig. 1. In this embodiment, the image analysis system 100 includes a computer device 1, image capture devices 2a, 2b, and 2c (collectively, image capture devices 2), light projecting devices 3a, 3b, and 3c (collectively, light projecting devices 3), and a display device 4. The computer device 1 is mutually coupled with at least the image capture device 2 and the display device 4 via a bus or a network, and is optionally coupled with the light projecting device 3.

[0013] The computing device 1 is any information processing device including at least a calculation function. The computing device 1 generates an image based on an imaging signal received from the imaging device 2, and determines the movement of the imaging target O based on a speckle pattern in the image.

[0014] The imaging device 2 includes one or more CCD image sensors or CMOS image sensors, captures an image of the object O, and transmits a capture signal to the computer device 1. In this embodiment, the imaging device 2 is implemented by a camera. It is desirable that the imaging device 2 has more image sensors arranged in order to generate an image with high pixel count.

[0015] In this embodiment, in order to determine the continuous movement of the subject O, a plurality of imaging devices 2 (imaging devices 2a, 2b, and 2c) are provided to capture images of the subject O at a plurality of different positions, but the number of imaging devices 2 may be any number equal to or greater than one. For example, when one imaging device 2 is provided, the imaging device 2 may be a rotating and / or movable light source capable of capturing images of the subject O in any direction.

[0016] The light projecting device 3 is a light source that irradiates coherent light toward the subject O. Coherent light refers to light in which the phase relationship between light waves at any two points in a light beam is constant over time and exhibits perfect coherence even when the light beam is divided by any method and then superimposed again with a large optical path difference. In this embodiment, the light projecting device 3 emits laser light as the coherent light.

[0017] As with the imaging device 2, a plurality of light projecting devices 3 (light projecting devices 3a, 3b, and 3c) are provided to irradiate laser light onto the object O at a plurality of different positions in order to determine the continuous movement of the object O, but the number of light projecting devices 3 may be any number equal to or greater than one. For example, when one light projecting device 3 is provided, the light projecting device 3 may be a rotating and / or movable light source capable of emitting laser light in all directions.

[0018] The display device 4 is a display that displays the movement of an object as a result of the movement of the object determined by the computing device 1. The display device 4 is not an essential component of this embodiment, but when this embodiment is applied to, for example, observing sound vibrations transmitted to an object, the sound vibrations can be visually represented.

[0019] In this embodiment, the computer device 1, the imaging device 2, the light projecting device 3, and the display device 4 are implemented by independent devices, but are not limited to such a configuration. A single device in which all of them are integrated may be provided, or a device in which any of them are combined may exist.

[0020] Next, detailed components of the computer device 1 will be described with reference to Fig. 2. The computer device 1 includes a controller 11, a memory 12, a storage device 13, an input driver 14, and an output driver 15. The memory 12, the storage device 13, the input driver 14, and the output driver 15 are each coupled to the controller 11 through an internal bus and controlled by the controller 11.

[0021] The control device 11 is also called a processor, and includes a central processing unit (CPU) and a graphic processing unit (GPU). The control device 11 generates an image based on an imaging signal received from the imaging device 2, and performs calculations such as determining the movement of the object O based on a speckle pattern in the image. The control device 11 may control the imaging device 2 and the light projector 3 coupled to the computer device 1. In this case, for example, the control device 11 may control the timing at which the light projector 3 irradiates the object O with laser light, and the timing at which the imaging device 2 photographs the object O (by transmitting control signals to the imaging device 2 and the light projector 3).

[0022] The memory 12 is a volatile data storage device that stores computer-executable instructions processed by the control device 11, data after arithmetic processing according to the instructions, etc. The memory 12 may be implemented with a RAM (random access memory) (e.g., an SRAM (static RAM) and a DRAM (dynamic RAM)), etc.

[0023] The storage device 13 is a non-volatile data storage device that stores a program including the above-mentioned computer-executable instructions. The storage device 13 may be implemented by a non-volatile semiconductor memory such as a read-only memory (ROM), a magnetic storage device (such as a hard disk drive), an optical disk, or the like. Note that data such as the program may be stored in a network attached storage (NAS) and / or a storage area network (SAN) in addition to or instead of the storage device 13.

[0024] The input driver 14 receives input from an external device coupled to the computer device 1, such as receiving an image capture signal from the imaging device 2, and transmits the input to the control device 11. The output driver 15 receives output from the control device to an external device coupled to the computer device 1, such as transmitting an image signal to the display device 4, and transmits the output to the external device.

[0025] Next, a speckle pattern in an image generated by the imaging device 2 capturing an image of the object O will be described with reference to Fig. 3. As described above, when the object O moves, the laser light irradiated by the light projecting device 3 scatters and interferes in response to the movement, causing a change in the speckle pattern.

[0026] 3 shows three images generated by photographing an object O moving in the direction indicated by a dashed arrow A at three photographing points. The imaging device 2a photographs the object O at photographing point S1, the imaging device 2b photographs the object O at photographing point S2, and the imaging device 2c photographs the object O at photographing point S3. In this embodiment, the object O is photographed at each of the three different positions, that is, photographing points S1 to S3, but the object O may also be photographed at the same position. In other words, the object O is photographed multiple times at different positions or the same position at different points in time.

[0027] An image generated by the imaging device 2a photographing the subject O (by the control device 11) corresponds to image I1. Similarly, an image generated by the imaging device 2b photographing the subject O corresponds to image I2, and an image generated by the imaging device 2c photographing the subject O corresponds to image I3.

[0028] As described above, when determining the continuous movement of the object O to be photographed by analyzing the speckle pattern, the object O is photographed at a plurality of photographing points. At each photographing point, the laser light irradiated from the light projector 3 is scattered and interfered with, causing a change in the speckle pattern. As shown in FIG. 3, the speckle patterns in the three images I1 to I3 are different from each other.

[0029] In this embodiment, a specific region in an image containing a speckle pattern is considered as a plane, and 3D plane parameters around a specific pixel (x, y) in the plane are estimated. In this embodiment, this specific planar region is called a "local plane." With reference to FIG. 4, the local planes in three images generated by photographing a moving target O (three times) are described.

[0030] Fig. 4 shows three local planes LP1 to LP3. The local plane LP1 corresponds to a particular local plane in the image I1 shown in Fig. 3. The local plane LP2 corresponds to a local plane corresponding to the local plane LP1 in the image I2 shown in Fig. 3. The local plane LP3 corresponds to a local plane corresponding to the local plane LP1 and the local plane LP2 in the image I3 shown in Fig. 3.

[0031] 3, since the images I1 to I3 are images generated by capturing an image of a moving target O, the local plane LP2 is displaced in its coordinate position in the image relative to the local plane LP1. Similarly, the local plane LP3 is displaced in its coordinate position in the image relative to the local planes LP1 and LP2.

[0032] Fig. 4 shows the displacement of local planes LP2 and LP3 with respect to local plane LP1. The displacement is represented by the distance between pixels in the two local planes. In the example shown in Fig. 4, a specific pixel P1a in local plane LP1 and an adjacent pixel P1b around pixel P1a are used as the reference.

[0033] First, consider a local plane as frame i, and estimate the 3D plane parameters around pixel P1a(x,y) in frame i according to equation (1).

[0034]

number

[0035]

number

[0036]

number

[0037] As shown in FIG. 4, the 3D plane parameters around pixel P2a in local plane LP2 are i (x,y) and the 3D plane parameters around pixel P3b in local plane LP3 are

[0038]

number

[0039] It is expressed as:

[0040] 3 represent the movement of the object O, the offset between pixel P1a in the local plane LP1 and pixel P2a in the local plane LP2 is represented as a displacement d1(d(x,y)). The displacement d1 corresponds to the movement of the object O between the photographing point S1 and the photographing point S2 shown in FIG.

[0041] The displacement d1 is calculated according to equation (2).

[0042]

number

[0043] Similarly, the offset between pixel P1b in local plane LP1 and pixel P2b in local plane LP3 is a displacement d2(

[0044]

number

[0045] ) The displacement d2 corresponds to the movement of the object O between the photographing point S1 and the photographing point S3 shown in FIG.

[0046] By calculating the above-mentioned displacements d1 and d2, the change in the speckle pattern in the image can be analyzed. The speckle pattern is sensitive to the relative position between the imaging device 2 and the object O. Therefore, the movement of the object O can be determined by analyzing the change in the speckle pattern in each image generated by capturing the object O multiple times.

[0047] When the speckle pattern changes suddenly, the speckle patterns before and after the change become uncorrelated. This means that the continuous movement of the object O cannot be determined, and even slight changes cannot be accurately determined. The image analysis system 100 according to the present embodiment determines the movement of the object O based on the speckle pattern while maintaining the consistency of the speckle pattern, regardless of the change in the speckle pattern.

[0048] The process executed by the image analysis system 100 will be described with reference to the flowchart of Fig. 5. The process described in Fig. 5 analyzes changes in speckle patterns in images I1 and I2 generated by photographing a moving target O at photographing points S1 and S2.

[0049] First, the light projecting device 3a and the light projecting device 3b each irradiate the object O with a laser beam (step S501). Next, the imaging device 2a captures the object O at the imaging point S1, transmits an imaging signal to the computer device 1, and the control device 11 of the computer device 1 generates an image I1 based on the imaging signal (step S502). The imaging device 2b captures the object O at the imaging point S2, transmits an imaging signal to the computer device 1, and the control device 11 generates an image I2 based on the imaging signal (step S503).

[0050] Next, the control device 11 extracts a local plane LP1 from the image I1 (step S504). The local plane LP1 is extracted by identifying a specific region using a technique such as the Hough transform. In the following, the local plane LP1 is regarded as the frame i. In this embodiment, the local plane LP1 corresponds to a rectangular region, but the shape of the region may have any shape.

[0051] Furthermore, the control device 11 extracts a local plane LP2 corresponding to the local plane LP1 (frame i) based on the local plane LP1 (step S505). The local plane LP2 is extracted by identifying an area corresponding to the local plane LP1, for example, using a projective transformation using a projection matrix. In the following, the local plane LP1 is regarded as frame j. The processing of step S505 also identifies pixels P2a and P2b in frame j that correspond to pixels P1a and P1b in frame i.

[0052] Next, the control device 11 calculates the distance between pixels for each pixel in each frame to obtain the displacement between frame i and frame j (step S506). Here, the luminance of a specific pixel P1a(x,y) in frame i is expressed as I i (x,y) and the pixel set (x k ,y k ) is the luminance set for I i (x k ,y k ) where k represents the number of dimensions of pixels in a frame. In this embodiment, k=112 shall be.

[0053] Also, the luminance of the pixel P2a(x, y) corresponding to the pixel P1a in the frame j is I j (x, y), and the luminance set for the pixel set (x k , y k ) is I j (x k , y k ). The distance between pixels in step S506 is calculated by the Euclidean distance between the above-described feature vectors.

[0054]

Number

[0055] The luminance set I i (x k , y k ) forms the feature vector F i = I i (x 1 , y 1 ), …, I i (x k , y k ). Also, the luminance set I j (x k , y k ) forms the feature vector F j = I j (x 1 , y 1 ), …, I j (x k , y k ).

[0056] Next, the control device 11 calculates the low-dimensional vector Ψ i for the frame i by a dimensionality reduction technique that reduces the dimensionality k to the dimensionality l (l < k) based on the 3D plane parameter ψ i (x, y) of the frame i (step S507).

[0057] The above-mentioned dimensionality reduction techniques include, for example, a diffusion map and an isomap. In this embodiment, the diffusion map is used to generate a low-dimensional vector Ψ i is calculated.

[0058]

number

[0059] The input for dimensionality reduction is a set of feature vectors F i (x,y) (i=1,...,N), where N is the number of pixels in frame i.

[0060] The control device 11 also calculates the 3D plane parameter ψ of the frame j. j Based on (x,y), a low-dimensional vector Ψ for frame j is obtained by a dimensionality reduction technique (diffusion map) that reduces the dimensionality k to dimensionality l. j Calculate (step S508).

[0061] The dimensionality reduction techniques mentioned above also include diffusion maps and Isomaps, etc. In this embodiment, the diffusion map is used to generate a low-dimensional vector Ψ, which is a feature vector for frame i in reduced dimensionality l, according to Equation (5). j Here, it is preferable that l, which is the number of dimensions reduced in steps S507 and S508, is any value between 3 and 20, for example.

[0062]

number

[0063] The input for dimensionality reduction is a set of feature vectors F j (x,y)(j=1,...,N'), where N' is the number of pixels in frame j.

[0064] With the reduced dimensionality l, the distance is calculated according to equation (6) instead of equation (3).

[0065]

number

[0066] The processing in steps S507 and S508 involves calculating the displacement between the feature vector for frame i and the feature vector for frame j in a reduced dimension number l, based on the displacement (Euclidean distance) between the feature vector for frame i and the feature vector for frame j calculated in the dimension number k.

[0067] By reducing the number of dimensions in steps S507 and S508, the distance is calculated in the reduced number of dimensions l, as described below. For example, if the object motion does not include motion in the depth direction relative to the image capture device 2, but only motion parallel to the image capture device 2, calculating the displacement in a high number of dimensions will generate noise in the calculation. By calculating the distance (according to equation (6)) in the reduced number of dimensions l, this noise can be removed.

[0068] Furthermore, by calculating the displacement in the reduced dimension l, the displacement of the feature vector between each pixel in frame i and frame j becomes small, so that the linearity of the speckle pattern can be maintained.

[0069] Furthermore, the amount of calculation required to calculate the distance can be reduced by calculating the distance in the reduced number of dimensions l. In particular, in equation (3), the distance is calculated for each pixel, so calculating the distance without reducing the number of dimensions requires an enormous amount of calculation.

[0070] Next, the controller 11 calculates the reduced dimensional vector Ψ for frame i according to equation (7). i(step S509). The control device 11 also applies a linear transformation to the low-dimensional vector Ψ for frame i according to equation (8). i A linear transformation is applied to (step S510).

[0071] The process in step S509 is to define the luminance set in frame i as I i (x l ,y l ) (l represents the number of dimensions to which the dimensionality reduction was applied in step S507). Similarly, the process of step S510 approximates the displacement of the feature vector between each pixel of I j (x l ,y l ) to approximate the displacement of the feature vector between each pixel of

[0072]

number

[0073]

number

[0074] In equations (7) and (8), M(x, y) is a 3×l transformation matrix, and Ψ i (x,y) is the l×1 column vector for pixel P1a(x,y).

[0075] In equations (7) and (8), we use constraints E according to equations (9) and (10) to approximate the displacement of feature vectors between a particular pixel and pixels within a particular range of that pixel in a frame. S This constraint is imposed, for example, on pixel P2a (x 1 ,y 1 ) and pixel P1a in the local plane LP1 (frame i) (distance (displacement) between pixels d1) and pixel P2b (x 2 ,y 2) and pixel P1b (distance (displacement) d2 between pixels). In this example, the distance d1 between pixels and the distance d2 between pixels correspond to the change (displacement) in distance to an actual point corresponding to the same pixel as seen from the imaging device 2, but may be another definition expressing the distance between pixels.

[0076]

number

[0077]

number

[0078] Pixel P2a(x 1 ,y 1 ) and pixel P2b(x 2 ,y 2 ) are constrained according to equation (11).

[0079]

number

[0080]

number

[0081] is the element-wise product.

[0082] In equation (10), let m be the column vector obtained by applying the linear matrix M(x,y) to the vector of all pixels. Since the constraint is expressed by Am = 0, and A is the coefficient matrix calculated from equation (11), m is A T The eigenvector associated with the smallest eigenvalue of A is given as: Once the linear matrix M(x,y) is calculated, a linear transformation is applied to the low-dimensional vector ψ, calculated according to (7) and (8). i (x,y) and a low-dimensional vector ψ j(x,y) are spatially consistent (i.e., the displacement of the feature vectors of pixels within a frame, represented by the low-dimensional vectors, is within a certain range).

[0083] One problem in computing eigenvectors is that, for example, we have a low-dimensional vector Ψ for a pixel in frame i. i The difference is that the temporal distribution of (x,y) is different. i If (x,y) is almost zero, then Ψ i Since (x,y) does not affect the error, the magnitude of M(x,y) is approximately equal to 1, and the parameters for other pixels are approximately zero. Therefore, before calculating the coefficient matrix A along the time axis according to equation (12), Ψ i Normalize the standard deviation of (x,y).

[0084]

number

[0085] Psi ik is Ψ i is the kth component of

[0086]

number

[0087] is Ψ ik , and ε is a small number to avoid division by zero.

[0088] For example, since the load of calculating eigenvectors from an image captured and generated by an imaging device 2 having high resolution is high, the transformation matrix M(x, y) is calculated after subsampling the pixels.

[0089] Returning to the description of FIG. 5, the control device 11 calculates the linearly transformed low-dimensional vector ψ i(x,y) and the low-dimensional vector ψ to which the linear transformation was applied in step S510 j Based on (x, y), the displacement between frame i and frame j is calculated for each pixel (step S511).

[0090] The calculation of the transformation matrix M(x,y) described above does not take into account the displacement calculated by equation (3). For frame i, the low-dimensional vector ψ i (x,y) is a feature vector F i The transformation matrix M(x,y) is optimized to preserve the distance between (x,y).

[0091] By assuming that the distance (displacement) between frame i and frame j is very small, we can define a low-dimensional vector ψ i The distance indicated by (x,y) is the feature vector F i A constraint to preserve the distance denoted by (x,y) is added to the error function to be minimized according to equations (13) and (14).

[0092]

number

[0093]

number

[0094] E s are the spatial constraints defined by equation (10) and λ is their weight. Once the transformation matrix M(x,y) is optimized, the plane parameters ψ i (x,y) is optimized and the pixel-by-pixel displacement is ψ i It is given as the third element of (x,y).

[0095] The initial guess ψ for the nonlinear minimization of Eq. (13) i ' is the above calculated ψ after normalizing the pixel-by-pixel magnitude according to equations (15), (16), and (17). iis given by (x,y).

[0096]

number

[0097]

number

[0098]

number

[0099] Since the computed transformation matrix M(x,y) is subsampled, the transformation matrix M(x,y) is interpolated before optimizing the parameters to obtain the displacement for the image at the original resolution. The interpolation is performed using an initial guess ψ i ' The transformation is done by interpolating the displacement of the subsampled feature vector, which is calculated by using the transformation matrix M(x,y) and the original resolution pixel P1a(x,y).

[0100]

number

[0101]

number

[0102] The linear form of this transformation matrix M(x,y) is obtained for all pixels in the local patch, so that the transformation matrix M(x,y) is obtained as a least squares solution. Once the transformation matrix M(x,y) is obtained for all pixels, the optimization is applied to the image at the original resolution.

[0103] The displacement between frame i and frame j can be calculated by the process shown in Fig. 5. In the example described in Fig. 5, only the displacement between two frames is calculated, but when determining the continuous movement of object O, the processes from step S501 to step S511 are repeated. That is, a moving object is photographed an arbitrary number of times, and the displacement between the local planes of the corresponding number of images is calculated.

[0104] As described above, even if the movement of the object O is slight, the speckle pattern may change significantly depending on the distance between the object O and the imaging device 2. According to the present embodiment, the displacement of the feature vector is approximated for pixels in each local plane, so that the change in the feature vector in the local plane is small. And, the displacement of the feature vector is approximated for pixels in the subsequent local plane, so that the change in the feature vector between the local planes is also small. This leads to maintaining the correlation before and after the change in the speckle pattern.

[0105] In addition, when calculating displacement, the number of dimensions is reduced (from k to l, where l is preferably any number between 3 and 20), thereby reducing the noise that occurs when calculating displacement with a high number of dimensions, and reducing the amount of calculation.

[0106] By the processing according to the present embodiment, for example, the movement of an object can be visually expressed according to the change in the changing speckle pattern. An example will be described with reference to Fig. 7 and Fig. 8. Fig. 7 shows a state in which a finger is pressing the surface of a canvas-like cloth. When the surface of the cloth is pressed with a finger, vibration is transmitted from the pressed point due to the force of the finger.

[0107] In this state, by photographing the surface of the cloth multiple times at any point (the same or different points) of the cloth transmitting the vibration and analyzing the change in the speckle pattern according to the process described in Fig. 5, it is possible to express the change in vibration, for example, as shown in Fig. 8. The horizontal axis of the graph shown in Fig. 8 indicates the x-axis (i.e., the distance traveled laterally by the wave due to the vibration), and the vertical axis indicates the displacement (i.e., the displacement between the local planes for the two images).

[0108] The graph shown in Figure 8 is displayed on the display device 4 based on the calculated displacements under the control of the control device 11. The displacements correspond to the amplitude of the vibrations, and the x-axis is derived by summing the displacements. In this way, slight changes in the movement of the object can be visually represented by analyzing the speckle patterns in the images generated from the moving object.

[0109] As described above, the image analysis system 100 according to this embodiment has been described. According to this embodiment, even a slight movement of an object can be determined with high accuracy. Furthermore, a moving object O (object) can be expressed according to a change in the speckle pattern. For example, the movement of the object O may be dynamically displayed on the display device 4 according to the determined change in the speckle pattern.

[0110] It should be noted that the hardware components described in the above embodiment are merely exemplary, and other configurations are possible. Also, the order of the processes described in the above embodiment does not necessarily have to be performed in the order described, and may be performed in any order. Furthermore, additional steps may be newly added without departing from the basic concept of the present invention.

[0111] Furthermore, the image determination according to an embodiment of the present invention is implemented by a computer program executed by the computer device 1, and the computer program may be stored in a non-transitory storage medium. Examples of non-transitory storage media include read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disk devices, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs).

Claims

1. A computing device for determining motion of a subject, comprising: generating a first image including a first speckle pattern based on a first imaging signal generated by an imaging device capturing an image of the object at a first time; generating a second image including a second speckle pattern based on a second imaging signal generated by the imaging device imaging the object at a second time; Calculating a first feature vector from luminance in a first set of pixels surrounding a particular pixel in the first image, and approximating a displacement of the first feature vector between the first set of pixels; calculating a second feature vector from luminance in a second set of pixels surrounding a pixel corresponding to the particular pixel in the second image, and approximating a displacement of the second feature vector between the second set of pixels; determining a change between the first speckle pattern and the second speckle pattern by calculating a distance in a feature vector space between the approximated first feature vector and the approximated second feature vector for each particular pixel; A computing device comprising a control device configured to:

2. The control device includes: Calculating the first feature vector and the second feature vector in a first dimensionality; calculating the first feature vector with a second number of dimensions that is less than the first number of dimensions based on the first feature vector; calculating the second feature vector with the second dimensionality based on the second feature vector; 10. The computing device of claim 1, further configured to:

3. The computer device of claim 2 , wherein the control unit is further configured to calculate the first feature vector and the second feature vector according to a diffusion map.

4. 4. The computer device according to claim 2, wherein the second number of dimensions is any one of 3 to 20.

5. The control device includes: Extracting a first local plane from the first image by determining a specific region; and further configured to extract a second local plane from the second image by determining a region corresponding to the first local plane; the first set of pixels is a set of pixels in the first local plane; the second pixel set is a pixel set in the second local plane; A computing device according to any one of claims 1 to 3.

6. The control device includes: approximating the displacement of the first feature vector among the first set of pixels by applying a linear transformation to the first feature vector; approximating the displacement of the second feature vector among the second set of pixels by applying a linear transformation to the second feature vector.

10. The computing device of claim 1, further configured to:

7. 7. The computing device of claim 6, wherein the controller is further configured to apply the linear transformation to the first feature vector and the second feature vector under a constraint that pixel-to-pixel displacements for each of the particular pixels between the first set of pixels and the second set of pixels are similar.

8. 8. The computing device of claim 1, wherein the control device displays the movement of the subject on a display device based on the change between the first speckle pattern and the second speckle pattern.

9. 1. A method for determining motion of an object, the method comprising: generating a first image including a first speckle pattern based on a first imaging signal generated by an imaging device capturing an image of the target at a first time; generating a second image including a second speckle pattern based on a second imaging signal generated by the imaging device imaging the object at a second time; calculating a first feature vector from luminance in a first set of pixels surrounding a particular pixel in the first image, and approximating a displacement of the first feature vector between the first set of pixels; calculating a second feature vector from luminance in a second set of pixels surrounding a pixel corresponding to the particular pixel in the second image, and approximating a displacement of the second feature vector between the second set of pixels; determining a change between the first speckle pattern and the second speckle pattern by calculating a distance in feature vector space between the approximated first feature vector and the approximated second feature vector for each particular pixel; A method comprising:

10. A computer program comprising computer executable instructions which, when executed by a computing device, cause the computing device to perform the method of claim 9.

Images