MOTION AMPLIFICATION DEVICE AND METHODS OF USING SAME - Patent application

Abstract

A motion amplification device according to an embodiment includes an encoder that receives any adjacent first and second frames in a video, decomposes the first frame into first shape information and first texture information, and decomposes the second frame into second shape information and second texture information, a first module that generates a frame in which the motion of an object is amplified based on the first shape information, the second shape information, and the second texture information, a second module that analyzes the magnitude of the motion based on the first shape information, the second shape information, and the first texture information, and a third module that generates amplified video data that displays the magnitude of the motion on the motion-amplified frame.

Description

[Technical field]

[0001] The present disclosure relates to a motion amplification device and a method for using the same. [Background technology]

[0002] When a problem occurs in equipment used in industrial sites, abnormal vibrations or behaviors occur. For example, in the case of aging rotating equipment, abnormal vibrations can occur in the direction of the rotation axis rather than the direction of rotation. A common method is for an inspector to check for defects in equipment with the naked eye, but this has problems such as the difficulty of confirming the magnitude of vibration when the vibration frequency is high, and the inspector's subjective judgment varies, making long-term monitoring cumbersome.

[0003] For this reason, motion magnification technology is used to amplify and visualize minute motions in order to visualize defects in equipment. Motion magnification technology generally uses the Eulerian method, which estimates motion from changes in the physical quantities of objects passing through each point fixed in space. Summary of the Invention [Problem to be solved by the invention]

[0004] A problem with motion amplification techniques is that they do not provide an accurate numerical value for the magnitude of motion, which is a necessary element for determining whether equipment is defective.

[0005] In addition, the magnitude of the amplified motion by the conventional motion amplifier is not equal to the preset amplification coefficient, and the quality of the amplified image is poor.

[0006] The present disclosure is intended to solve such problems, and aims to provide a numerical value of the motion along with the motion amplification of the equipment.

[0007] The present disclosure provides an enhanced visual indication of an object's motion while simultaneously providing a quantitative analysis of the motion. [Means for solving the problem]

[0008] A motion amplification device according to one embodiment includes an encoder that receives any adjacent first and second frames in a video, decomposes the first frame into first shape information and first texture information, and decomposes the second frame into second shape information and second texture information, a first module that generates a frame in which the motion of an object is amplified based on the first shape information, the second shape information, and the second texture information, a second module that analyzes the magnitude of the motion based on the first shape information, the second shape information, and the first texture information, and a third module that generates amplified video data that displays the magnitude of the motion on the motion-amplified frame.

[0009] The first module can generate new shape information by multiplying a difference between the first shape information and the second shape information by a preset amplification coefficient, and can generate a frame with amplified motion by synthesizing the generated shape information, the first shape information, and the second texture information.

[0010] The second module is configured to calculate a change in each pixel between the first frame and the second frame based on the first shape information and the second shape information, and analyze a magnitude of motion of the object based on the calculated change in each pixel.

[0011] The second module is configured to analyze the magnitude of object motion using a Convolutional Neural Network (CNN) trained to analyze the magnitude of motion from shape information of any input frame.

[0012] The third module further includes a memory storing a vibration threshold for the object and an output unit for outputting amplified image data, and when the magnitude of the motion exceeds the vibration threshold, the third module determines that the area where the motion exceeding the threshold is detected is a dangerous area requiring confirmation, and generates amplified image data to display the dangerous area on the amplified image data.

[0013] A motion amplification method according to one embodiment includes the steps of receiving any adjacent first and second frames in an image, decomposing the first frame into first shape information and first texture information and decomposing the second frame into second shape information and second texture information, generating a frame in which the motion of the object is amplified based on the first shape information, the second shape information, and the second texture information, analyzing the magnitude of the motion based on the first shape information, the second shape information, and the first texture information, and generating amplified image data that displays the magnitude of the motion on the motion-amplified frames.

[0014] The step of generating a frame with amplified motion may include the steps of multiplying a difference between the first shape information and the second shape information by a preset amplification factor to generate new shape information, and synthesizing the generated shape information, the first shape information, and the second texture information to generate the frame with amplified motion.

[0015] The step of analyzing the magnitude of motion may include a step of calculating a change in each pixel between the first frame and the second frame based on the first shape information and the second shape information, and analyzing the magnitude of motion of the object based on the calculated change in each pixel.

[0016] The step of analyzing the magnitude of motion may further include a step of analyzing the magnitude of motion of the object using a Convolutional Neural Network (CNN) trained to analyze the magnitude of motion from shape information of any input frame.

[0017] The step of generating a frame in which the motion is amplified may include a step of determining, when the magnitude of the motion exceeds a vibration threshold prestored for the object, an area in which motion exceeding the threshold is detected as a dangerous area requiring confirmation, and generating amplified image data to display the dangerous area on the amplified image data.

[0018] A recording medium according to an embodiment may store a program for performing a motion amplification method.

[0019] A program according to an embodiment may be stored on a recording medium to perform the motion amplification method. Effect of the Invention

[0020] According to at least one embodiment of the present disclosure, more accurate motion amplification results can be obtained.

[0021] At least one embodiment of the present disclosure facilitates analyzing the motion of an object.

[0022] At least one embodiment of the present disclosure provides a visual indication of portions of an object that require attention. [Brief description of the drawings]

[0023] [Figure 1] 1 is a block diagram of a motion amplification system according to one embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram of a control unit of the motion amplifying device of the present invention. [Diagram 3] 3 is a flow chart of a motion amplification method according to the present invention. [Figure 4] FIG. 2 shows an example of a screen displayed on the output section of a motion amplification device according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the attached drawings, and the same or similar components are denoted by the same or similar reference numerals, and the duplicated description thereof will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed for the sake of ease of specification preparation only, and do not have any meaning or role that is different from each other. In addition, when describing the embodiments disclosed in the present specification, if it is determined that a detailed description of such known technology may obscure the gist of the embodiments disclosed in the present specification, the detailed description will be omitted. In addition, the attached drawings are merely for making the embodiments disclosed in the present specification easy to understand, and the technical ideas disclosed in the present specification are not limited by the attached drawings, and it should be understood that the drawings include all modifications, equivalents, and alternatives included in the idea and technical scope of the present invention.

[0025] It should be understood that in this application, terms such as "comprise" or "have" are intended to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0026] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0027] FIG. 1 is a block diagram of a motion amplification system according to one embodiment of the present invention.

[0028] The motion amplification system 1 includes a camera 10 and a motion amplification device 20 .

[0029] The camera 10 and the motion amplifier 20 can be linked via a network.

[0030] In the present invention, a network refers to a connection structure that allows information exchange between each node such as a device and a server. Examples of such networks include, but are not limited to, a local area network (LAN), a wide area network (WAN), a broadband network (BBN), a wireless LAN (WLAN), a Long Term Evolution (LTE), a LTE Advanced (LTE-A), a Code-Division Multiple Access (CDMA), a Wideband Code Division Multiplex Access (WCDMA) (registered trademark), a Universal Mobile Telecommunication System (UMTS), a Wireless Broadband (WiBro), a Global System for Mobile Communications (GSM), a Bluetooth Low Energy (BLE), a Bluetooth (Bluetooth) (registered trademark), a Zigbee, an Ultra-Wide Band (UWB), an ANT, a Wi-fi, an infrared data association (IrDA), and a Personal Area Network (PAN).

[0031] The camera 10 is a device that captures an object whose motion is to be analyzed, and may include various types of image sensors such as a charge coupled device (CCD) and a complementary metal oxide semiconductor (CMOS). For example, the camera 10 may capture images of equipment at an operating site.

[0032] The camera 10 can capture an image of an object in real time. The image captured by the camera 10 is transmitted to the motion amplifier 2 via a network.

[0033] The motion amplifier 20 is a device that analyzes the image received from the camera 10, detects minute motions present in the image, and amplifies and analyzes the detected minute motions.

[0034] The motion amplification device 20 includes a communication unit 210 , a control unit 230 , a memory unit 250 , and an output unit 270 .

[0035] The communication unit 210 is for communicating with the camera 10 via a network. The communication unit 210 can transmit an image received from the camera 10 to the control unit 230.

[0036] The control unit 230 may include an artificial neural network that learns from the images stored in the memory unit 250 .

[0037] The artificial neural network includes a plurality of layers, and the layers are connected by a plurality of operations to which weights are applied. In other words, the artificial neural network includes a plurality of layers including a plurality of operations to which weights are applied. Here, the plurality of layers including a plurality of operations may include a convolution layer that performs a convolution operation, a pooling layer that performs a downsampling operation, an unpooling layer (UL) that performs an upsampling operation, a deconvolution layer (DL) that performs a deconvolution operation, etc.

[0038] Meanwhile, training an artificial neural network means providing an input having a known output to the neural network to generate a predicted output, comparing the output predicted through the neural network with the known output, and modifying the algorithm of the artificial neural network so that the difference is reduced. For example, the artificial neural network is trained using a gradient decent method. This can be repeated several times, and the artificial neural network can produce more accurate outputs as it is repeated.

[0039] The control unit 230 can detect minute motion according to time in an image including a plurality of frames using an artificial neural network, and amplify the detected minute motion. The control unit 230 can generate an amplified image including the amplified minute motion. The amplified image is an image in which an area in which minute motion is detected in a transmitted image is emphasized. That is, the amplified image is an image reconstructed by amplifying minute motion by a preset amplification coefficient. The amplified image is composed of a plurality of amplified image frames.

[0040] The control unit 230 may analyze the movement displacement of each pixel using an artificial neural network. Also, the control unit 230 may derive the magnitude of vibration of the micro-motion based on the analyzed movement displacement of each pixel. Hereinafter, the movement displacement of each pixel is referred to as the magnitude of the micro-motion. The control unit 230 may generate data such that the magnitude of the micro-motion is displayed together in the image. Furthermore, the control unit 230 may generate amplified image data such that the magnitude of the micro-motion is displayed together in the amplified image. The amplified image data is data in which the analyzed magnitude of the micro-motion is displayed together in the amplified image.

[0041] The control unit 230 may include an artificial neural network that additionally learns the generated amplified image and amplified image data in addition to learning the images stored in the memory unit 250 .

[0042] The control unit 230 may determine whether the magnitude of the analyzed minute motion exceeds a preset threshold. If the magnitude of the minute motion exceeds the threshold, the control unit 230 may determine that the area where the minute motion exceeding the threshold is detected is a dangerous area requiring confirmation.

[0043] The control unit 230 may transmit the amplified image and the amplified image data to the output unit 300 .

[0044] The memory unit 250 stores the image received from the camera 10, the amplified image generated by the control unit 230, amplified image data including the magnitude of the minute motion, any data set required for training the artificial neural network of the control unit 230, a vibration threshold value for each piece of equipment, etc. Here, the memory unit 250 has been described as being located within the motion amplification device 20, but it may be a separate database located outside the motion amplification device 20 and communicating with the motion amplification device 20.

[0045] The output unit 270 may be a display that outputs data transmitted from the control unit 230. For example, the output unit 270 may output an amplified image or amplified image data. The output unit 270 may also output an image in which the magnitude of motion is displayed within the image received from the camera 10.

[0046] The user can determine via the output unit 270 whether or not the object photographed by the camera 10 requires confirmation.

[0047] FIG. 2 is a diagram showing the configuration of the control unit of the motion amplifying device of the present invention.

[0048] The control unit 230 includes an encoder 231 , a first module 233 , a second module 235 , and a third module 237 .

[0049] The encoder 231 is configured to decompose an input frame into shape information (Shape) and texture information (Texture). Specifically, the encoder 231 performs spatial decomposition on each of the two input frames, and can obtain shape information (Shape n) and texture information (Texture n) for each frame.

[0050] The encoder 231 receives two arbitrary adjacent frames and shape information and texture information for the two frames from the memory unit 250. The encoder 231 learns to satisfy a regularization term so as to decompose the input frames into shape information and texture information. The regularization term may include, for example, a constraint that the shape information must be the same even if the texture changes within one frame in order to decompose the shape information, or a constraint that the textures within two adjacent frames must be the same in order to decompose the texture information. The encoder 231 learns until it outputs already known shape information and texture information for the two frames transmitted from the memory unit 250.

[0051] Thereafter, the trained encoder 231 can receive any two adjacent frames (Frame n, Frame n+1) in the image transmitted from the communication unit 210. The encoder 231 can transmit decomposed shape information (Shape n, Shape n+1) and texture information (Texture n, Texture n+1) for each frame to the first module 233 and the second module 235.

[0052] The first module 233 is configured to generate an amplified frame for the frame received from the encoder 231, and includes a manipulator 2331 and a decoder 2333.

[0053] The amplifier 2331 is configured to amplify the difference between the shape information (Shape n, Shape n+1), that is, the fine motion. Specifically, the amplifier 2331 may receive the decomposed shape information (Shape n, Shape n+1) for two consecutive frames from the encoder 231, and then obtain the difference between the shape information. Thereafter, the amplifier 2331 may multiply the difference between the shape information by a specific amplification coefficient (α) to generate new shape information (Shape n+1') including the amplified fine motion. Here, the amplification coefficient (α) may be a preset coefficient.

[0054] The amplifier 2331 can transmit the new shape information (Shape n+1′) to the decoder 2333 .

[0055] The decoder 2333 is configured to combine texture information and shape information into one frame.

[0056] Specifically, the decoder 2333 can synthesize the texture information (Texture n+1) decomposed by the encoder 231 and the shape information (Shape n+1') amplified by the amplifier 2331 into one frame to generate a frame (Frame n+1') with globally amplified motion.

[0057] The second module 235 is configured to analyze the magnitude of a minute motion between frames received from the encoder 231. Specifically, the second module 235 can analyze the magnitude of the minute motion by applying an optical flow technique to two consecutive frames input to the encoder 231 to generate a flow map indicating the result of calculating the motion at every pixel.

[0058] The optical flow technique is a technique for estimating motion information, such as instantaneous velocity, of an object moving spatially within a frame by calculating partial derivatives with respect to the spatial and temporal coordinates of the object. Specifically, the optical flow technique can obtain information about the motion of an object in the two frames, such as the motion intensity, by finding the correspondence between two input frames using the pixel changes between adjacent frames and the time interval between adjacent frames. Such optical flow techniques belong to the Lagrangian method.

[0059] The second module 235 may analyze the magnitude of the motion of the object using a trained artificial neural network. Meanwhile, in FIG. 2, the second module 235 uses a Convolutional Neural Network (CNN) 2351 model when analyzing the magnitude of the fine motion, but is not limited thereto, and the second module 235 may use any artificial neural network.

[0060] The CNN 2351 is an artificial neural network composed of convolutional layers. The CNN 2351 is trained to analyze the magnitude of motion using an arbitrary data set transferred from the memory unit 250. The CNN 2351 is trained until the loss value, which is the difference between the analysis result and the correct answer, becomes equal to or less than a reference value.

[0061] The second module 235 may analyze the magnitude of motion between the shape information of frames input from the encoder 231 via the trained CNN 2351. In other words, the flow map may be a value obtained by inputting frames to the pre-trained CNN 2351.

[0062] In summary, the second module 235 receives shape information (Shape n, Shape n+1) and texture information (Texture n) and calculates the pixel change between the two pieces of shape information (Shape n, Shape n+1), enabling quantitative analysis of the movement displacement for each pixel.

[0063] As described above, the second module 235 may use an artificial neural network to analyze the magnitude of object motion in frames received from the camera 10 via the communication unit 210 .

[0064] In conclusion, the motion amplification device 20 can generate an amplified image via the first module 233 and generate a flow map by analyzing the magnitude of motion for an object via the second module 235.

[0065] The third module 237 may generate amplified image data that displays the magnitude of the motion analyzed by the second module 235 on the frame in which the motion is amplified by the first module 233. The third module 237 may further generate amplified image data that further displays a warning display on a portion having vibration exceeding a threshold when the movement displacement of the analyzed object exceeds a certain threshold.

[0066] However, in a method of generating an amplified image with amplified motion using deep learning, it may be difficult to completely decompose a frame into shape information and texture information by simply adding constraints to the encoder and having it learn. If a frame cannot be completely decomposed into shape information and texture information, a wobble effect may occur in which the boundary of an object with minute motion and the background are both amplified and wobbly. In this case, since it is impossible to accurately measure the magnitude of the motion, an amplified image that does not match the preset amplification coefficient may be output.

[0067] In response to this, the motion amplification device 20 provides the shape information (Shape n, Shape n+1) and texture information (Texture n, Texture n+1) output from the encoder 231 not only to a first module 233, which is a motion amplification model based on Euler's method, but also to a second module 235, which is an optical flow model based on Lagrange's method.

[0068] The motion amplifier 20 is data efficient because it is possible to train both the first module 233 and the second module 235 with one data set.

[0069] Thereafter, by providing information on the flow map, which is the output of the second module 235, to the first module 233, the encoder 231 can completely resolve shape information and texture information for the input frame according to the flow map, so that the possibility of wobble effects occurring is low. Also, the magnitude of the object motion can be accurately analyzed through the second module 235, and an amplified image that matches the magnitude of the amplification coefficient that the first module 233 is trying to amplify can be obtained.

[0070] FIG. 3 is a flow chart of a motion amplification method according to the present invention.

[0071] First, the encoder 231 receives the n-th frame and the (n+1)-th frame (S301).

[0072] The nth frame and the n+1th frame may be adjacent frames. The encoder 231 may be trained to decompose a frame input to the encoder 231 into shape information and texture information using an arbitrary data set pre-stored in the memory unit 250.

[0073] Then, the encoder 231 decomposes the nth frame into the nth shape information and the nth texture information, and decomposes the n+1th frame into the n+1th shape information and the n+1th texture information (S303). The encoder 231 can transmit the decomposed nth shape information and nth texture information, and the n+1th shape information and n+1th texture information to the first module 233 and the second module 235.

[0074] The first module 233 multiplies the difference between the nth shape information and the (n+1)th shape information by an amplification factor to generate amplified (n+1)th shape information (S305).

[0075] The first module 233 synthesizes the amplified (n+1')th shape information, the nth shape information, and the (n+1)th texture information to generate the (n+1)th frame in which motion is amplified (S307). Thus, the first module 233 can generate an amplified image in which motion is emphasized.

[0076] At the same time, the second module 235 calculates the change in each pixel between the nth frame and the n+1th frame based on the nth shape information and the n+1th shape information (S309).

[0077] The second module 235 quantitatively analyzes (S311) the displacement based on the calculated change in each pixel.

[0078] Thereafter, the third module 237 displays the digitized movement displacement on the (n+1)th frame in which the motion is amplified through the output unit 270 (S313).

[0079] When the movement displacement of the object analyzed through the output unit 270 exceeds a certain threshold, the third module 237 displays a warning display on a portion having vibration exceeding the threshold. In this case, the certain threshold may be a vibration threshold for each facility stored in the memory 250 in advance.

[0080] FIG. 4 shows an example of a screen displayed on the output of the motion amplifier when the camera captures a rotating installation.

[0081] 4(a) is a screen showing one frame of an image captured by the camera 10 when the camera 10 captures an equipment rotating around the y-axis. The camera 10 may be installed to capture a problematic part of the equipment that needs to be observed. In this case, the camera 10 can capture an image of the equipment in real time and transmit it to the motion amplification device 20 via a network.

[0082] The problem here is that the equipment can vibrate parallel to the axis of rotation, i.e. in the y-axis direction. Motion parallel to the axial direction is undesirable behavior for the equipment and needs to be monitored.

[0083] The motion amplification device 20 can receive video from the camera 10 , amplify axially parallel motion via a first module 233 , and analyze the motion via a second module 235 .

[0084] FIG. 4(b) is an example of the amplified video data.

[0085] In (b) of Fig. 4, the movement displacement for the motion of the equipment analyzed through the second module 235 is displayed together with the amplified image generated by the first module 233. For example, the behavior of the equipment in the y-axis direction is displayed.

[0086] FIG. 4(c) is an example of amplified image data in which a warning display appears in a portion of an object having vibration exceeding a threshold value.

[0087] The memory unit 250 may store a threshold value of vibration that may occur when the equipment photographed by the camera 10 is operating safely. When the equipment vibrates beyond a pre-stored threshold value, it may be in an unsafe state. When the equipment vibrates in the y-axis direction beyond a certain threshold value, the control unit 230 may display a warning message on the relevant portion.

[0088] According to the motion amplification device of the present disclosure, the vibration of the equipment can be amplified and visualized on a screen so that it can be grasped with the naked eye.

[0089] In addition, the motion amplification device according to the present disclosure can quantitatively analyze the magnitude of motion of equipment suspected of being defective, and quantitatively measure the magnitude of vibration of the equipment, thereby making it possible to grasp the degree of the defect according to the measurement results.

[0090] In addition, the motion amplification device according to the present disclosure has an economic advantage by replacing contact sensors. Costs can be reduced by replacing expensive contact sensors for local diagnostic analysis and measurement with a GPU and a camera. Furthermore, unlike contact sensors, a single moving sensor camera is used, which allows intuitive diagnosis over a wide area and allows multiple equipment diagnosis with a small number of sensors. This allows accurate monitoring of defects in equipment used in various industrial sites and early diagnosis of safe facility defects.

[0091] The above-described embodiments may be realized in the form of a computer program executable by various components on a computer, and such a computer program may be recorded on a computer-readable medium, which may include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memory units, etc.

[0092] Unless the steps constituting the method according to the embodiment are clearly described in a certain order or are described to the contrary, the steps are performed in any suitable order. The order of the steps is not necessarily intended to limit the present invention. In the present invention, the use of all examples or exemplary terms (such as, for example, etc.) is merely for the purpose of explaining the present invention in detail, and is not intended to limit the scope of the present invention. In addition, a person skilled in the art will understand that various modifications, combinations and changes can be made within the scope of the claims or their equivalents.

[0093] Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited to these embodiments, and various modifications and improvements made by those having ordinary skill in the art to which the present invention pertains also fall within the scope of the present invention.

Claims

1. an encoder for receiving any first and second adjacent frames in a video, and for decomposing the first frame into first shape information and first texture information, and for decomposing the second frame into second shape information and second texture information; a first module for generating a frame in which motion of an object is amplified based on the first shape information, the second shape information, and the second texture information; a second module for analyzing a magnitude of the motion based on the first shape information, the second shape information, and the first texture information; a third module for generating amplified image data that displays the magnitude of the motion on the motion-amplified frame; 4. A motion amplifying device comprising:

2. the first module multiplies a difference between the first shape information and the second shape information by a preset amplification factor to generate new shape information, and synthesizes the generated shape information and the second texture information to generate a frame in which the motion is amplified; 2. The motion amplification device of claim 1.

3. the second module is configured to calculate a change in each pixel between the first frame and the second frame based on the first shape information and the second shape information, and analyze a magnitude of motion of the object based on the calculated change in each pixel.

3. The motion amplifier of claim 2.

4. The second module is configured to analyze the magnitude of motion of the object using a Convolutional Neural Network (CNN) trained to analyze the magnitude of motion from shape information of any input frame.

4. The motion amplifier of claim 3.

5. a memory storing a vibration threshold value for the object; an output unit that outputs the amplified video data; Further comprising: The third module, when the magnitude of the motion exceeds the vibration threshold, determines an area where the motion exceeding the threshold is detected as a dangerous area requiring confirmation, and generates the amplified image data so as to display the dangerous area on the amplified image data.

4. The motion amplifier of claim 3.

6. receiving any first and second frames adjacent to each other in an image, decomposing the first frame into first shape information and first texture information, and decomposing the second frame into second shape information and second texture information; generating a frame in which motion of an object is amplified based on the first shape information, the second shape information, and the second texture information; analyzing a magnitude of the motion based on the first shape information, the second shape information, and the first texture information; generating amplified image data that displays a magnitude of the motion on a frame in which the motion is amplified; A motion amplification method comprising:

7. The generating of the motion amplified frames comprises: generating new shape information by multiplying a difference between the first shape information and the second shape information by a preset amplification factor, and synthesizing the generated shape information and the second texture information to generate a frame in which the motion is amplified. The motion amplification method of claim 6.

8. The step of analyzing the magnitude of the motion includes: calculating a change in each pixel between the first frame and the second frame based on the first shape information and the second shape information, and analyzing a magnitude of motion of the object based on the calculated change in each pixel. The motion amplification method of claim 7.

9. The step of analyzing the magnitude of the motion includes: The method further includes analyzing the magnitude of motion of the object using a Convolutional Neural Network (CNN) trained to analyze the magnitude of motion from shape information of an arbitrary input frame. The motion amplification method of claim 8.

10. The generating of the motion amplified frames comprises: and generating the amplified image data so as to display the danger area on the amplified image data when the magnitude of the motion exceeds a vibration threshold prestored for the object and the area where the motion exceeding the threshold is detected is determined to be a danger area requiring confirmation.

10. The motion amplification method of claim 9.

11. A recording medium storing a program for carrying out the method according to any one of claims 6 to 10.

12. A program stored on a recording medium so as to perform the method according to any one of claims 6 to 10.

Images