Vibration analysis device, vibration analysis method and program

Abstract

To provide a vibration analyzer, a method for analyzing a vibration, and a program that can precisely analyze a vibration even when the amplitude of an analysis target is unknown.SOLUTION: A vibration analyzer 1 includes: an analysis target recognition unit 212 for recognizing an analysis target from a first time-series image by a method of mechanical learning; and a vibration analysis unit 213 for calculating the state of the vibration as an analysis target by template matching on the basis of a second time-series image with a higher sampling rate than that of the first time-series image and the coordinates of the analysis target recognized by the analysis target recognition unit 212.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a vibration analysis device, a vibration analysis method, and a program. [Background technology]

[0002] In recent years, with the advancement of image recognition technology, vibration analysis methods using image recognition have been developed. For example, Patent Document 1 discloses a method of measuring the vibration of a propeller shaft using template matching. Vibration analysis using template matching is performed by calculating the amount of displacement of an analysis target from the similarity of the image of the analysis target. In such a method using template matching, it is possible to detect fluctuations at a sub-pixel level depending on the analysis conditions, and highly accurate analysis can be performed. However, since the amount of calculation is large and the time required for calculation is long, fluctuation detection is generally performed using images of a local range (ROI: Region Of Interest) rather than a wide range of images.

[0003] Another image recognition method that has been developed is a technology that uses machine learning techniques to track an object of analysis. In this method, a pre-trained model is used to process an image and recognize the object of analysis, thereby calculating the displacement of the object of analysis. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-32496 A Summary of the Invention [Problem to be solved by the invention]

[0005] When performing vibration analysis using image recognition, it is required to detect the displacement of the analysis target with high accuracy, that is, to detect minute displacements at a high sampling rate. Therefore, a method is used to analyze the vibration state by analyzing the image of the region of interest around the analysis target from time-series images using local template matching, image correlation, etc.

[0006] However, when performing vibration analysis using the local object tracking method, if the amplitude (displacement) of the object to be analyzed becomes large, the amount of movement of the object to be analyzed within the region of interest will also become large, and in some cases, the object may move outside the region of interest, making it impossible to analyze the vibration state.

[0007] In addition, when analyzing the displacement of an object to be analyzed by image recognition using machine learning techniques, it is possible to detect and recognize multiple objects to be analyzed over a wide area, but the measurement accuracy of the displacement amount is low (at the level of a few pixels) and the calculation time required for image recognition is long.

[0008] The present invention has been made in consideration of the above-mentioned circumstances, and has an object to provide a vibration analysis device, a vibration analysis method, and a program that are capable of performing vibration analysis with high accuracy even when the amplitude of the object to be analyzed is unknown. [Means for solving the problem]

[0009] In order to achieve the above object, a vibration analysis device according to a first aspect of the present invention comprises: By using machine learning techniques, Includes multiple analysis targets From the first time series The above an analysis target recognition unit that recognizes an analysis target; a vibration analysis unit that calculates a vibration state of an analysis object by template matching based on a second time-series image having a higher sampling rate than the first time-series image and the coordinates of the analysis object recognized by the analysis object recognition unit. 、 The sampling rate of the first time-series images is adjusted based on the largest amplitude among the amplitudes of the plurality of analysis targets.

[0010] Moreover, the first time-series image is a thinned image of the second time-series image. This may also be the case.

[0011] Furthermore, the coordinates of the analysis target are three-dimensional coordinates calculated from a plurality of second time-series images taken from different viewpoints. This may also be the case.

[0014] A vibration analysis method according to a second aspect of the present invention comprises: By using machine learning techniques, Includes multiple analysis targets From the first time series The above an analysis target recognizing step of recognizing an analysis target; and a vibration analysis step of calculating a vibration state of the analysis object by template matching based on a second time-series image having a sampling rate higher than that of the first time-series image and the coordinates of the analysis object recognized in the analysis object recognition step. fruit, The sampling rate of the first time-series images is adjusted based on the largest amplitude among the amplitudes of the plurality of analysis targets.

[0015] Moreover, the program according to the third aspect of the present invention comprises: Computer, By using machine learning techniques, Includes multiple analysis targets From the first time series The above an analysis target recognition unit that recognizes an analysis target; a vibration analysis unit that calculates a vibration state of an analysis object by template matching based on a second time-series image having a higher sampling rate than the first time-series image and the coordinates of the analysis object recognized by the analysis object recognition unit. At the same time, A sampling rate of the first time-series images is adjusted based on the largest amplitude among the amplitudes of the plurality of analysis targets. Effect of the Invention

[0016] According to the vibration analysis device, vibration analysis method, and program of the present invention, the analysis object is recognized using a machine learning method at a low sampling rate, and the vibration of the analysis object is analyzed using a template matching method at a high sampling rate, so that it is possible to perform vibration analysis with high accuracy even when the amplitude of the analysis object is unknown. [Brief description of the drawings]

[0017] [Figure 1] 1 is a functional block diagram of a vibration analysis device according to an embodiment of the present invention. [Diagram 2] 4 is a flowchart showing a flow of vibration analysis according to the embodiment. [Diagram 3] FIG. 1A is a schematic diagram showing an analysis object and a vibration analysis device, and FIG. 1B is a plan view of the analysis object. [Figure 4] These are examples of images of the subject to be analyzed captured by the camera units, where (A) is an image captured by the left camera and (B) is an image captured by the right camera. [Diagram 5] 10A and 10B show examples of vibration analysis results, in which (A) is a graph showing the change in displacement over time, (B) is a graph showing the change in vibration frequency over time, and (C) is an enlarged view of the graphs for one analysis target in (A) and (B). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] A vibration analysis device 1 according to an embodiment of the present invention will be described below with reference to the drawings. As shown in the block diagram of Fig. 1, the vibration analysis device 1 according to the present embodiment includes a camera unit 10 and a control unit 20.

[0019] The camera unit 10 is for taking a video including an object of vibration analysis, and is provided with a camera for taking time-series images at a predetermined basic frame rate F0. The type and number of cameras of the camera unit 10 are not particularly limited, and there may be one or more cameras. The camera unit 10 according to this embodiment is provided with a right camera 11R and a left camera 11L. This allows the vibration analysis device 1 to detect the three-dimensional position of the object of analysis using a stereo method from images taken from different viewpoints, and therefore allows the three-dimensional vibration state of the object of analysis to be analyzed.

[0020] The analysis target in the shooting range captured by the camera unit 10 may be one or more. For example, the analysis target may be a marker attached to each part of an automobile, a machine tool, etc., where the vibration state is to be analyzed. In this case, it is preferable that the marker is one that can be tracked by image recognition with high accuracy, and for example, an AR marker such as an ArUco marker can be used.

[0021] The right camera 11R and the left camera 11L capture images including the analysis target at a predetermined basic frame rate F0, and output the image data at each time.

[0022] The control unit 20 is, for example, a computer device, and includes a control unit 21, a storage unit 22, a display unit 23, and an input unit 24, as shown in FIG.

[0023] The control unit 21 is composed of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and controls the operation of the vibration analysis device 1. The control unit 21 also analyzes the vibration state of the analysis target based on image data of the analysis target captured by the camera unit 10. The control unit 21 loads various operation programs and data stored in the ROM, storage unit 22, etc. of the control unit 21 into the RAM and operates the CPU and GPU, thereby realizing each function of the control unit 21 shown in FIG. 1. As a result, the control unit 21 operates as an image data acquisition unit 211, an analysis target recognition unit 212, and a vibration analysis unit 213.

[0024] The image data acquisition unit 211 controls the right camera 11R and the left camera 11L of the camera unit 10 to capture an image including an analysis target. The image data acquisition unit 211 also acquires captured image data in time series from the camera unit 10.

[0025] The analysis target recognition unit 212 detects an analysis target included in the captured image and calculates the coordinates of the analysis target based on the image data acquired by the image data acquisition unit 211. When multiple analysis targets are detected in the captured image, the analysis target recognition unit 212 calculates the coordinates of each analysis target.

[0026] The vibration analysis unit 213 analyzes the vibration state of the analysis object based on the image acquired by the image data acquisition unit 211 and the coordinates of the analysis object calculated by the analysis object recognition unit 212. More specifically, the vibration analysis unit 213 sets a region of interest based on the coordinates of the analysis object calculated by the analysis object recognition unit 212, and calculates the amount of displacement of the analysis object within the region of interest by template matching. In addition, the vibration analysis unit 213 calculates the amplitude, vibration frequency, etc. of the analysis object from the time change of the calculated displacement of the analysis object, and analyzes the vibration state of the analysis object.

[0027] The storage unit 22 is a non-volatile memory such as a hard disk or a flash memory, and stores a template image of the analysis target, a calculation program for analyzing the vibration state of the analysis target from time-series image data, data indicating the vibration state of the analysis target calculated by the vibration analysis unit 213, etc.

[0028] The display unit 23 is a display device, such as a liquid crystal display, provided in the control unit 20, which is a computer device. The display unit 23 displays images including the analysis target captured by the right camera 11R and the left camera 11L of the camera unit 10, the vibration analysis results by the vibration analysis unit 213, and the like.

[0029] The input unit 24 is an input device for inputting settings of markers to be analyzed (selection of a template image), instructions to start and end the analysis, etc. The input unit 24 is a keyboard, a touch panel, a mouse, etc., provided in the control unit 20.

[0030] Next, a vibration analysis method using the vibration analysis device 1 will be described with reference to the flowchart of Fig. 2. In this embodiment, as shown in Fig. 3(A) and (B), an example will be described in which an iron plate with multiple ArUco markers attached is vibrated by a vibration tester, photographed, and the vibration of each part to which the ArUco markers to be analyzed are attached is analyzed from the photographed time-series images.

[0031] First, as analysis condition settings, a basic frame rate F0 of the right camera 11R and the left camera 11L of the camera unit 10, a first frame rate F a , the second frame rate F t In this embodiment, the base frame rate F0 and the second frame rate F t 1000fps, the first frame rate F aIn other words, the vibration analysis by template matching is performed using the second time-series images consisting of all the images captured by the right camera 11R and the left camera 11L, and the AI ​​recognition is performed using the first time-series images consisting of the thinned captured images.

[0032] Furthermore, the control unit 20 of the vibration analysis device 1 uses the camera unit 10 to capture an image including the analysis target, and performs camera calibration. Specifically, the position and identification number of each marker that is the analysis target are detected, and calibration parameters for the two-dimensional and three-dimensional positions of each marker are calculated. In this embodiment, the two-dimensional and three-dimensional positions of the markers are calculated using two cameras, the right camera 11R and the left camera 11L, so if there are four or more markers, camera calibration can be performed automatically based on the calculated coordinates of each marker.

[0033] After the camera calibration is completed, the control unit 20 acquires a template image to be used for template matching (step S2). Specifically, an initial image captured by the camera unit 10 is transmitted from the image data acquisition unit 211 to the analysis target recognition unit 212. The analysis target recognition unit 212 then detects markers to be analyzed from the initial image, and stores images of regions of interest near each marker as template images in the storage unit 22 together with the identification numbers of the markers.

[0034] When the vibration tester starts to apply vibration to the iron plate and an instruction to start analysis is input to the input unit 24, the image data acquisition unit 211 controls the camera unit 10 to capture the right image I captured by the right camera 11R at a basic frame rate F0. R ,The left image I is taken by the left camera 11L. L The image data acquisition unit 211 also acquires image data of the acquired right image I R , Left image I L The image data is transmitted to the analysis target recognition unit 212 (step S3).

[0035] Next, as an analysis target recognition step, the analysis target recognition unit 212 recognizes a first frame rate F a (=20 fps). Specifically, the time T of the received image data is detected at the first frame rate F a When the time T of the received image data is within the time interval t (50 ms) corresponding to the first frame rate F (YES in step S4), the analysis target recognition unit 212 detects the analysis target by AI recognition. a If it is not the time of the time interval t (50 ms) corresponding to (NO in step S4), the analysis object recognition unit 212 does not perform recognition processing of the analysis object by AI recognition, and proceeds to vibration analysis processing of the analysis object that has already been AI-recognized.

[0036] The analysis target recognition unit 212 detects a first frame rate F a In this embodiment, the position (three-dimensional coordinates) of the analysis target is recognized from the first time-series images consisting of the captured images thinned out so that R , Left image I L The three-dimensional position of the object to be analyzed is calculated using the stereo method.

[0037] Furthermore, detection of an analysis target as AI recognition according to this embodiment is performed using a pre-trained model that is trained by a machine learning technique such as a neural network so as to recognize an analysis target from an input image. This type of AI recognition is a suitable method for detecting an analysis target from the entire captured area, but performing recognition processing on the entire image may take a long time for calculations. Therefore, in this embodiment, the sampling rate of the first time-series image for which AI recognition is performed (first frame rate F a ) to obtain the sampling rate of the second time-series image for template matching (second frame rate F t This increases the time interval of the first time-series image, ensuring the calculation time required for AI recognition.

[0038] In addition, the accuracy of position detection of the analysis target by AI recognition is generally lower than the accuracy required for vibration analysis. Therefore, in this embodiment, the vibration analysis device 1 transmits the position and identification information of the analysis target recognized by the analysis target recognition unit 212 to the vibration analysis unit 213 (step S6), and the vibration analysis unit 213 performs detailed vibration analysis.

[0039] Next, as a vibration analysis step, the vibration analysis unit 213 sets a region of interest for each analysis target based on the information of the analysis target received from the analysis target recognition unit 212 (step S7). That is, when the vibration analysis unit 213 receives analysis target information from the analysis target recognition unit 212, it updates the region of interest, and when it does not receive analysis target information from the analysis target recognition unit 212, it uses the region of interest that has already been set. At the start of the analysis, the region of interest for each analysis target used for creating the template in the initial setting is set as the region of interest for each analysis target in the vibration analysis unit 213.

[0040] The vibration analysis unit 213 detects the second frame rate F t Then, the displacement of the analysis object is calculated by using template matching for the region of interest of each analysis object that has been set (step S8). More specifically, the vibration analysis unit 213 performs template matching between an image of the previous time step of the second time series image and an image of the newly captured current time step, and calculates the displacement of the analysis object. In the frame at the start of capturing, template matching is performed between the initial template of step S2 and the image of the current time step. The method of image matching in step S8 is not particularly limited, and for example, phase only correlation (POC) can be used. This makes it possible to detect the displacement of the analysis object with high accuracy at the sub-pixel level.

[0041] The vibration analysis unit 213 stores the time change of the position of each analysis target calculated in step S8 in the storage unit 22 as vibration information of the analysis target (step S9). The vibration analysis unit 213 may also display the vibration information on the display unit 23. In this case, the vibration analysis unit 213 may apply FFT (Fast Fourier Transform) to the acquired vibration information to display information on the vibration frequency together with the displacement amount of each analysis target.

[0042] Thereafter, the vibration analysis device 1 performs the vibration analysis in steps S3 to S9 until the vibration analysis is completed (NO in step S10). When a termination condition is satisfied, such as the completion of a predetermined analysis time set at the start of the vibration analysis or input of an instruction to terminate the vibration analysis by the user (YES in step S10), the vibration analysis device 1 terminates the vibration analysis.

[0043] 5(A) to (C) are diagrams showing examples of vibration analysis results according to this embodiment. As shown in the graph of the time change of the displacement amount in FIG. 5(A) and the graph of the time change of the vibration frequency in FIG. 5(B), it can be seen that the vibration state of each of the markers, which are analysis objects, attached to the vibrating iron plate at 20 locations can be acquired. More specifically, even if the magnitude of the amplitude differs depending on the marker position, the vibration characteristics of the iron plate, etc., and it is difficult to predict the amplitude, it is possible to acquire the vibration state of each analysis object. In addition, as shown in FIG. 5(C), which is an enlarged view of the graph of one of the markers in FIG. 5(A) and (B), it can be seen that the vibration state of each analysis object, such as the time change of the three-dimensional displacement amount and the time change of the vibration frequency, can be appropriately analyzed.

[0044] As described above, the vibration analysis device 1 according to this embodiment detects the analysis target by performing image recognition on the entire captured image using a machine learning technique at a low sampling rate, so that even if the amplitude of the analysis target is unknown, the analysis target can be appropriately tracked and vibration analyzed. Moreover, the vibration analysis device 1 according to this embodiment analyzes the vibration of the analysis target by template matching at a high sampling rate, so that it is possible to perform vibration analysis with high accuracy for each analysis target included in the captured image.

[0045] In the present embodiment, the first time-series image used for recognizing the analysis target and the second time-series image used for vibration analysis are the same image, i.e., the first time-series image is a thinned image of the second time-series image, but this is not limited. For example, the first time-series image and the second time-series image may be images taken at different times. Furthermore, as long as the correlation between the positions (coordinates) of the analysis target to be recognized is clear, the first time-series image and the second time-series image may be images taken by different cameras.

[0046] In this embodiment, the vibration analysis unit 213 performs template matching between an image at the immediately previous time step and a newly captured image at the current time step to calculate the amount of displacement of the analysis target, but this is not limited to this. For example, template matching may be performed between an image of the first frame, i.e., an image at the start of analysis, and an image at the current time step to calculate the amount of displacement of the analysis target.

[0047] In addition, in the present embodiment, the sampling rate of the first time-series image is constant, but this is not limiting, and the sampling rate may be adjusted based on the amplitude (displacement) of each analysis target. In this case, for example, if the largest amplitude of each analysis target is 50% or more of the size of a preset region of interest, the sampling rate of the first time-series image may be increased, and if it is 10% or less, the sampling rate of the first time-series image may be decreased. This reduces the risk that the analysis target will fall outside the region of interest, thereby improving the accuracy of vibration analysis. [Industrial Applicability]

[0048] The present invention is suitable for vibration analysis of an object in which it is difficult to directly measure the vibration using a pickup, etc. In particular, the present invention is suitable for vibration analysis in which the magnitude (amplitude) of the vibration to be analyzed is unknown. [Explanation of symbols]

[0049] 1 vibration analysis device, 10 camera unit, 11R right camera, 11L left camera, 20 control unit, 21 control section, 211 image data acquisition section, 212 analysis target recognition section, 213 vibration analysis section, 22 storage section, 23 display section, 24 input section

Claims

1. an analysis target recognition unit that recognizes a plurality of analysis targets from a first time-series image including the analysis targets by a machine learning technique; a vibration analysis unit that calculates a vibration state of an analysis object by template matching based on a second time-series image having a higher sampling rate than the first time-series image and the coordinates of the analysis object recognized by the analysis object recognition unit, The sampling rate of the first time-series image is adjusted based on the largest amplitude among the amplitudes of the plurality of analysis targets. A vibration analysis device characterized by:

2. the first time-series image is a thinned image of the second time-series image; 2. The vibration analysis device according to claim 1.

3. The coordinates of the analysis target are three-dimensional coordinates calculated from a plurality of second time-series images captured from different viewpoints.

3. The vibration analysis device according to claim 1 or 2.

4. an analysis target recognition step of recognizing a plurality of analysis targets from a first time-series image including the analysis targets by a machine learning technique; a vibration analysis step of calculating a vibration state of the analysis object by template matching based on a second time-series image having a higher sampling rate than the first time-series image and the coordinates of the analysis object recognized in the analysis object recognition step, The sampling rate of the first time-series image is adjusted based on the largest amplitude among the amplitudes of the plurality of analysis targets. A vibration analysis method comprising:

5. Computer, an analysis target recognition unit that recognizes a plurality of analysis targets from a first time-series image including the analysis targets by a machine learning technique; a vibration analysis unit that calculates a vibration state of an analysis object by template matching based on a second time-series image having a higher sampling rate than the first time-series image and the coordinates of the analysis object recognized by the analysis object recognition unit, and adjusting a sampling rate of the first time-series images based on the largest amplitude among the amplitudes of the plurality of analysis targets; program.

Images