Vibration measurement method, system, computer device and storage medium

Abstract

The present invention relates to a vibration measurement method, system, computer equipment, and storage medium. A target attached to a measured object is photographed using an area array camera, and two columns or rows of one-dimensional images are selected from the image on the viewing plane to obtain a first straight line and a second straight line; the positions of the first projected straight line and the second projected straight line in the target coordinate system and the viewing plane coordinate system are obtained; at least three target points that are not on the same straight line are selected from the first projected straight line and the second projected straight line, and the rotation matrix and translation vector from the viewing plane coordinate system to the target coordinate system are calculated based on the coordinates of the target points in the target coordinate system and the viewing plane coordinate system. The present invention can complete vibration measurement using a single area array camera, and the measurement method is simple and convenient for practical application. The present invention collects two columns or two rows of one-dimensional images on the viewing plane of the area array camera image sensor as raw data, greatly reducing the actual data reading amount, thereby increasing the data acquisition frequency.

Description

Technical Field

[0001] The present invention relates to the technical field of video vibration measurement, and in particular to a vibration measurement method, system, computer equipment and storage medium. Background Art

[0002] Video vibrometer technology is a non-contact vibration measurement method based on white light imaging technology. This method uses a CCD / CMOS camera to continuously capture an image sequence of the object to be measured (usually with a special target attached), and then uses an image analysis algorithm to obtain the vibration information of the target.

[0003] The key to video vibrometer technology is the acquisition and analysis of target images. Currently, most video vibrometer systems require an area array camera as the target image acquisition device.

[0004] Existing area array video vibrometer systems typically require two area array cameras to obtain 3D vibration parameters. Based on the principle of dual-viewing vision, both cameras must be calibrated for internal and external parameters before testing. These calibrated area array cameras capture two 2D image sequences of the target in the measured area. Based on the principle of stereo vision, the 3D motion of the measured point is reconstructed to produce a vibration curve.

[0005] However, existing area array camera-based vibrometer systems use two-dimensional images as raw data. The large amount of data from these two-dimensional images limits the data acquisition rate of these systems. Furthermore, the need for two high-speed cameras makes testing very expensive, and the large size and weight of the equipment also significantly hinder their practical application. Therefore, current area array camera-based video vibrometer systems cannot meet the needs of high-speed vibration measurement. Summary of the Invention

[0006] Based on this, it is necessary to provide a vibration measurement method, system, computer equipment and storage medium to address the problems of high cost and inconvenience in testing two high-speed cameras.

[0007] A vibration measurement method comprises the following steps:

[0008] Using an area array camera to photograph a target attached to the object to be measured, collecting an image on a viewing plane of an image sensor of the area array camera, selecting two columns or two rows of one-dimensional images from the image on the viewing plane, and obtaining a first straight line and a second straight line based on the two columns or two rows of one-dimensional images;

[0009] The projection of the first straight line through the optical center of the area array camera intersects the target plane at the first projection straight line, and the projection of the second straight line through the optical center of the area array camera intersects the target plane at the second projection straight line;

[0010] Establishing a viewing plane coordinate system on the viewing plane, establishing a target coordinate system on the target plane, and obtaining positions of the first projection line and the second projection line in the target coordinate system;

[0011] Get the positions of the first projection line and the second projection line in the viewing plane coordinate system;

[0012] Take at least three target points that are not on the same straight line from the first projection line and the second projection line, and calculate the rotation matrix and translation vector from the viewing plane coordinate system to the target coordinate system based on the coordinates of the target points in the target coordinate system and the viewing plane coordinate system.

[0013] The above method can complete vibration measurement using a single area array camera. Only one area array camera needs to be calibrated. The measurement method is simple and easy to use in practice. At the same time, the above method collects two columns or two rows of one-dimensional images on the viewing plane of the area array camera image sensor as raw data, which greatly reduces the actual data reading amount and thus improves the data collection frequency.

[0014] In one embodiment, selecting two columns or two rows of one-dimensional images from an image on a viewing plane, and obtaining a first straight line and a second straight line based on the two columns or two rows of one-dimensional images includes:

[0015] Select two columns of one-dimensional images from the image on the viewing plane, and if the width of the one-dimensional image is greater than one pixel unit, fit the two columns of one-dimensional images respectively to obtain a first straight line and a second straight line;

[0016] or;

[0017] Two rows of one-dimensional images are selected from the image on the viewing plane. If the width of the one-dimensional image is greater than one pixel unit, the two rows of one-dimensional images are fitted respectively to obtain a first straight line and a second straight line.

[0018] In one embodiment, obtaining positions of the first projection line and the second projection line in the target coordinate system includes:

[0019] Steps to obtain the position of the first projection line in the target coordinate system:

[0020] Obtaining coordinates of intersection points of the first straight line and the target image on the viewing plane in the viewing plane coordinate system, where there are at least five intersection points of the first straight line and the target image on the viewing plane;

[0021] Obtaining the geometric relationship of the target;

[0022] According to the coordinates of the intersection points of the first straight line and the target image on the viewing plane in the viewing plane coordinate system, combined with the geometric relationship of the target, the coordinates of the intersection points of the first projected straight line and the target in the target coordinate system are obtained, and there are at least five intersection points between the first projected straight line and the target;

[0023] Obtaining a position of the first projection line in the target coordinate system according to the coordinates of the intersection of the first projection line and the target in the target coordinate system;

[0024] The step of obtaining the position of the second projection straight line in the target coordinate system corresponds to the step of obtaining the position of the first projection straight line in the target coordinate system.

[0025] In one embodiment, obtaining positions of the first projection line and the second projection line in the viewing plane coordinate system includes:

[0026] Steps to obtain the position of the first projection line in the view plane coordinate system:

[0027] Get the intrinsic parameters of the area array camera;

[0028] Combining the coordinates of the intersection point of the first straight line and the target image on the viewing plane in the viewing plane coordinate system, the coordinates of the intersection point of the first projection straight line and the target in the target coordinate system, and the intrinsic parameters of the area array camera, the coordinates of the intersection point of the first projection straight line and the target in the viewing plane coordinate system are obtained;

[0029] Obtaining a position of the first projection line in the viewing plane coordinate system according to the coordinates of the intersection of the first projection line and the target in the viewing plane coordinate system;

[0030] The step of obtaining the position of the second projection straight line in the viewing plane coordinate system corresponds to the step of obtaining the position of the second projection straight line in the viewing plane coordinate system.

[0031] In one embodiment, the target includes at least five target lines, wherein at least three target lines intersect.

[0032] A system for implementing the above-mentioned vibration measurement method comprises:

[0033] A target attached to the object being measured, an area array camera arranged opposite the target, and a control center;

[0034] Area scan cameras have multi-ROI capabilities;

[0035] The control center is connected to the area array camera to set the area array camera parameters, receive area array camera data, and obtain vibration measurement results.

[0036] The above system uses an area array camera with multi-ROI function. When acquiring images, the camera only needs to collect two (rows) and columns of data. The data volume is three orders of magnitude lower than the original two-dimensional image. Therefore, in theory, the data acquisition speed can be increased by three orders of magnitude.

[0037] In one embodiment, the target includes a plurality of straight target lines and / or curved target lines.

[0038] In one embodiment, the target is a five-line target, including a first target line, a second target line, a third target line, a fourth target line and a fifth target line, wherein the first target line, the second target line, the third target line, the fourth target line and the fifth target line are straight lines, wherein the first target line, the third target line and the fifth target line are parallel to each other, and the second target line, the third target line and the fourth target line intersect at a point.

[0039] In one embodiment, the target is a five-line target, including a first target line, a second target line, a third target line, a fourth target line and a fifth target line. The first target line, the second target line and the third target line are straight lines, and the first target line, the second target line and the third target line intersect at a point. The fourth target line and the fifth target line are curved lines, and the fourth target line and the fifth target line intersect with the first target line, the second target line and the third target line respectively.

[0040] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the vibration measurement method described above is implemented.

[0041] A computer storage medium stores a computer program, which implements the above-mentioned vibration measurement method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a schematic diagram of the principle of the binocular vibration measurement system;

[0043] Figure 2 This is a schematic diagram of the measurement results of the binocular vibrometer system;

[0044] Figure 3 is a flow chart of a vibration measurement method in one embodiment;

[0045] Figure 4 is a schematic diagram of a five-line target in one embodiment;

[0046] Figure 5 is a schematic diagram of a five-line target in one embodiment;

[0047] Figure 6 is a schematic diagram of capturing an image on a viewing plane of an area array camera image sensor in one embodiment;

[0048] Figure 7 In one embodiment, the straight line L R Schematic diagram of the position on the five-line target plane;

[0049] Figure 8 In one embodiment, the straight line L R Schematic diagram of the position on the viewing plane of the area array camera image sensor;

[0050] Figure 9 In one embodiment, the straight line LI Schematic diagram of the position on the five-line target plane;

[0051] Figure 10 In one embodiment, the straight line L I Schematic diagram of the position on the viewing plane of the area array camera image sensor;

[0052] Figure 11 6-DOF pose information experimentally measured in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0054] It should be noted that the terms "first, second, and third" used in the embodiments of the present invention are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that the terms "first, second, and third" may interchangeably represent a specific order or precedence, where permitted. It should be understood that the terms "first, second, and third" may interchangeably represent objects, where appropriate, such that the embodiments of the present invention described herein may be implemented in an order other than that illustrated or described herein.

[0055] The terms "including" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or (module) units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus.

[0056] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0057] In this document, "plurality" refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0058] For binocular vibrometer systems, the key steps of measurement are the acquisition and analysis of target images. Currently, most video vibrometer systems use array cameras as image acquisition devices. The principle of array video vibrometer is as follows: Figure 1 As shown in the figure, two area array cameras with calibrated internal and external parameters are used to obtain two two-dimensional image sequences of the target on the measured area. According to the principle of stereo vision, the three-dimensional motion of the measured point is reconstructed, and the vibration curve is obtained as shown in the figure. Figure 2 shown.

[0059] The final data acquisition frequency of the video vibrometer system depends on the image acquisition speed of the area array camera, and the image acquisition speed of the area array camera generally depends on the speed of image data transmission between the camera and the computer, or within the camera.

[0060] For common digital cameras, after completing photoelectric conversion, the sensor must store image data in real time via a data interface to a computer hard drive. Therefore, the image acquisition frequency of these cameras is primarily determined by the data interface's transmission speed limitations. Common data interfaces include 1394a / b, GigE, USB 3.0, and Ethernet. Due to these interface transmission speed limitations, the acquisition parameters for cameras currently transmitting images via USB 3.0 are shown in Table 1. As can be seen, the acquisition frame rate of area scan cameras is generally low and decreases dramatically as the captured image resolution increases.

[0061] Table 1. Collection parameters of a series of PointGray cameras

[0062]

[0063] For example, when acquiring images with a high-speed camera, image data must be transferred and stored in the camera's internal high-speed memory during the acquisition process. After acquisition is complete, the image data is transferred and stored on the computer's hard drive. Therefore, the acquisition frame rate of a high-speed camera is primarily determined by the limitations of the camera's internal transmission bandwidth. For example, at full resolution (1024*1024 pixels), the Photron SA5 high-speed camera can achieve a frame rate of up to 7000 fps (data acquisition speed of approximately 7GB / s). However, the transfer speed from the camera's internal memory to the computer's hard drive is generally slow. The Photron SA5 uses a Gigabit Ethernet port (transmission speed of 12.5MB / s) as its data transmission interface. Due to the limited transmission bandwidth between the camera's internal memory and the hard drive, the image data from the high-speed camera takes a long time to be transferred to the computer, making real-time image processing and analysis impossible.

[0064] In addition, high-speed cameras have limited memory capacity, which results in shorter measurement times. The Photron SA5 high-speed camera has 10GB of memory. When the image acquisition resolution is 1024*1024 and the acquisition frame rate is 7000fps, the measurement time is only 1.33s, which is not sufficient for vibration testing conditions. According to experience in actual vibration measurements, the data acquisition frequency should generally be higher than 10 times the measured vibration frequency. Based on this rule, video vibration measurement with an ordinary digital camera can only be used to measure the motion of a simple pendulum (approximately 3 times per second); while video vibration measurement with a high-speed camera can be used to measure the rotation of a high-speed galvanometer (approximately 700 times per second), but the vibration measurement time is very short, only 1.33 seconds.

[0065] Measuring three-dimensional vibration parameters requires two high-speed cameras, which is very expensive. The equipment is also bulky and heavy, making it very inconvenient for practical vibration measurement applications. Therefore, current video vibrometer systems based on area array cameras cannot meet the needs of high-speed vibration measurement.

[0066] This embodiment provides a vibration measurement method, such as Figure 3 , including the following steps:

[0067] Step 100: photographing a target attached to the object under test using an area array camera, acquiring an image on a viewing plane of an image sensor of the area array camera, selecting two columns or two rows of one-dimensional images from the image on the viewing plane, and obtaining a first straight line and a second straight line based on the two columns or two rows of one-dimensional images;

[0068] Step 200: The projection of the first straight line through the optical center of the area array camera intersects the target plane at the first projection straight line, and the projection of the second straight line through the optical center of the area array camera intersects the target plane at the second projection straight line;

[0069] Establishing a viewing plane coordinate system on the viewing plane, establishing a target coordinate system on the target plane, and obtaining positions of the first projection line and the second projection line in the target coordinate system;

[0070] Step 300: Obtain the positions of the first projection line and the second projection line in the viewing plane coordinate system;

[0071] Step 400: Take at least three target points that are not on the same straight line from the first projection line and the second projection line, and calculate the rotation matrix and translation vector from the viewing plane coordinate system to the target coordinate system based on the coordinates of the target points in the target coordinate system and the viewing plane coordinate system.

[0072] Conventional 3D video vibrometers based on area array cameras require at least two area array cameras, and the measurement system must be jointly calibrated during system deployment. However, the above method can complete vibration measurement using a single area array camera, requiring only the calibration of one area array camera. This method is simple and easy to use in practice.

[0073] At the same time, the above method collects two columns or two rows of one-dimensional images on the viewing plane of the area array camera image sensor as raw data, which greatly reduces the actual data reading amount and thus improves the data collection frequency.

[0074] In one embodiment, selecting two columns or two rows of one-dimensional images from an image on a viewing plane, and obtaining a first straight line and a second straight line based on the two columns or two rows of one-dimensional images includes:

[0075] Select two columns of one-dimensional images from the image on the viewing plane, and if the width of the one-dimensional image is greater than one pixel unit, fit the two columns of one-dimensional images respectively to obtain a first straight line and a second straight line;

[0076] or;

[0077] Two rows of one-dimensional images are selected from the image on the viewing plane. If the width of the one-dimensional image is greater than one pixel unit, the two rows of one-dimensional images are fitted respectively to obtain a first straight line and a second straight line.

[0078] Furthermore, if the width of the one-dimensional image is greater than one pixel unit, two columns or two rows of one-dimensional images are fitted separately. The intersection of the one-dimensional image and the target can be determined first. Since the width is greater than one pixel unit, the intersection itself also contains multiple pixel units, and the position of the intersection can be fitted out, such as selecting the position of the most central pixel unit as the intersection, or the position with the maximum or minimum pixel grayscale as the intersection. There is no restriction on the fitting selection here; the one-dimensional image and the target can have multiple intersections. After obtaining the fitted intersection, the multiple fitted intersections are fitted again to obtain the first straight line and the second straight line. The method of fitting the straight lines can be implemented by the least squares method, etc., and the implementation method is not restricted here.

[0079] Specifically, a region of interest (ROI) is set for the area array camera, and two columns or two rows of one-dimensional images on a viewing plane of the area array camera image sensor are collected.

[0080] In one embodiment, in step 200, obtaining the positions of the first projection line and the second projection line in the target coordinate system includes:

[0081] Step 210: Obtaining the position of the first projection line in the target coordinate system, including steps 211-214;

[0082] Step 211: Obtaining coordinates of intersection points of the first straight line and the target image on the viewing plane in the viewing plane coordinate system, where there are at least five intersection points of the first straight line and the target image on the viewing plane;

[0083] Step 212: Obtaining the geometric relationship of the target;

[0084] Step 213: Based on the coordinates of the intersection points of the first straight line and the target image on the viewing plane in the viewing plane coordinate system and in combination with the geometric relationship of the target, the coordinates of the intersection points of the first projected straight line and the target in the target coordinate system are obtained. There are at least five intersection points between the first projected straight line and the target.

[0085] Step 214: Obtaining the position of the first projection line in the target coordinate system according to the coordinates of the intersection of the first projection line and the target in the target coordinate system;

[0086] Step 220: Obtain the position of the second projection line in the target coordinate system; this corresponds to the step of obtaining the position of the first projection line in the target coordinate system.

[0087] Specifically, in step 211, the coordinates of the intersection of the first straight line and the target image on the viewing plane in the viewing plane coordinate system are obtained, and the steps include: comparing the pixel value of the point on the viewing plane through which the first straight line passes with the set threshold value, and if the pixel value is less than the set threshold value or the pixel value is greater than the set threshold value, then the point is judged to be the intersection point.

[0088] Specifically, in step 213, assuming that the intersection point of the first straight line and the target image on the viewing plane is P, and the corresponding intersection point of the first projection straight line and the target is Q, according to the similarity principle and the invariant intersection ratio theorem, based on the geometric relationship, the coordinates of the intersection point of the first projection straight line and the target in the target coordinate system can be obtained.

[0089] In one embodiment, in step 300, obtaining the positions of the first projection line and the second projection line in the viewing plane coordinate system includes:

[0090] Step 310: Obtain the position of the first projection line in the viewing plane coordinate system, including steps 311-313;

[0091] Step 311: Obtaining the intrinsic parameters of the area array camera;

[0092] Step 312: Combining the coordinates of the intersection point of the first straight line and the target image on the viewing plane in the viewing plane coordinate system, the coordinates of the intersection point of the first projection straight line and the target in the target coordinate system, and the intrinsic parameters of the area array camera, obtain the coordinates of the intersection point of the first projection straight line and the target in the viewing plane coordinate system;

[0093] Step 313: Obtain the position of the first projection line in the viewing plane coordinate system according to the coordinates of the intersection point of the first projection line and the target in the viewing plane coordinate system;

[0094] Step 320: Obtain the position of the second projection line in the viewing plane coordinate system; this corresponds to the step of obtaining the position of the first projection line in the viewing plane coordinate system.

[0095] Specifically, the intrinsic parameters of the area array camera include the coordinates of the principal point and the equivalent focal length of the area array camera.

[0096] Furthermore, the target may include at least five target lines, wherein at least three target lines intersect, which can better reflect the geometric relationship of the target. When the number of target lines is greater than five, the solution is not unique during calculation and can be optimized through an algorithm.

[0097] Specifically, in step 312, the coordinates of the intersection point of the first straight line and the target image on the viewing plane in the viewing plane coordinate system, the coordinates of the intersection point of the first projection straight line and the target in the target coordinate system, and the intrinsic parameters of the area array camera are combined, according to the similarity principle and the cross ratio invariance theorem, and based on the geometric relationship, the coordinates of the intersection point of the first projection straight line and the target in the viewing plane coordinate system are obtained.

[0098] In one embodiment, at least three target points that are not on the same straight line are selected from the first projection line and the second projection line: the first point, the second point and the third point, and the target points are selected according to their positions in the target coordinate system X. B Y B Z B and the viewing plane coordinate system X C Y C Z C The coordinates in the view plane coordinate system X are calculated as follows C Y C Z C To target coordinate system X B Y B Z B The rotation matrix R and translation vector T are:

[0099]

[0100] R8 = R0R4 - R1R3,

[0101] Among them, r0, r1, r2, r3, r4, r5, r6, r7 and r8 are the parameters of the rotation matrix R, T X 、T y and T z is the parameter of the translation vector T, X B1 and Y B1 The first point taken from the first and second projection lines is in the coordinate system X B Y B Z B The X-axis coordinate and the Y-axis coordinate, X B2 and Y B2 The second point taken from the first and second projection lines is in the coordinate system X B Y B Z B The X-axis coordinate and Y-axis coordinate in B3 and Y B3The third point taken from the first and second projection lines is in the coordinate system X B Y B Z B The corresponding X-axis coordinates and Y-axis coordinates in C1 、Y C1 and Z C1 The first point taken from the first and second projection lines is in the coordinate system X C Y C Z C The X-axis coordinate, Y-axis coordinate and Z-axis coordinate in C2 、Y C2 and Z C2 The second point taken from the first and second projection lines is in the coordinate system X C Y C Z C The X-axis coordinate, Y-axis coordinate and Z-axis coordinate, X C3 、Y C3 and Z C3 The third point taken from the first and second projection lines is in the coordinate system X C Y C Z C The X-axis coordinate, Y-axis coordinate, and Z-axis coordinate in .

[0102] This embodiment provides a vibration measurement system, including:

[0103] A target attached to the object being measured, an area array camera arranged opposite the target, and a control center;

[0104] Area scan cameras have multi-ROI capabilities;

[0105] The control center is connected to the area array camera to set the area array camera parameters, receive area array camera data, and obtain vibration measurement results.

[0106] This system employs a novel measurement principle to acquire the target's motion parameters. Using an area array camera with multi-ROI functionality as the image acquisition device, two columns (rows) of image elements are activated during measurement. Combined with a planar target, this system can measure high-speed vibrating objects using a single area array camera. Since the camera only needs to capture two (rows) of data during image acquisition, the data volume is three orders of magnitude lower than that of the original two-dimensional image. Therefore, theoretically, acquisition speed can be increased by three orders of magnitude. However, in practice, considering the image signal-to-noise ratio and the feasibility of acquisition circuit and algorithm design, image acquisition speed can be increased by two orders of magnitude.

[0107] In this system, the shape design of the target is not unique.

[0108] In one embodiment, the target includes a plurality of straight target lines and / or curved target lines.

[0109] In one embodiment, the target is a five-line target, including a first target line, a second target line, a third target line, a fourth target line and a fifth target line, wherein the first target line, the second target line, the third target line, the fourth target line and the fifth target line are straight lines, wherein the first target line, the third target line and the fifth target line are parallel to each other, and the second target line, the third target line and the fourth target line intersect at a point.

[0110] Specifically, such as Figure 4 The target is a five-line target, including a first target line L1, a second target line L2, a third target line L3, a fourth target line L4 and a fifth target line L5, wherein the first target line L1, the third target line L3 and the fifth target line L5 are parallel to each other, the distance between the first target line L1 and the third target line L3 is U, the distance between the third target line L3 and the fifth target line L5 is V, the second target line L2, the third target line L3 and the fourth target line L4 intersect at point O, the angle between the second target line L2 and the third target line L3 is α, and the angle between the third target line L2 and the fourth target line L3 is β.

[0111] In one embodiment, the target is a five-line target, including a first target line, a second target line, a third target line, a fourth target line and a fifth target line. The first target line, the second target line and the third target line are straight lines, and the first target line, the second target line and the third target line intersect at a point. The fourth target line and the fifth target line are curved lines, and the fourth target line and the fifth target line intersect with the first target line, the second target line and the third target line respectively.

[0112] Specifically, such as Figure 5 The target is a five-line target, including a first target line L1, a second target line L2, a third target line L3, a fourth target line L4 and a fifth target line L5, wherein the first target line L1, the second target line L2 and the third target line L3 intersect at point O, the angle between the first target line L1 and the second target line L2 is α, and the angle between the second target line L2 and the third target line L3 is β; the fourth target line L4 and the fifth target line L5 are parallel curves, and the fourth target line L4 and the fifth target line L5 intersect with the first target line L1, the second target line L2 and the third target line L3 respectively.

[0113] like Figure 4 、 Figure 5 Both five-line targets shown in the figure can be used for posture measurement. Figure 4 Taking the straight line target shown as an example, the measurement process is described in detail:

[0114] The target contains five lines, L1-L5. L1, L3, and L5 are parallel to each other with known distances U and V. L2, L3, and L4 intersect at point O with known angles α and β. By attaching the linear target to the vibration plate under test, the six-degree-of-freedom spatial pose parameters (rotation matrix R, translation vector T) of the object under test can be determined from the two columns of target image data captured by the camera.

[0115] like Figure 6 As shown, the target is first attached to the object to be tested.

[0116] Collect images on the visual plane of the area array camera image sensor: Set the region of interest (ROI) on the area array camera and use one area array camera to collect two columns or two rows of one-dimensional images on the visual plane of the area array camera image sensor. The one-dimensional images include the first image C1 and the second image C2. The line segment formed by the two endpoints of the first image C1 through the optical center O' intersects the five-line target plane at the straight line L. R The line segment formed by the two endpoints of the second image C2 through the optical center O' intersects the five-line target plane at the straight line L I , each area array camera lens corresponds to an optical center O′.

[0117] Analyze the first image C1 and the second image C2 to obtain the six-degree-of-freedom spatial pose parameters of the object to be measured to complete the vibration measurement. The six-degree-of-freedom spatial pose parameters include the rotation matrix R and the translation vector T, where:

[0118] Establish coordinate system X on the plane of the five-line target B Y B Z B , use the cross ratio invariance theorem on the viewing plane formed by the ROI setting of the area array camera and the five-line target plane to determine the straight line L I and L R In coordinate system X B Y B Z B Position in

[0119] Establish coordinate system X on the viewing plane C Y C Z C , using the pre-calibrated intrinsic parameters of the array camera and the straight line L I and L R The coordinates of the intersection point with the five-line target are calculated by geometric relationship. I and L R In coordinate system X C Y C Z C The pre-calibrated intrinsic parameters of the area array camera include the coordinates of the principal point and the equivalent focal length of the area array camera;

[0120] From the straight line L I and L R Take at least three points that are not on the same line: the first point, the second point and the third point, according to their coordinate system X B Y B Z B and coordinate system X C Y C ZC The coordinates in the coordinate system X are calculated as follows C Y C Z C To coordinate system X B Y B Z B The rotation matrix R and translation vector T are:

[0121]

[0122] r8=r0r4-r1r3

[0123] Among them, r0, r1, r2, r3, r4, r5, r6, r7 and r8 are the parameters of the rotation matrix R, T X 、T y and T z is the parameter of the translation vector T, X B1 and Y B1 From the straight line L I and L R The first point taken out in the coordinate system X B Y B Z B The X-axis coordinate and the Y-axis coordinate, X B2 and Y B2 From the straight line L I and L R The second point taken out is in the coordinate system X B Y B Z B The X-axis coordinate and Y-axis coordinate in B3 and Y B3 From the straight line L I and L R The third point taken out in the coordinate system X B Y B Z B The corresponding X-axis coordinates and Y-axis coordinates in C1 、Y C1 and Z C1 From the straight line L I and L R The first point taken out in the coordinate system X C Y C Z C The X-axis coordinate, Y-axis coordinate and Z-axis coordinate in C2 、Y C2 and Z C2 From the straight line L I and L R The second point taken out is in the coordinate system X C Y C Z CThe X-axis coordinate, Y-axis coordinate and Z-axis coordinate, X C3 、Y C3 and Z C3 From the straight line L I and L R The third point taken out in the coordinate system X C Y C Z C The X-axis coordinate, Y-axis coordinate, and Z-axis coordinate in .

[0124] In one embodiment, Figure 7 , use the cross ratio invariance theorem on the area array camera viewing plane and the target plane to determine the straight line L R In coordinate system X B Y B Z B The specific steps include:

[0125] Obtain the intersection points a, b, c, d, and e of the first image C1 and the second image C2 with the target image on the viewing plane of the area array camera. If, in actual operation, the first image C1 and the second image C2 are obtained as a single column or a single row of pixels, such as 2000*1 or 1*2000, then calculate the coordinates of the intersection point.

[0126] If the first image C1 and the second image C2 obtained in actual operation are not a single column or a single row of pixels, but multiple columns or rows of pixels, such as 2000*10 or 10*2000, then each intersection has more than ten pixels. At this time, the coordinates of the intersection to be found are obtained based on fitting, for example, the position with the maximum or minimum grayscale is the intersection.

[0127] By applying the cross ratio invariance theorem twice on the visual plane and the five-line target plane, the straight line L R The first target line L1, the second target line L2, the third target line L3, the fourth target line L4 and the fifth target line L5 on the five-line target plane intersect at points A, B, C, D and E. A, B, C, D and E are object points in the five-line target. a, b, c, d and e are image points on the viewing plane corresponding to A, B, C, D and E. The normal of the third target line L3 passing through point A intersects the third target line at point A'. The angle between the straight line OA and OB is γ, which is expressed as:

[0128]

[0129] Among them, the angle between the second target line L2 and the third target line L3 is α, and the angle between the third target line L3 and the fourth target line L4 is β. The angles α and β are both target structure parameters. CR is the intersection ratio of the four image points a, b, c, and d, which is obtained by analyzing the coordinates of the image points. The value of γ is calculated by the angle α, angle β, and CR, and then the distance D between points O and A' is calculated. OA' , the X-axis coordinate of point A is DOA' , the Y axis coordinate is -U, the Z axis coordinate is 0, and the calculation point A is in the coordinate system X B Y B Z B The coordinate method of the four points B, C, D, and E in the coordinate system X is used to calculate the coordinates of the four points B, C, D, and E in the coordinate system X B Y B Z B Then determine the coordinates of L R In coordinate system X B Y B Z B The position in.

[0130] Furthermore, we determine the straight line L I In X B Y B Z B Method of position in coordinate system and determination of straight line L R In coordinate system X B Y B Z B The same method is used for the position in .

[0131] In one embodiment, Figure 8 , using the pre-calibrated intrinsic parameters of the array camera and the straight line L I and L R The coordinates of the intersection point with the five-line target are used to calculate the straight line L through geometric relationships. R In coordinate system X C Y C Z C The specific steps include:

[0132] The distance D between the optical center O' and point A is calculated by geometric relationship O'A , and then we get point A in coordinate system X C Y C Z C The coordinates of point A in coordinate system X C Y C Z C The coordinates in are calculated as follows:

[0133]

[0134] Among them, l a Represents the distance from the image point a to the optical center O', and F is the equivalent focal length F X 、F Y The average value, C X 、C Y are the principal point coordinates of the array camera and the equivalent focal length F X 、F Y And the coordinates of the principal point of the array camera CX 、C Y Both can be obtained by pre-calibrating the array camera, x a is the image point a in the coordinate system X C Y C Z C The X-axis coordinate, y a is the image point a in the coordinate system X C Y C Z C The Y-axis coordinate of A in the coordinate system X is calculated by the above method. C Y C Z C The coordinates of the CA is the X-axis coordinate of coordinate A", Y CA is the Y-axis coordinate of coordinate A", Z CA is the Z-axis coordinate of coordinate A", and points B, C, D, and E are in the X-axis coordinate system. C Y C Z C The coordinates of A in the coordinate system X are obtained by C Y C Z C The coordinates of points B, C, D and E are calculated by the coordinate system X C Y C Z C The corresponding coordinate points are B”, C”, D’ and E”, and finally L is determined. R In the coordinate system X C Y C Z C The position in.

[0135] Specifically, a The calculation method is as follows:

[0136]

[0137] Specifically, the distance D between the optical center O' and point A O'A The calculation method is as follows:

[0138]

[0139] Where v is the angle between the straight line EO' and the straight line EA, D AE is the distance between point A and point E, passing through point A and point E in the coordinate system X B Y B Z B Calculate the coordinates in D AE , through D AE and ∠AO'E, calculate the distance D between the optical center O' and point A O'A .

[0140] Specifically, the calculation method of ∠AO'E is as follows:

[0141] ∠AO'E=∠AO'B+∠BO'C+∠CO'D+∠DO'E;

[0142] The calculation method of ∠AO'B is as follows:

[0143]

[0144] Where ∠AO`B is the angle between lines O`A and O`B, and r is the rth column of the viewing plane. The methods for calculating ∠BO'C, ∠CO'D, and ∠DO'E are the same as those for calculating ∠AO'B.

[0145] Specifically, the calculation method of the angle v is as follows:

[0146] v=π-∠AO'Eu

[0147] D O'B ×sin∠AO'B=D AB ×sin u

[0148] D O'B ×sin∠BO'E=D AD ×sin v

[0149] Where u is the angle between the straight line O'A and the straight line AE, ∠BO'E=∠BO'C+∠CO'D+∠DO'E, D AB is the distance between point A and point B, D AB Through point A and point B in coordinate system X B Y B Z B The coordinates in D are calculated. AD Through point A and point D in coordinate system X B Y B Z B The coordinates in are calculated.

[0150] Determine the straight line L I In coordinate system X C Y C Z C The method of determining the position of the straight line L R In coordinate system X C Y C Z C The same method is used for the position in .

[0151] In one embodiment, calculate B, C, D and E in coordinate system X B Y B Z B The coordinate method in is:

[0152] The normal line of the third target line L3 passing through point E intersects the third target line L3 at point E'. The angle between the straight line OD and OE is γ', which is expressed as:

[0153]

[0154] Among them, CR2 is the intersection ratio of the four image points b, c, d, and e, which is obtained by analyzing the coordinates of the image points. The value of γ' is calculated by the angle α, angle β and CR2, and then the distance D between points O and E' is calculated. OE’ , the X-axis coordinate of point E is D OE’ , the Y-axis coordinate is V, the Z-axis coordinate is 0, and the straight line L is calculated based on the coordinates of points A and E R According to the relationship between the straight line L R The coordinates of points B, C, and D can be calculated using the relationship.

[0155] In one embodiment, the method for calculating ∠BO'C, ∠CO'D and ∠DO'E is:

[0156]

[0157]

[0158]

[0159] Where ∠BO'C is the angle between O'B and O'C, ∠CO'D is the angle between O'C and O'D, ∠DO'E is the angle between O'D and O'E, r is the rth column on the viewing plane, y c is the image point c on X C Y C Z C The vertical coordinate, y d is the image of point d at X C Y C Z C The vertical coordinate, y e is the image of point e at X C Y C Z C The vertical coordinate of .

[0160] In one embodiment, Figure 9 , calculate the straight line L I In coordinate system X B Y B Z B The location method is:

[0161] By applying the cross ratio invariance theorem twice on the visual plane and the five-line target plane, the straight line L IThe first target line L1, the second target line L2, the third target line L3, the fourth target line L4 and the fifth target line L5 on the five-line target plane intersect at points F, G, H, I and J. F, G, H, I and J are object points in the five-line target. f, g, h, i and j are image points on the viewing plane corresponding to F, G, H, I and J. The normal of the third target line L3 passing through point F intersects the third target line at point F'. The angle between the straight lines OF and OG is γ", and the angle γ" is expressed as:

[0162]

[0163] The angle between the second target line L2 and the third target line L3 is α, and the angle between the third target line L3 and the fourth target line L4 is β. The angles α and β are both target structural parameters. CR3 is the intersection ratio of the four image points f, g, h, and i, which is obtained by analyzing the image point coordinates. The value of γ" is calculated by the angles α, β, and CR3, and then the distance D between points O and F' is calculated. OF' , the X-axis coordinate of point F is D OF' , the Y-axis coordinate is -U, and the Z-axis coordinate is 0;

[0164] The normal line of the third target line L3 passing through point J intersects the third target line L3 at point J'. The angle between the straight lines OI and OJ is γ'', and γ'' is expressed as:

[0165]

[0166] Among them, CR4 is the intersection ratio of the four image points g, h, i, and j, which is obtained by analyzing the coordinates of the image points. The value of γ"' is calculated by the angle α, angle β and CR4, and then the distance D between points O and J' is calculated. OJ' , the X-axis coordinate of point J is D OJ' , the Y-axis coordinate is V, the Z-axis coordinate is 0, and the straight line L is calculated based on the coordinates of points F and J I According to the relationship between the straight line L I The coordinates of points G, H, and I are calculated by the relationship. The coordinates of points F and G are used to determine L. I In the coordinate system X B Y B Z B The position in.

[0167] In one embodiment, Figure 10 , calculate the straight line L I In coordinate system X C Y C Z C The position method in is:

[0168] Calculate the point F in coordinate system X C Y C Z CCoordinates in:

[0169]

[0170]

[0171]

[0172] v'=π-∠FO'Gu'

[0173] D O'G ×sin∠FO'G=D FG ×sin u',

[0174] D O'G ×sin∠GO'J=D FI ×sin v'

[0175] ∠FO'J=∠FO'G+∠GO'H+∠HO'I+∠IO'J,

[0176]

[0177]

[0178]

[0179]

[0180] Among them, l f Represents the distance from the image point f to the optical center, and F is the equivalent focal length F X 、F Y The average value, C X 、C Y are the principal point coordinates of the array camera and the equivalent focal length F X 、F Y And the coordinates of the principal point of the array camera C X 、C Y Both can be obtained by pre-calibrating the array camera, y f is the image point f in coordinate system X C Y C Z C The vertical coordinate, y g is the image point g in the coordinate system X C Y C Z C The vertical coordinate, y j is the image point j in the coordinate system X C Y C Z C The vertical coordinate of the line JO' is v', the angle between the line JO' and the line JF, and u' is the angle between the line O'F and the line FJ.B Y B Z B The coordinates in and ∠FO'J are used to calculate the distance D between the optical center O' and point F. O'F , calculate F in the coordinate system X by the above method C Y C Z C The coordinates of the CF is the X-axis coordinate of coordinate F", Y CF is the Y-axis coordinate of coordinate F”, Z CF The coordinate F' is the Z-axis coordinate of point J' in the X-axis coordinate system. C Y C Z C The coordinate calculation method in is the same as the method for calculating the coordinates of point F”. The coordinates of points F” and J” are used to determine L I In the coordinate system X C Y C Z C The position in.

[0181] In one embodiment, the calculated point J" is in the coordinate system X C Y C Z C The coordinate positions in are as follows:

[0182] Point J is in the coordinate system X C Y C Z C The coordinates in are calculated as follows:

[0183]

[0184]

[0185]

[0186] v'=π-∠FO'Gu'

[0187] D O'G ×sin∠FO'G=D FG ×sin u',

[0188] D O'G ×sin∠GO'J=D FI ×sin v'

[0189] ∠FO'J=∠FO'G+∠GO'H+∠HO'I+∠IO'J,

[0190]

[0191]

[0192]

[0193]

[0194] Among them, l j Represents the distance from the image point j to the optical center, y f is the image point f in coordinate system X C Y C Z C The vertical coordinate, y g is the image point g in the coordinate system X C Y C Z C The vertical coordinate, y j is the image point j in the coordinate system X C Y C Z C The vertical coordinate of the coordinate system X passes through points I and J. B Y B Z B Calculate the coordinates in D IJ , through D IJ And ∠IO'J calculate the distance D between the optical center O' and point J O'J , calculate J in the coordinate system X by the above method C Y C Z C The coordinates of CJ is the X-axis coordinate of coordinate J", Y CJ is the Y-axis coordinate of coordinate J", Z CJ is the Z-axis coordinate of coordinate J".

[0195] Based on the examples described above, in one embodiment, a computer device is further provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the vibration measurement methods described in the above embodiments is implemented.

[0196] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. For example, in the embodiments of the present invention, the program can be stored in a storage medium of a computer system and executed by at least one processor in the computer system to implement the processes including the embodiments of the above-described sleep assistance methods. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0197] Accordingly, in one embodiment, a storage medium is further provided, on which a computer program is stored, wherein when the program is executed by a processor, any one of the vibration measurement methods in the above embodiments is implemented.

[0198] The above-mentioned embodiment achieves high-speed measurement using a low-speed camera. Most area array camera-based image acquisition systems use two-dimensional images as raw data. The large amount of data from these two-dimensional images limits the data acquisition rate of these systems. However, the measurement method proposed in the above-mentioned embodiment uses two columns of images as raw data (ultra-sparse images), significantly reducing the data volume and thus increasing the data acquisition frequency.

[0199] The above-mentioned embodiment achieves multi-DOF measurement using a monocular camera. To address the information loss after image dimensionality reduction, a five-line target is used. By leveraging the cross-ratio invariance theorem and spatial geometric relationships to supplement this information, the 6-DOF spatial pose parameters of the object being measured can be calculated from just two columns of image data.

[0200] In a specific embodiment, a Photon Focus MV1-D1312I camera was used. Its full-pixel resolution was 1312 x 1082 and its maximum frame rate was 108 fps. By using ROI settings to capture two lines of image data, reducing the resolution to 1312 x 2, the maximum frame rate reached 26,315 fps. After implementing the ROI settings, the image acquisition frame rate increased by 243 times.

[0201] The multi-DOF pose measurement of the monocular multi-ROI camera is verified: the target is fixed to the base by a spring. By applying a multi-DOF vibration excitation, the target vibrates in multiple degrees of freedom. The Photon focus camera is used to capture the target at high speed with a frame rate of 800fps. Through the above embodiment, the pose information of the target with 6 degrees of freedom is obtained, namely, translation in 3 directions (the three lines of curves on the left) and rotation in 3 directions (the three lines of curves on the right). The pose results are shown as follows: Figure 11 shown.

[0202] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0203] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A vibration measurement method, characterized in that: The following steps are involved: photographing a target attached to the object under test by an area array camera, collecting an image on a viewing plane of an image sensor of the area array camera, selecting two columns or two rows of one-dimensional images from the image on the viewing plane, and obtaining a first straight line and a second straight line based on the two columns or two rows of one-dimensional images; The projection of the first straight line through the optical center of the area array camera intersects the target plane at the first projection straight line, and the projection of the second straight line through the optical center of the area array camera intersects the target plane at the second projection straight line; Establishing a viewing plane coordinate system on the viewing plane, establishing a target coordinate system on the target plane, and obtaining positions of the first projection line and the second projection line in the target coordinate system; Obtaining positions of the first projection line and the second projection line in the viewing plane coordinate system; Take at least three target points that are not on the same straight line from the first projection line and the second projection line, and calculate the rotation matrix and translation vector from the viewing plane coordinate system to the target coordinate system based on the coordinates of the target points in the target coordinate system and the viewing plane coordinate system; Obtaining positions of the first projection line and the second projection line in the target coordinate system includes: Steps for obtaining the position of the first projection line in the target coordinate system: Obtaining coordinates of intersections of the first straight line and the target image on the viewing plane in a viewing plane coordinate system, where there are at least five intersections of the first straight line and the target image on the viewing plane; obtaining a geometric relationship of the target; Obtaining, based on the coordinates of the intersections of the first straight line and the target image on the viewing plane in the viewing plane coordinate system and in combination with the geometric relationship of the target, the coordinates of the intersections of the first projected straight line and the target in the target coordinate system, where the first projected straight line has at least five intersections with the target; Obtaining a position of the first projection line in the target coordinate system according to the coordinates of the intersection of the first projection line and the target in the target coordinate system; The step of obtaining the position of the second projection straight line in the target coordinate system corresponds to the step of obtaining the position of the first projection straight line in the target coordinate system; Obtaining positions of the first projection line and the second projection line in the viewing plane coordinate system includes: Steps for obtaining the position of the first projection line in the viewing plane coordinate system: Get the intrinsic parameters of the area array camera; Combining the coordinates of the intersection point of the first straight line and the target image on the viewing plane in the viewing plane coordinate system, the coordinates of the intersection point of the first projection straight line and the target in the target coordinate system, and the intrinsic parameters of the area array camera, to obtain the coordinates of the intersection point of the first projection straight line and the target in the viewing plane coordinate system; Obtaining a position of the first projection line in the viewing plane coordinate system according to the coordinates of the intersection of the first projection line and the target in the viewing plane coordinate system; The step of obtaining the position of the second projection straight line in the viewing plane coordinate system corresponds to the step of obtaining the position of the second projection straight line in the viewing plane coordinate system.

2. The vibration measurement method according to claim 1, wherein: Selecting two columns or two rows of one-dimensional images from the image on the viewing plane, and acquiring a first straight line and a second straight line according to the two columns or two rows of one-dimensional images, comprising: Selecting two columns of one-dimensional images from the image on the viewing plane, and if the width of the one-dimensional image is greater than one pixel unit, fitting the two columns of one-dimensional images respectively to obtain the first straight line and the second straight line; or; Two rows of one-dimensional images are selected from the image on the viewing plane. If the width of the one-dimensional image is greater than one pixel unit, the two rows of one-dimensional images are fitted respectively to obtain the first straight line and the second straight line.

3. The vibration measurement method according to claim 1 or 2, characterized in that: The target includes at least five target lines, wherein at least three target lines intersect.

4. A vibration measurement system for implementing the method according to any one of claims 1 to 3, characterized in that: include: A target attached to the object being measured, an area array camera arranged opposite the target, and a control center; The area array camera has a multi-ROI function; The control center is connected to the area array camera to receive data from the area array camera and obtain vibration measurement results.

5. The vibration measurement system according to claim 4, characterized in that The target includes a plurality of straight target lines and / or curved target lines.

6. The vibration measurement system according to claim 4, characterized in that The target is a five-line target, including a first target line, a second target line, a third target line, a fourth target line and a fifth target line. The first target line, the second target line, the third target line, the fourth target line and the fifth target line are straight lines, wherein the first target line, the third target line and the fifth target line are parallel to each other, and the second target line, the third target line and the fourth target line intersect at a point.

7. The vibration measurement system according to claim 4, characterized in that The target is a five-line target, including a first target line, a second target line, a third target line, a fourth target line and a fifth target line. The first target line, the second target line and the third target line are straight lines, and the first target line, the second target line and the third target line intersect at a point. The fourth target line and the fifth target line are curved lines, and the fourth target line and the fifth target line intersect with the first target line, the second target line and the third target line respectively.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the vibration measurement method according to any one of claims 1 to 3 when executing the computer program.

9. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the vibration measurement method according to any one of claims 1 to 3 is implemented.

Images