A method, apparatus, electronic device, and storage medium for measuring vibration frequency based on video phase.

Abstract

The present invention discloses a vibration frequency measurement method, device, electronic device, and storage medium based on video phase, belonging to the field of vibration frequency measurement technology. The method collects a video stream containing a target object, extracts a region of interest from the video stream, and crops it to generate a video stream to be measured; uses a pre-configured filter to filter the Y channel image to obtain a phase sequence for characterizing the vibration signal; sets transformation parameters according to the phase sequence, and uses linear frequency modulation transformation to obtain a spectrum corresponding to the phase sequence. The spectrum can be refined while the observation time is fixed, improving the frequency resolution and achieving high-precision vibration frequency measurement in a short time; compared to traditional visual vibration measurement, the method uses spatial phase information to extract vibration signals, has the characteristics of being resistant to illumination changes, image noise, and being sensitive to small vibrations, and can improve the accuracy of vibration measurement.

Description

Technical Field

[0001] This invention belongs to the field of vibration measurement technology based on video image processing. Specifically, it relates to a vibration frequency measurement method, device, electronic device, and storage medium based on video phase. Background Technology

[0002] Vibration is a ubiquitous phenomenon that reflects the intrinsic characteristics and operational state of objects. Vibration measurement is of great significance in fields such as structural health monitoring, equipment performance optimization, fault detection, and target detection and identification. Currently, vibration measurement methods are mainly divided into two categories: contact measurement methods and non-contact measurement methods. Common contact measurement methods include piezoelectric ceramic sensors, accelerometers, and gyroscope sensors. The measurement process is time-consuming and labor-intensive, and the bonding of the sensor may affect the dynamic characteristics of the structure itself. Non-contact measurement methods include laser Doppler vibration meters, eddy current sensors, and microwave interferometers. These devices are expensive and can only achieve vibration measurements at a limited number of discrete points, unable to achieve full-field vibration measurement.

[0003] Vision-based vibration measurement methods, as an emerging non-contact approach, capture vibration signals using a camera and combine image and video processing algorithms to measure vibration parameters such as amplitude and frequency. Traditional vision-based vibration measurement techniques employ algorithms such as feature point tracking, edge detection, and template matching to acquire vibration signals from videos, and then calculate the target vibration spectrum and peak frequency using Fast Fourier Transform (FFT). These methods rely on attaching targets to the object under test and are significantly affected by brightness variations and image noise, exhibiting poor robustness. Furthermore, the time-domain processing framework of these methods is based on traditional Fourier analysis, which is limited by the inverse relationship between observation time and the minimum spectral interval, making it difficult to obtain high spectral resolution within a short timeframe. This limitation restricts the practicality of vision-based vibration measurement methods in high-precision frequency measurement scenarios. Summary of the Invention

[0004] To address the problem that existing vibration measurements are limited by the relationship between observation time and minimum spectral interval, making it difficult to obtain high spectral resolution in a short time, this invention provides a vibration frequency measurement method based on video phase. This method can refine the spectrum and improve frequency resolution when the observation time is fixed, enabling high-precision vibration frequency measurement in a short time.

[0005] To solve the above problems, the present invention adopts the following technical solution.

[0006] The first aspect of this invention provides a vibration frequency measurement method based on video phase, the method comprising:

[0007] Acquire a video stream containing the target object, extract the region of interest from the video stream, and crop it to generate the video stream to be tested;

[0008] The color space of each frame in the video stream under test is converted to determine the YIQ image of the target object, and the Y channel image is extracted.

[0009] The Y-channel image is filtered using a pre-configured filter to obtain a phase sequence that characterizes the vibration signal.

[0010] Based on the phase sequence, the transformation parameters are set, and the spectrum corresponding to the phase sequence is obtained by linear frequency modulation transformation.

[0011] The amplitude is determined based on the spectrum, and the amplitude spectrum of the vibration signal of the target object is obtained. The maximum value of the amplitude spectrum is identified, and the frequency value corresponding to the maximum value is selected as the vibration frequency of the target object.

[0012] As an example, the step of filtering the Y-channel image using a pre-configured complex controllable filter includes:

[0013] By configuring complex controllable filters of different scales and directions as pre-configured filters, spatial filtering processing is performed on the Y-channel image to obtain spatial phase matrices of different scales and directions.

[0014] The spatial phase matrices of different scales and directions obtained from processing each frame of the image are arranged in the temporal order of the frames to obtain a phase tensor containing five dimensions: scale, direction, image x-coordinate, image y-coordinate, and time.

[0015] The phase tensor is averaged over scale, direction, image x-coordinate, and image y-coordinate to obtain the phase sequence characterizing the vibration signal.

[0016] As an example, configuring complex controllable filters of different scales and orientations as pre-configured filters includes the following steps:

[0017] Configure a polar coordinate image, which includes a polar radius image and a polar angle image, and convert the plane rectangular coordinates of the image into polar coordinates representation;

[0018] Configure the high-pass template HM and low-pass template LM for each scale;

[0019] Configure the bandpass template for each scale. For the i-th scale, its bandpass template can be obtained by multiplying the high-pass template of the i-th scale with the low-pass template of the (i-1)-th scale, as shown in the following formula:

[0020] BM i =HM i ⊙LMi-1 ; where ⊙ represents the Hadamard product, which is the multiplication of elements at corresponding positions;

[0021] Configure directional templates to divide the entire image plane into T directions, and set transition zones to form T directional templates; the specific formula for the directional templates is as follows:

[0022]

[0023]

[0024] Among them, OM j This represents the direction template for the j-th direction; j represents the direction index; Θ represents the polar angle matrix; λ(j) represents the intermediate variable used to calculate the direction template.

[0025] The bandpass template at each scale is multiplied by the templates in T directions to form a pre-configured complex controllable filter;

[0026] C i，j =BM i ⊙OM j (11)

[0028] Where C i，j Let OM represent a complex controllable filter at the i-th scale and j-th direction, where ⊙ represents the Hadamard product, i.e., element-wise multiplication at corresponding positions; j BM represents the bandpass template at the i-th scale; j This represents the direction template for the j-th direction.

[0029] As an example, the phase tensor P obtained by processing each frame of the video image i，j The phase sequence is obtained by averaging (x, y, k) over scale, orientation, image x-coordinate, and image y-coordinate, respectively, and the formula is as follows:

[0030] p(k)=∑ i，j ∑ x，y P i，j (x, y, k)

[0031] P(k) is the calculated phase sequence; k represents the k-th frame image, x represents the horizontal coordinate of the image, y represents the vertical coordinate of the image, i represents the scale number, and j represents the direction number.

[0032] As an example, the steps of setting the transformation parameters according to the phase sequence include:

[0033] Based on the frame rate of the video stream to be tested, the frequency band range and the desired frequency measurement resolution are specified in advance;

[0034] The transformation parameters are set according to the frequency band range and the desired frequency measurement resolution, which is obtained by measuring the shortest interval between two spectral lines.

[0035] As an example, the formula for selecting the frequency value corresponding to the maximum value as the vibration frequency of the target object is as follows:

[0036] f m =k m *fs / N

[0037] Where f m k represents the target vibration frequency. m Indicates the position corresponding to the maximum amplitude spectrum; N represents the total number of video frames; f s This indicates the video frame rate.

[0038] A second aspect of the present invention provides a vibration frequency measurement device based on video phase, comprising:

[0039] The video acquisition module is used to acquire a video stream containing the target object, extract the region of interest from the video stream, and crop it to generate a video stream to be tested.

[0040] The color conversion module is used to convert the color space of each frame in the video stream under test, determine the YIQ image of the target object, and extract the Y channel image.

[0041] The phase extraction module uses a pre-configured filter to filter the Y-channel image to obtain a phase sequence that characterizes the vibration signal.

[0042] The spectrum extraction module is used to set transformation parameters according to the phase sequence and obtain the spectrum corresponding to the phase sequence by using linear frequency modulation transformation.

[0043] The vibration measurement module is used to determine the amplitude based on the spectrum, obtain the amplitude spectrum of the vibration signal of the target object, identify the maximum value of the amplitude spectrum, and select the frequency value corresponding to the maximum value as the vibration frequency of the target object.

[0044] As an example, the phase extraction module includes:

[0045] The configuration unit is used to configure complex controllable filters of different scales and directions as pre-configured filters to perform spatial filtering processing on the Y channel image to obtain spatial phase matrices of different scales and directions.

[0046] The sorting unit is used to arrange the phase matrix obtained from processing each frame of image in chronological order of the frames to obtain phase tensors of different scales and directions.

[0047] The calculation unit is used to average the phase tensor over all scales, directions, and spatial positions to obtain a phase sequence characterizing the vibration signal.

[0048] A third aspect of the present invention provides an electronic device including a processor, an input device, an output device, and a memory, wherein the processor, the input device, the output device, and the memory are connected in sequence, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method described above.

[0049] The fourth invention provides a readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described above.

[0050] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0051] This invention, in its example, acquires a video stream containing a target object, extracts the region of interest from the video stream, and crops it to generate a video stream to be tested. Using a pre-configured filter, the Y-channel image is filtered to obtain a phase sequence characterizing the vibration signal. Based on the phase sequence, transformation parameters are set, and linear frequency modulation (LFM) transformation is used to obtain the spectrum corresponding to the phase sequence. This allows for refinement of the spectrum and improved frequency resolution when the observation time is fixed, enabling high-precision vibration frequency measurement in a short time. Compared to traditional visual vibration measurement, using spatial phase information for vibration signal extraction provides resistance to changes in illumination, image noise, and sensitivity to minute vibrations, thus improving the accuracy of vibration measurement. Attached Figure Description

[0052] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0053] Figure 1 A flowchart of a vibration frequency measurement method based on video phase provided in this embodiment of the invention. Figure 1 ;

[0054] Figure 2 A block diagram of a vibration frequency measurement device based on video phase provided in an embodiment of the present invention;

[0055] Figure 3 A flowchart of a vibration frequency measurement method based on video phase provided in this embodiment of the invention. Figure 2 ;

[0056] Figure 4 The illustration shows a block diagram of an electronic device according to an embodiment of this application;

[0057] Figure 5 This is a schematic diagram of a measurement scenario provided in an embodiment of the present invention;

[0058] Figure 6 This is a vibration spectrum obtained in an embodiment of the present invention. Detailed Implementation

[0059] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. It is obvious that the term "exemplary" herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0060] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0061] Exemplary scenario

[0062] Vibration measurement has applications in structural health monitoring, equipment performance optimization, fault detection, and target detection and identification. In one scenario, vibration videos of a target object can be pre-captured; this target object could be an engine, rope, or other object requiring observation. During filming, ensure the camera remains stationary throughout the entire process to acquire a video stream containing the vibration of the target object.

[0063] like Figure 2 As shown, in a laboratory setting, a modal exciter can be connected to a signal generator. Under the excitation of the signal generator, the modal exciter produces stable sinusoidal vibrations. A vibrator is placed on top of the modal exciter. A camera is fixed on a tripod, with its lens aimed at the vibrator at the top of the exciter, ensuring the camera remains stationary throughout the recording process; video acquisition begins, capturing a video stream containing the vibrations of the target object.

[0064] Exemplary methods

[0065] The time-domain processing framework of traditional visual vibration measurement technology is based on traditional Fourier analysis. It is limited by the inverse relationship between observation time and minimum spectral interval, making it difficult to obtain high spectral resolution in a short time. This also limits the practicality of vision-based vibration measurement methods in high-precision frequency measurement scenarios.

[0066] like Figure 1 and 3 As shown, this example provides a vibration frequency measurement method based on video phase to address the above problem, specifically including the following steps:

[0067] S110: Acquire a video stream containing the target object, extract the region of interest from the video stream, and crop it to generate the video stream to be tested.

[0068] Specifically, the video stream of the target object can be captured by a camera or transmitted via a receiving server. The ROI (Region of Interest) in the video is selected and cropped to generate the test video stream. The ROI refers to the area containing the target object; during cropping, it is ensured that the overall outline of the target object basically fills the video frame.

[0069] S120: Convert the color space of each frame in the video stream to be tested, determine the YIQ image of the target object, and extract the Y channel image.

[0070] Specifically, the video stream under test needs to undergo color gamut conversion, that is, the color space of the region of interest is converted from RGB to YIQ, so that the brightness information and chromaticity information of the video image are separated, which facilitates the separate processing of the brightness information of the video stream in the future.

[0071] Here, YIQ color is the color space used by the NTSC (National Television Standards Committee) color television system. The Y channel stores the luminance information of the image, while the I and Q channels store the chrominance information. For specific conversion rules, please refer to formula (1). Extract the Y channel image of each frame of the video stream under test for subsequent processing.

[0072]

[0073] S130: Using a pre-configured filter, the Y-channel image is filtered to obtain a phase sequence for characterizing the vibration signal.

[0074] Specifically, complex controllable filters of different scales and orientations are configured to process the Y-channel image, resulting in spatial phase matrices of different scales and orientations. Here, scale refers to the spatial frequency band of the complex controllable filter, and orientation refers to the spatial orientation of the complex controllable filter.

[0075] The phase matrices of different scales and directions obtained from processing each frame of the image are arranged in chronological order to form a phase tensor containing five dimensions: scale, direction, image x-coordinate, image y-coordinate, and time. The phase tensor is then averaged over all scales, directions, image x-coordinate, and image y-coordinate to obtain the phase sequence representing the vibration signal.

[0076] S140: Set the transformation parameters according to the phase sequence, and use linear frequency modulation transformation to obtain the spectrum corresponding to the phase sequence.

[0077] Specifically, the Chirp-Z transform is used to calculate the spectrum of the vibration signal, which serves as the vibration frequency measurement with frequency resolution.

[0078] As an example, the step of setting the transformation parameters according to the phase sequence includes:

[0079] Based on the frame rate of the video stream to be tested, pre-specify the frequency band range and the desired frequency measurement resolution.

[0080] The transformation parameters are set according to the frequency band range and the desired frequency measurement resolution, which is obtained by measuring the shortest interval between two spectral lines.

[0081] S150: Determine the amplitude based on the spectrum to obtain the amplitude spectrum of the vibration signal of the target object, identify the maximum value of the amplitude spectrum, and select the frequency value corresponding to the maximum value as the vibration frequency of the target object.

[0082] Specifically, the spectrum X(z) generated after the Chirp-Z transform of the phase sequence obtained in step S140 is... k The amplitude is obtained by calculating the amplitude spectrum and the position corresponding to the maximum value, as shown in formula (2).

[0083]

[0084] where k m This indicates the position corresponding to the maximum value of the amplitude spectrum.

[0085] According to formula (3), k m Convert to frequency value:

[0086] f m =k m *fs / N (3)

[0087] Where f m The target vibration frequency is represented by f; N represents the total number of video frames; f s This indicates the video frame rate.

[0088] In one embodiment, for Figure 2 The spectral diagram obtained from the oscillator measurement is as follows: Figure 3 As shown in Table 1, the results of multiple measurements are as follows.

[0089] Table 1

[0090]

[0091] In the table above, the error is calculated as: Error = |Measured value - Standard value| / Standard value. Compared to traditional vibration measurement methods, the above results achieve higher measurement accuracy and lower frequency measurement error within the same observation time.

[0092] As an example, to extract vibration signals using spatial phase information and to achieve resistance to illumination changes, image noise, and sensitivity to minute vibrations, this example improves upon the complex controllable filter as follows:

[0093] S131: Configure a polar coordinate image, which includes a polar radius image R and a polar angle image θ, and convert the plane rectangular coordinates of the image into polar coordinates.

[0094] Essentially, this involves converting the Cartesian coordinate representation of the Y-channel image into polar coordinates, which facilitates subsequent control of the frequency band and direction of each sub-filter. The horizontal and vertical coordinates of the image matrix are normalized to a predetermined range before being converted to polar coordinates.

[0095] S132: Configure the high-pass template HM and low-pass template LM for each scale.

[0096] For a filter radius r at a certain scale i i Based on the polar coordinates obtained in the first step, if regions with a polar radius greater than 1 are retained while regions with a polar radius less than 1 are discarded, a high-pass template for that scale is formed; otherwise, a low-pass template for that scale is formed. The high-pass filter template for this example is given by formula (4):

[0097]

[0098] Where HM represents the high-pass template, and r represents the polar radius; r i This represents the filter radius at a certain scale i;

[0099] Based on the high-pass template, a low-pass template of this scale can be designed, as shown in formula (5).

[0100]

[0101] Where LM represents a low-pass template and HM represents a high-pass template.

[0102] In a preferred embodiment, to ensure that the filter radius does not exceed the boundary of the Y-channel image, the filter radius r i The value of starts from 1 and gradually decreases, decreasing successively in geometric steps of 0.5, as follows:

[0103] r i =2 -(i-1) , i = 1, 2, ..., ht+1 (6)

[0104]

[0105] r i The filter radius is represented by ht; M represents the image height; N represents the image width; and ht represents the lower limit of image scaling by a factor of 2.

[0106] S133: Configure the bandpass template for each scale. For the i-th scale, its bandpass template BM i Equation (8) can be obtained by multiplying the high-pass template of the i-th scale with the low-pass template of the (i-1)-th scale.

[0107] BM i =HM i ⊙LM i-1 (8)

[0108] S134: Configure the direction template. To obtain motion information in different directions, the entire image plane is divided into T directions (T = 4 is generally chosen here), and a transition zone is set. The specific formula for the direction template is as follows:

[0109]

[0110]

[0111] Among them, OM j The direction template for the j-th direction is represented by λ(j); j represents the direction number; T represents the number of directions set; Θ represents the polar angle matrix; and λ(j) represents the intermediate variable for calculating the direction template.

[0112] S135: Multiplying the bandpass template at each scale with the templates in T directions respectively will form a pre-configured complex controllable filter.

[0113] C i，j =BM i ⊙OM j (11)

[0114] Where C i，j Let OM represent a complex controllable filter at the i-th scale and j-th direction, where ⊙ represents the Hadamard product, i.e., element-wise multiplication at corresponding positions; j BM represents the bandpass template at the i-th scale; j This represents the direction template for the j-th direction.

[0115] As an example, the local phase of the Y-channel image can be obtained by filtering in the spatial frequency domain using the complex controllable filter configured above, as shown in Equation (13):

[0116] P i，j =arg(IDFT(C i，j ⊙DFT(Y))) (13)

[0117] Where Y is the original image Y-channel matrix, DFT represents Discrete Fourier Transform, IDFT represents Inverse Discrete Fourier Transform; arg(.) represents the argument of a complex number; ⊙ represents the Hadamard product of the matrix, i.e., the element-wise multiplication; P i，j The final obtained local phase, considering the spatial coordinates of the image, is represented as P. i，j (x, y).

[0118] Then, the phase tensor P is obtained by processing each frame of the video. i，j (x, y, k) (k represents the k-th frame image), the phase changes of all layers and all positions are averaged to obtain the phase sequence, as shown in formula (14).

[0119] p(k)=∑ i，j ∑ x，y P i，j (x, y, k) (14)

[0120] P(k) is the calculated phase sequence; k represents the k-th frame image, x represents the horizontal coordinate of the image, y represents the vertical coordinate of the image, i represents the scale number, and j represents the direction number.

[0121] As another example, the specific steps for obtaining the spectrum corresponding to the phase sequence by setting the transformation parameters according to the phase sequence and using linear frequency modulation transformation are as follows:

[0122] S141: Chirp-Z transform parameter settings.

[0123] Based on the video frame rate fs, the pre-specified frequency band range [fl, fh], and the desired frequency measurement resolution F, set the transformation parameters. The transformation parameters include the number of frequency domain sampling points M, the starting point of spectral analysis A, and the contour step size W; specifically set as in formulas (15)-(17).

[0124] M=(f-fl) / F (15)

[0125] A = exp(j*2π*fl / fs) (16)

[0126] W=exp(j*2π*(f-fl) / fs) (17)

[0127] Where j represents the imaginary unit; exp represents an exponential function with the natural constant as its base.

[0128] S142: Chirp-Z transform of phase sequence.

[0129] Let the total length of the phase sequence p(n) be N, h L (n) The sequence is the constructed intermediate variable, and L represents h. L (n) is the length of the sequence, and L = N + M - 1; the specific algorithm steps are as follows:

[0130] ① Formation of h L (n) Sequence:

[0131] ② Calculate H L (k)=DFT[ L [n]0≤k≤L-1 (19)

[0132] ③Calculation

[0133] ④ Calculate Y(k)=DFT[y(n)]0≤k≤L-1 (21)

[0134] ⑤ Calculate Y(k)H(k) (22)

[0135] ⑥ Calculate V(k)=IDFT[Y(k)H(k)]0≤k≤L-1 (23)

[0136] ⑦ Calculation

[0137] Where DFT(.) represents the Discrete Fourier Transform, IDFT(.) represents the Inverse Discrete Fourier Transform, and H... L (k), y(n), Y(k), and V(k) are all intermediate variables constructed during the transformation process, and X(z) k This refers to the spectrum generated after the Chirp-Z transform of the phase sequence.

[0138] This example demonstrates a vibration frequency measurement method based on video phase and Chirp-Z transform. This method achieves a frequency resolution significantly higher than traditional Fourier transform within the same observation time, making it suitable for high-precision vibration frequency measurements under short time conditions. The invention utilizes spatial phase information for vibration signal extraction, exhibiting resistance to illumination variations, image noise, and sensitivity to minute vibrations. Compared to traditional visual vibration measurement methods, it offers higher frequency measurement accuracy.

[0139] Exemplary device

[0140] like Figure 2 As shown, a vibration frequency measurement device based on video phase includes:

[0141] Video acquisition module 20 is used to acquire a video stream containing the target object, extract the region of interest from the video stream, and crop it to generate a video stream to be tested;

[0142] The color conversion module 30 is used to convert the color space of each frame of the video stream under test, determine the YIQ image of the target object, and extract the Y channel image.

[0143] The phase extraction module 40 uses a pre-configured filter to filter the Y-channel image to obtain a phase sequence for characterizing the vibration signal.

[0144] The spectrum extraction module 50 is used to set transformation parameters according to the phase sequence and obtain the spectrum corresponding to the phase sequence by using linear frequency modulation transformation;

[0145] The vibration measurement module 60 is used to determine the amplitude based on the spectrum, obtain the amplitude spectrum of the vibration signal of the target object, identify the maximum value of the amplitude spectrum, and select the frequency value corresponding to the maximum value as the vibration frequency of the target object.

[0146] Specifically, the phase extraction module includes:

[0147] The configuration unit is used to configure complex controllable filters of different scales and directions as pre-configured filters to perform spatial filtering processing on the Y channel image to obtain spatial phase matrices of different scales and directions.

[0148] The sorting unit is used to arrange the spatial phase matrices of different scales and directions obtained from processing each frame of image in the order of the frame time to obtain a phase tensor containing five dimensions: scale, direction, image x-coordinate, image y-coordinate, and time.

[0149] The calculation unit is used to average the phase tensor with respect to scale, direction, image horizontal coordinate, and image vertical coordinate to obtain a phase sequence characterizing the vibration signal.

[0150] Exemplary electronic devices

[0151] Below, for reference Figure 1 This application describes an electronic device according to embodiments thereof. The electronic device may be a mobile device itself, or a standalone device independent of it, which can communicate with the mobile device to receive collected input signals from it and send selected target decision actions to it.

[0152] Figure 4 A block diagram of an electronic device according to an embodiment of this application is illustrated.

[0153] like Figure 4 As shown, the electronic device 10 includes one or more processors 11 and memory 12.

[0154] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0155] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the decision-making behavior and decision-making methods of the various embodiments of this application described above, and / or other desired functions.

[0156] In one example, electronic device 10 may further include input device 13 and output device 14, these components being interconnected via a bus system and / or other forms of connection mechanism (not shown). For example, input device 13 may include various devices such as an on-board diagnostic system (OBD), a unified diagnostic service (UDS), an inertial measurement unit (IMU), a camera, lidar, millimeter-wave radar, ultrasonic radar, vehicle-to-everything (V2X) communication, etc. Input device 13 may also include, for example, a keyboard, a mouse, etc. Output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0157] Of course, for the sake of simplicity, Figure 4Only some of the components of the electronic device 10 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0158] Exemplary computer program products and computer-readable storage media

[0159] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the decision-making methods according to various embodiments of this application as described in the "Exemplary Methods" section above.

[0160] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0161] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the decision-making methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0162] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0163] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0164] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0165] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0166] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0167] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A vibration frequency measurement method based on video phase, characterized in that, The method includes: Acquire a video stream containing the target object, extract the region of interest from the video stream, and crop it to generate the video stream to be tested; The color space of each frame in the video stream under test is converted to determine the YIQ image of the target object, and the Y channel image is extracted. The Y-channel image is filtered using a pre-configured filter to obtain a phase sequence that characterizes the vibration signal. Based on the phase sequence, the transformation parameters are set, and the spectrum corresponding to the phase sequence is obtained by linear frequency modulation transformation. The amplitude is determined based on the spectrum, the amplitude spectrum of the vibration signal of the target object is obtained, the maximum value of the amplitude spectrum is identified, and the frequency value corresponding to the maximum value is selected as the vibration frequency of the target object. The steps for setting transformation parameters based on the phase sequence include: Based on the frame rate of the video stream to be tested, the frequency band range and the desired frequency measurement resolution are specified in advance; The transformation parameters are set according to the frequency band range and the desired frequency measurement resolution, which is obtained by measuring the shortest interval between two spectral lines.

2. The vibration frequency measurement method based on video phase according to claim 1, characterized in that, The step of filtering the Y-channel image using a pre-configured complex controllable filter includes: By configuring complex controllable filters of different scales and directions as pre-configured filters, spatial filtering processing is performed on the Y-channel image to obtain spatial phase matrices of different scales and directions. The spatial phase matrices of different scales and directions obtained from processing each frame of the image are arranged in the temporal order of the frames to obtain a phase tensor containing five dimensions: scale, direction, image x-coordinate, image y-coordinate, and time. The phase tensor is averaged over scale, direction, image x-coordinate, and image y-coordinate to obtain the phase sequence characterizing the vibration signal.

3. The vibration frequency measurement method based on video phase according to claim 2, characterized in that, The steps for configuring complex controllable filters of different scales and orientations as pre-configured filters include: Configure a polar coordinate image, which includes a polar radius image and a polar angle image, and convert the plane rectangular coordinates of the image into polar coordinate representation; Configure the high-pass template HM and low-pass template LM for each scale; Configure the bandpass template for each scale, for the first i At this scale, its bandpass template can be used to... i The high-pass template of the first scale and the first i The result is obtained by multiplying the low-pass templates of size -1, as shown in the following formula: ; where ⊙ represents the Hadamard product, which is the multiplication of elements at corresponding positions; Configure the orientation template to divide the entire image plane into T There are T directional templates, and transition zones are set to form T directional templates; the specific formulas for the directional templates are as follows: in, OM j This represents the direction template for the j-th direction. j Indicates the direction number; Represents the polar angle matrix; Intermediate variables representing the calculation direction template; The bandpass template at each scale is multiplied by the templates in T directions to form a pre-configured complex controllable filter; Where C i,j Let represent a complex controllable filter at the i-th scale and j-th direction, and let ⊙ represent the Hadamard product, which is the multiplication of elements at corresponding positions. OM j BM represents the bandpass template at the i-th scale; j This represents the direction template for the j-th direction.

4. The vibration frequency measurement method based on video phase according to claim 2, characterized in that, The phase tensor P obtained by processing each frame of the video image i,j The phase sequence is obtained by averaging (x, y, k) over scale, orientation, image x-coordinate, and image y-coordinate, respectively, and the formula is as follows: P(k) is the calculated phase sequence; k represents the k-th frame image, x represents the horizontal coordinate of the image, and y represents the vertical coordinate of the image; i 'j' represents the scale number, and 'j' represents the direction number.

5. The vibration frequency measurement method based on video phase according to claim 1, characterized in that, The formula for selecting the frequency corresponding to the maximum value as the vibration frequency of the target object is as follows: in f m Indicates the desired vibration frequency; k m This indicates the position corresponding to the maximum value of the amplitude spectrum; N Indicates the total number of video frames; f s This indicates the video frame rate.

6. A vibration frequency measurement device based on video phase, characterized in that, include: The video acquisition module is used to acquire a video stream containing the target object, extract the region of interest from the video stream, and crop it to generate a video stream to be tested. The color conversion module is used to convert the color space of each frame in the video stream under test, determine the YIQ image of the target object, and extract the Y channel image. The phase extraction module uses a pre-configured filter to filter the Y-channel image to obtain a phase sequence that characterizes the vibration signal. The spectrum extraction module is used to set transformation parameters according to the phase sequence and obtain the spectrum corresponding to the phase sequence by using linear frequency modulation transformation. The vibration measurement module is used to determine the amplitude based on the spectrum, obtain the amplitude spectrum of the vibration signal of the target object, identify the maximum value of the amplitude spectrum, and select the frequency value corresponding to the maximum value as the vibration frequency of the target object. The steps for setting transformation parameters based on the phase sequence include: Based on the frame rate of the video stream to be tested, the frequency band range and the desired frequency measurement resolution are specified in advance; The transformation parameters are set according to the frequency band range and the desired frequency measurement resolution, which is obtained by measuring the shortest interval between two spectral lines.

7. The vibration frequency measurement device based on video phase according to claim 6, characterized in that, The phase extraction module includes: The configuration unit is used to configure complex controllable filters of different scales and directions as pre-configured filters to perform spatial filtering processing on the Y channel image to obtain spatial phase matrices of different scales and directions. The sorting unit is used to arrange the phase matrix obtained from processing each frame of image in chronological order of the frames to obtain phase tensors of different scales and directions. The calculation unit is used to average the phase tensor over all scales, directions, and spatial positions to obtain a phase sequence characterizing the vibration signal.

8. An electronic device, characterized in that, The device includes a processor, an input device, an output device, and a memory, which are connected in sequence. The memory is used to store a computer program, which includes program instructions. The processor is configured to call the program instructions to execute the method as described in any one of claims 1-6.

9. A readable storage medium, characterized in that, The storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-6.

Images