A full vector laser speckle displacement detection method, device and system integrated with a YOLO image recognition model
By integrating the YOLO image recognition model and Fourier transform, combined with the sliding table mechanical structure, the problems of manual dependence and hardware limitations in the existing technology are solved, realizing fast and accurate full-vector laser speckle displacement detection, reducing costs and improving robustness and measurement accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIAMEN UNIV TAN KAH KEE COLLEGE
- Filing Date
- 2026-04-03
- Publication Date
- 2026-07-03
Smart Images
Figure CN122335979A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of precision optical measurement and artificial intelligence, and in particular to an integrated method, device and system for full-vector laser speckle displacement detection that combines the YOLO image recognition model. Background Technology
[0002] In existing laser speckle-based displacement, vibration, or deformation measurement schemes, the classic image processing and calculation process is as follows: The system first acquires two speckle images of the object before and after displacement; then, the two images are superimposed, synthesized, and grayscaled; subsequently, a two-dimensional Fourier transform is performed on the synthesized image to generate an interference fringe pattern containing periodic brightness and darkness features in the frequency domain, such as... Figure 1 As shown; finally, existing technologies usually rely on setting a fixed absolute brightness threshold in the program to segment bright and dark stripes, and then extract the stripe spacing and use physical formulas to invert and calculate the actual displacement.
[0003] Disadvantages of existing technology:
[0004] 1. Data reading heavily relies on manual intervention or fixed thresholds. While the program can identify data within a specific measurement range, its robustness is extremely poor. When the laser output intensity is unstable or changes otherwise, traditional threshold algorithms are prone to errors in judging stripe spacing. Manual numerical reading is inefficient, resulting in a very low degree of automation in current technologies.
[0005] 2. Strong hardware dependence and difficult deployment: Most existing algorithms rely on high-performance computers and optical platforms, resulting in bulky equipment. This architecture is not conducive to deployment in confined spaces or extreme industrial environments.
[0006] 3. Installation errors affect accuracy: During the actual installation process, systematic errors will occur due to mechanical tolerances or alignment deviations. Existing technical solutions lack effective online compensation mechanisms, which affects the final measurement accuracy.
[0007] 4. The interference fringes obtained by the core algorithm of the existing technology through a single two-dimensional Fourier calculation have a fringe spacing that is inversely proportional to the actual displacement. Therefore, for large displacements, the measurement range is easily exceeded.
[0008] 5. Due to the conjugate symmetry of the Fourier transform power spectrum, the positive and negative directions of displacement cannot be distinguished solely by frequency domain characteristics, making it impossible to perform full vector measurement of displacement using current methods. Summary of the Invention
[0009] In view of this, the purpose of this invention is to provide an integrated full-vector laser speckle displacement detection method, device and system that combines the YOLO image recognition model, which can automatically, in real time and accurately identify the stripe spacing or related peak spacing. Compared with the traditional scheme that relies on manual or fixed thresholds, its robustness and automation level have been revolutionaryly improved, and the response speed can reach the millisecond level.
[0010] To achieve the above objectives, the present invention adopts the following technical solution: an integrated full-vector laser speckle displacement detection method combining a YOLO image recognition model, comprising the following steps:
[0011] Step S1: Control the laser to emit laser light, which is then expanded and collimated to irradiate the surface of the object under test perpendicularly. The image sensor continuously acquires the reference speckle image before displacement and the deformed speckle image after displacement.
[0012] Step S2: Perform feature transformation processing on the acquired reference speckle image and deformed speckle image based on the image preprocessing algorithm. The image preprocessing algorithm includes a joint transformation method of first-order two-dimensional Fourier transform and second-order two-dimensional Fourier transform.
[0013] Step S3: Input the image data after feature transformation processing in step S2 into the preset deep learning target detection model YOLO. The deep learning target detection model YOLO identifies and extracts the pixel spacing of interference fringes through a first two-dimensional Fourier transform or extracts the pixel offset of the relevant peak through a second two-dimensional Fourier transform.
[0014] Step S4: Based on the pixel spacing of the interference fringes or the pixel offset of the related peaks extracted by the deep learning target detection model YOLO, the magnitude of the displacement scalar is calculated using the regression function fitted by the sliding mechanical structure.
[0015] Step S5: Input the original spatial domain speckle image sequence before and after displacement into the orientation discrimination model of the parallel CNN-SVM fusion architecture to obtain the absolute orientation of displacement;
[0016] Step S6: Perform vector synthesis of the magnitude of the displacement scalar and the absolute direction of the displacement to output a complete two-dimensional displacement vector.
[0017] In a preferred embodiment, step S2, the joint conversion method of the first-order two-dimensional Fourier transform and the second-order two-dimensional Fourier transform, specifically includes:
[0018] Based on the distribution of characteristic frequency bands, the processing branch is adaptively selected:
[0019] Branch 1: When the displacement is determined to be small, perform a two-dimensional fast Fourier transform on the reference speckle image and the deformed speckle image to generate an interference fringe spectrum image;
[0020] Branch 2: When a large displacement is determined, the reference speckle image and the deformed speckle image are synthesized in the spatial domain to obtain a composite speckle image. The composite speckle image is then subjected to a first two-dimensional fast Fourier transform to obtain a power spectrum. The power spectrum is then subjected to a second two-dimensional fast Fourier inverse transform to generate a correlation peak image containing a central zero-order bright spot and symmetrically distributed secondary correlation peaks on both sides.
[0021] In a preferred embodiment, step S5 specifically includes: inputting the original spatial domain speckle image sequence before and after displacement into a convolutional neural network (CNN) to extract the asymmetric texture features of the speckle; flattening the extracted high-dimensional feature vectors and inputting them into a support vector machine (SVM) classifier to perform binary classification on the motion trend of the speckle sequence in order to accurately determine the absolute direction of displacement.
[0022] In a preferred embodiment, an error compensation step is also included: after the device is installed, a dataset of corresponding pixels and standard displacements is generated using the integrated slide mechanical structure, a linear regression model is constructed using the least squares method, and the system measurement values are calibrated to reduce systematic errors, improve the device's anti-interference ability, and reduce subsequent maintenance.
[0023] In a preferred embodiment, after calculating the magnitude of the displacement scalar in step S4, the reference space angle where the displacement is located is also calculated based on the horizontal and vertical offsets extracted from the YOLO model using the arctangent trigonometric function. This angle is then used to perform vector synthesis with the magnitude of the displacement scalar and the absolute direction of the displacement.
[0024] This invention also provides an integrated full-vector laser speckle displacement detection device incorporating a YOLO image recognition model, used to implement the aforementioned integrated full-vector laser speckle displacement detection method incorporating a YOLO image recognition model, characterized in that it includes:
[0025] Laser emitting module, used to emit lasers;
[0026] A laser beam expanding and collimating system is used to expand and collimate the laser emitted by the laser emitting module;
[0027] The image acquisition module is used to continuously acquire reference speckle images before displacement and deformed speckle images after displacement;
[0028] The sliding table mechanical structure is connected to the laser emitting module or the image acquisition module and is used to drive the reflective surface to generate a standard displacement for system calibration.
[0029] The data processing module is configured to execute the integrated full-vector laser speckle displacement detection method combining the YOLO image recognition model as described in any one of claims 1 to 5, and output a two-dimensional displacement vector.
[0030] In a preferred embodiment, the data processing module includes a local terminal or a cloud server; the device further includes an Internet of Things (IoT) communication module for uploading the acquired speckle images to the cloud server for processing, or for displaying the processing results on a client.
[0031] In a preferred embodiment, the laser emitting module, the laser beam expanding and collimating system, and the image acquisition module constitute a single-beam projection and non-contact speckle acquisition optical path structure; the laser beam expanding and collimating system includes a short focal length lens and a convex lens arranged sequentially.
[0032] This invention also provides an integrated full-vector laser speckle displacement detection system incorporating a YOLO image recognition model, comprising: the aforementioned integrated full-vector laser speckle displacement detection device incorporating a YOLO image recognition model; and a mobile terminal, wherein the mobile terminal communicates with a cloud server via a webpage to display the displacement magnitude, direction vector, and historical data curves in real time, and provides threshold alarms for abnormal displacements.
[0033] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned integrated full-vector laser speckle displacement detection method incorporating a YOLO image recognition model.
[0034] Compared with the prior art, the present invention has the following beneficial effects:
[0035] 1) This invention can respond to displacement in real time and quickly. Combined with a deep learning model, it can output parameters such as displacement angle and magnitude of the observed displacement surface in real time to achieve fully automatic data acquisition. Compared with traditional methods, the response speed and measurement accuracy are greatly improved.
[0036] 2) Existing technologies require local computers directly connected to photoelectric detection components to have extremely high computing and storage capabilities, resulting in high costs. This solution calculates and stores data in the cloud, reducing the computing power requirements of measurement equipment, significantly lowering costs, and expanding the user base and methods, such as allowing students to use their mobile phones for optical experiments.
[0037] 3) This device combines the YOLO artificial intelligence model, which greatly improves the robustness and automation of data acquisition and increases the response speed to the millisecond level.
[0038] 4) The innovative introduction of the second Fourier transform combined with the first Fourier transform greatly improves the limit resolution and measurement range of the same device by using the algorithm, giving the existing solution a larger dynamic range.
[0039] 5) Existing instruments are prone to measurement bias after long-term operation or redeployment. By introducing a linear regression model, such systematic errors can be fitted and their effects eliminated.
[0040] 6) Existing technologies can only measure the scalar magnitude of displacement. This solution introduces a CNN-SVM model to measure the direction of displacement, which cannot be measured by existing technologies. Attached Figure Description
[0041] Figure 1 To generate interference fringe patterns with periodic brightness and darkness features in the frequency domain using existing techniques;
[0042] Figure 2 This is a schematic diagram of the device structure according to a preferred embodiment of the present invention;
[0043] Figure 3 This is a flowchart illustrating a preferred embodiment of the present invention;
[0044] Figure 4 This is a schematic diagram of the image processing process of a preferred embodiment of the present invention, wherein (a) is a speckle image before displacement, (b) is a speckle image after displacement, (c) is a correlation peak image, (d) is a schematic diagram of YOLO identification of correlation peak intervals, (e) is an interference fringe pattern, and (f) is a schematic diagram of YOLO identification of fringe intervals.
[0045] Figure 5 This is a flowchart of the deep learning model processing according to a preferred embodiment of the present invention;
[0046] Figure 6 This is a flowchart illustrating the workflow of a linear regression model according to a preferred embodiment of the present invention.
[0047] Figure 7 This is a flowchart of cloud data processing according to a preferred embodiment of the present invention. Detailed Implementation
[0048] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0049] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.
[0050] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this application; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.
[0051] This invention provides a laser speckle displacement measurement method integrating the YOLO artificial intelligence algorithm, with reference to... Figures 2-7 This includes the following steps and equipment:
[0052] This invention provides a hardware module containing a laser 1, a laser beam expander and collimator system, and a CCD receiver. Specifically, to enable calibration or dynamic testing of displacement, a sliding mechanical structure 4 is integrated. This sliding mechanical structure 4 is connected to the laser emitting module or the CCD module and can drive the reflecting surface to move independently or in conjunction with the laser's principal optical axis in the X direction (lateral displacement) and Y direction (longitudinal displacement). A simplified structure is shown below. Figure 2 As shown. The device's operating procedure is as follows. Figure 3 As shown.
[0053] The optical path of the module that controls laser 1 to emit laser light is shown in the attached diagram. Figure 2 As shown, the laser emitted by laser 1 is first expanded by a short-focal-length lens 2 (L1), and then collimated by a convex lens 3 (L2) to obtain a parallel and coherent beam. This beam is then perpendicularly incident on the object's surface, and the reflected light forms a characteristic speckle field carrying information about the object's displacement. An image sensor (CCD) can continuously acquire images of this characteristic speckle field in real time. As the displacement surface shifts, consecutive image frames are recorded, thus obtaining the first speckle image before displacement and the second speckle image after displacement, as shown below. Figure 4 of (a) Figure 4 (b)
[0054] There are two approaches to processing speckle images.
[0055] Option 1: Import the data into a local terminal (computer) for recognition and calculation, and then display the results.
[0056] Option 2: Images can be uploaded to the cloud via an IoT module for processing and then displayed on the client (mobile phone). This cloud platform solution is also applicable to mobile phone photography. For some devices or scenarios that have built-in laser emission modules, such as in some optical experiments, users only need to fix their mobile phones in place and upload the speckle images taken by the phones to the cloud to achieve the same result.
[0057] The acquired raw speckle image is processed using an image preprocessing algorithm for feature transformation. Specifically, the image preprocessing algorithm innovatively employs a joint transformation method combining a first-order two-dimensional Fourier transform and a second-order two-dimensional Fourier transform. The processing procedure is as follows:
[0058] First, the system analyzes the acquired reference speckle image. Deformation speckle image after displacement A preliminary automatic assessment of the displacement magnitude is performed. Based on the distribution of characteristic frequency bands, the algorithm adaptively switches to different processing branches:
[0059] Branch 1: For minute displacements, perform a two-dimensional fast Fourier transform to extract the interference fringe spectrum image. When the system determines that it is in a minute displacement range, due to the small displacement, perform a two-dimensional fast Fourier transform to convert it to the frequency domain. This process will generate an interference fringe spectrum image with distinct periodic bright and dark alternating patterns (e.g., Figure 4 (As shown in (e)). According to the principle of optical interference, the pixel spacing of the fringes is inversely proportional to the actual physical displacement. This method can effectively amplify minute displacement features, providing a clear signal input for subsequent networks.
[0060] Branch Two: For large displacements, a dual 2D-FFT algorithm is used to extract relevant peak images. As the displacement increases, the interference fringes generated by the first transformation become extremely dense and may even alias. In this case, the system switches to Branch Two algorithm processing. A dual 2D-FFT algorithm is employed.
[0061] The specific processing procedure is as follows: First, the reference speckle image is processed in the spatial domain. Deformation speckle image after displacement Synthesizing yields a composite speckle image. Subsequently, a two-dimensional fast Fourier transform is performed, and its power spectrum is calculated. Then, the system performs a second 2D Fourier transform on the power spectrum, converting it back to the correlation domain to obtain the two-dimensional autocorrelation domain. The operation process satisfies the following mathematical formula:
[0062]
[0063]
[0064]
[0065] in, and These represent the two-dimensional forward and inverse Fourier transforms, respectively. Through the above double Fourier operations, the originally dense Young's fringes are globally integrated and sharpened, transforming into spatially concentrated pulse feature points, thus forming an image containing a central zero-order bright spot and symmetrically distributed secondary correlation peaks (e.g., Figure 4 (as shown in (c)).
[0066] Finally, the system uploads the generated interference fringe pattern or two-dimensional correlation peak image and inputs it into the YOLO object detection model deployed locally or in the cloud. The trained YOLO model then identifies the fringe spacing or coherent peak spacing (e.g., Figure 4 (as shown in (d)). This recognition process is as described in the deep learning model processing flow. Figure 5 As shown.
[0067] Subsequently, the image data, after feature transformation processing, is input into a pre-built deep learning object detection model, YOLO. This model is built on a convolutional neural network (CNN) architecture and utilizes relevant peak images and stripe patterns (such as...) beforehand. Figure 4 (c) Figure 4 The sample data (shown in (e)) is used to train the device to accurately identify and locate stripe intervals and related peak intervals, and output displacement information.
[0068] The stripe spacing D and the actual displacement S conform to the following formula:
[0069]
[0070] in: : The center pixel spacing of adjacent interference fringes identified and extracted by the YOLO model; : The sampling window size (total number of pixels) used during Fourier transform processing. : The physical size of a single pixel in an image acquisition device (camera sensor); : Magnification of the optical imaging system.
[0071] The correlation peak interval and the actual displacement S conform to the following formula:
[0072]
[0073] in The actual physical displacement of the object; , : The horizontal and vertical pixel offsets of the correlation peak relative to the image center; The physical size of a single pixel in an image acquisition device (camera sensor); Magnification of an optical imaging system.
[0074] After calculating the magnitude of the displacement scalar, the system further performs the synthesis of absolute two-dimensional displacement vectors.
[0075] First, the system extracts the horizontal offset from the YOLO model. With vertical offset The reference space angle where the displacement is located is calculated by using the arctangent trigonometric function. It satisfies the formula:
[0076]
[0077] However, due to the conjugate symmetry of the Fourier transform power spectrum, the positive and negative directions of the displacement cannot be distinguished solely by frequency domain characteristics (i.e., there exists...). (The direction is ambiguous). To address this, this system innovatively connects a direction discrimination model based on a CNN-SVM fusion architecture in parallel at the end of the algorithm.
[0078] The specific operating mechanism is as follows: The system extracts the original spatial domain speckle image sequence (or phase feature map) before and after displacement and inputs it into a convolutional neural network (CNN). Utilizing the powerful deep feature extraction capability of CNN, the asymmetric texture features of the speckle are captured; subsequently, the extracted high-dimensional feature vectors are flattened and input into a support vector machine (SVM) classifier to perform binary classification on the motion trend of the speckle sequence, thereby accurately obtaining the absolute direction of the physical displacement.
[0079] Finally, the system will calculate the magnitude of the displacement scalar and the reference space angle. The positive and negative direction indicators determined by the CNN-SVM model are used for vector synthesis to output a complete two-dimensional displacement vector with absolute direction arrows.
[0080] Specifically, this invention provides an error compensation algorithm that incorporates a linear regression model for system calibration. Constraints on laser collimation and CCD receiving angle are relaxed, allowing for automatic compensation via the linear regression model. This significantly increases the versatility of the equipment and method, while also reducing hardware costs. After each hardware installation, standard sampling data is generated using the built-in slide mechanical structure 4. System parameters are then corrected through data regression fitting to reduce systematic errors. The specific process is as follows:
[0081] An error compensation model is constructed using the least squares method. Assume the measured value is... The actual value (provided by the mechanical slide structure 4) is The calibrated output Represented as:
[0082]
[0083] Model parameters (scaling factor) and (Bias factor) minimizes the loss function get:
[0084]
[0085] Where: n: the number of calibration sample points; : No. Each sampling point is solved and This allows for the acquisition of optimal system calibration parameters. The workflow of its linear regression model is as follows: Figure 6 As shown.
[0086] Finally, this device also includes mobile terminals (including but not limited to smartphones and PCs) that can read and output cloud computing data via web pages, and provides a visual human-computer interaction interface for real-time display of displacement magnitude, direction vector, and historical data curves, and provides threshold alarms for abnormal displacement. Its cloud data processing flow is as follows: Figure 7 As shown.
Claims
1. A full vector laser speckle displacement detection method integrated with a YOLO image recognition model, characterized in that, Includes the following steps: Step S1: Control the laser to emit laser light, which is then expanded and collimated to irradiate the surface of the object under test perpendicularly. The image sensor continuously acquires the reference speckle image before displacement and the deformed speckle image after displacement. Step S2: Perform feature transformation processing on the acquired reference speckle image and deformed speckle image based on the image preprocessing algorithm. The image preprocessing algorithm includes a joint transformation method of first-order two-dimensional Fourier transform and second-order two-dimensional Fourier transform. Step S3: Input the image data after feature transformation processing in step S2 into the preset deep learning target detection model YOLO. The deep learning target detection model YOLO identifies and extracts the pixel spacing of interference fringes through a first two-dimensional Fourier transform or extracts the pixel offset of the relevant peak through a second two-dimensional Fourier transform. Step S4: Based on the pixel spacing of the interference fringes or the pixel offset of the related peaks extracted by the YOLO deep learning target detection model, the magnitude of the displacement scalar is analytically calculated using the regression function fitted by the slide mechanical structure. Step S5: Input the original spatial domain speckle image sequence before and after displacement into the orientation discrimination model of the parallel CNN-SVM fusion architecture to obtain the absolute orientation of displacement; Step S6: Perform vector synthesis of the magnitude of the displacement scalar and the absolute direction of the displacement to output a complete two-dimensional displacement vector.
2. The method according to claim 1, wherein the method is characterized in that, In step S2, the joint conversion method of the first-order two-dimensional Fourier transform and the second-order two-dimensional Fourier transform specifically includes: Based on the distribution of characteristic frequency bands, the processing branch is adaptively selected: Branch 1: When the displacement is determined to be small, perform a two-dimensional fast Fourier transform on the reference speckle image and the deformed speckle image to generate an interference fringe spectrum image; Branch 2: When a large displacement is determined, the reference speckle image and the deformed speckle image are synthesized in the spatial domain to obtain a composite speckle image. The composite speckle image is then subjected to a first two-dimensional fast Fourier transform to obtain a power spectrum. The power spectrum is then subjected to a second two-dimensional fast Fourier inverse transform to generate a correlation peak image containing a central zero-order bright spot and symmetrically distributed secondary correlation peaks on both sides.
3. The method according to claim 1, wherein the method is characterized in that, Step S5 specifically includes: inputting the original spatial domain speckle image sequence before and after displacement into a convolutional neural network (CNN) to extract the asymmetric texture features of the speckles; flattening the extracted high-dimensional feature vectors and inputting them into a support vector machine (SVM) classifier to perform binary classification on the motion trend of the speckle sequence in order to accurately determine the absolute direction of displacement.
4. The method according to claim 1, wherein the method is characterized in that, It also includes an error compensation step: after the equipment is installed, the integrated slide mechanical structure is used to generate a dataset of corresponding pixels and standard displacements, and the least squares method is used to build a linear regression model to calibrate the system measurement values, so as to reduce systematic errors, improve the equipment's anti-interference ability and reduce later maintenance.
5. The method according to claim 1, wherein the method is characterized in that, After calculating the magnitude of the displacement scalar in step S4, the reference space angle where the displacement is located is calculated based on the horizontal and vertical offsets extracted from the YOLO model using the arctangent trigonometric function. This angle is then used to perform vector synthesis with the magnitude of the displacement scalar and the absolute direction of the displacement.
6. An integrated full-vector laser speckle displacement detection device incorporating a YOLO image recognition model, used to implement the integrated full-vector laser speckle displacement detection method incorporating a YOLO image recognition model as described in any one of claims 1 to 5, characterized in that, include: Laser emitting module, used to emit lasers; A laser beam expanding and collimating system is used to expand and collimate the laser emitted by the laser emitting module; The image acquisition module is used to continuously acquire reference speckle images before displacement and deformed speckle images after displacement; The sliding table mechanical structure is connected to the laser emitting module or the image acquisition module and is used to drive the reflective surface to generate a standard displacement for system calibration. The data processing module is configured to execute the integrated full-vector laser speckle displacement detection method combining the YOLO image recognition model as described in any one of claims 1 to 5, and output a two-dimensional displacement vector.
7. The integrated vector laser speckle displacement detection device combined with a YOLO image recognition model according to claim 6, characterized in that, The data processing module includes a local terminal or a cloud server; the device also includes an Internet of Things (IoT) communication module, used to upload the collected speckle images to the cloud server for processing, or to display the processing results on the client.
8. The integrated vector laser speckle displacement detection device combined with a YOLO image recognition model according to claim 6, characterized in that, The laser emitting module, the laser beam expanding and collimating system, and the image acquisition module constitute a single-beam projection and non-contact speckle acquisition optical path structure; the laser beam expanding and collimating system includes a short focal length lens and a convex lens arranged in sequence.
9. A full vector laser speckle displacement detection system integrated with a YOLO image recognition model, characterized in that, include: An integrated full-vector laser speckle displacement detection device combining a YOLO image recognition model, as described in any one of claims 6 to 8; The mobile terminal communicates with the cloud server via a webpage to display the magnitude of displacement, direction vector, and historical data curves in real time, and provides threshold alarms for abnormal displacement.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the integrated full-vector laser speckle displacement detection method combining the YOLO image recognition model as described in any one of claims 1 to 5.