Dic strain detection apparatus and method based on display speckle image

Abstract

The application discloses a DIC strain detection method and device based on display speckle images, and the speckle is prepared on a display device and fixed on a stage; the detection device is calibrated to accurately determine the internal and external parameters of the system; the region of interest is meshed; a series of digital images are acquired by using an image acquisition device; for the region of interest, correlation calculation is performed according to a pre-defined correlation function, a corresponding subset is searched in the deformed image, and speckle strain is determined; all subsets are calculated to derive a strain field. The method has extremely loose requirements on an experimental environment, and has the advantages of full-field measurement, no need of physical film covering on a measured sample, strong anti-interference capability and the like, and can realize strain analysis on the display device.

Description

Technical Field

[0001] This invention relates to the field of optical measurement, and more specifically, to a DIC strain detection device and method based on display speckle images. Background Technology

[0002] With the rapid development of the information age, information display has become an indispensable part of the information industry, and displays, as carriers of information presentation, have received wider attention. The amount of time people spend on mobile phones and computers every day is enough to prove the importance of display technology. At the same time, people have put forward higher requirements for display devices in terms of power consumption, size, and flexibility.

[0003] Traditional LCD and flat panel displays can no longer meet the growing demands for display functionality, especially in flexible displays. The market demand for flexible display devices is increasingly strong, and the market potential for flexible display technology is enormous, making research on this technology of significant practical importance. The rapid development of intelligent portable products has prompted major manufacturers to increase their R&D efforts in flexible displays. However, flexible display technology is still not mature enough, and there is currently no comprehensive and systematic analytical method for flexible display modules. The complex mechanical behavior of materials in flexible displays has not been thoroughly explored, and there are still no effective solutions to the problems caused by excessive stress and severe deformation of the adhesive layer when flexible displays are bent at small radii. Currently, scholars have conducted extensive research on the bending stress of flexible displays from theoretical and simulation perspectives, achieving good results, but they cannot obtain the true bending stress of flexible displays, and the analysis results differ from the actual situation.

[0004] Digital image correction (DIC) is a digital image tracking technique based on the identifiable speckle pattern on the surface of the test sample. Traditional DIC methods involve physical coatings applied to the test sample using methods such as spraying or dyeing. However, the size of the speckle is difficult to control and has defects that cannot be repaired or adjusted. When performing bending or tensile tests, the speckle coating is very easy to peel off, limiting the number of tests that can be performed.

[0005] For major panel manufacturers, combining optical inspection and digital imaging technologies to form a complete and systematic inspection solution as soon as possible will enable them to occupy a more powerful position in the flexible display market. Summary of the Invention

[0006] To address the aforementioned problems, the present invention aims to provide a DIC strain detection device and method based on display speckle images.

[0007] In a first aspect, the present invention provides a DIC strain detection device based on displaying speckle images. The device includes a display control device, an image acquisition device, and an image processing device. The display control device controls the display of speckle images on the test sample. The image acquisition device is used to acquire image information of the entire deformation process of the test sample. The image processing device stores the image information acquired by the image acquisition device and performs strain analysis on the image information.

[0008] Furthermore, the DIC strain testing device also includes a light source emitting device disposed on the test sample to provide full-field illumination for the testing device. Preferably, the light source emitting device is an LED light source, which illuminates the test sample at different angles to provide a uniform light field. This allows the image acquisition device to collect accurate image information. Preferably, it includes two identical LED light sources placed symmetrically on both sides of the test sample.

[0009] Furthermore, the DIC strain detection device also includes a stage, which is used to load the sample to be tested and drive the sample, which has been shown with speckle images, to bend according to a preset motion mode, so as to ensure that the image acquisition device accurately captures the crease of the sample and realizes strain detection at the crease.

[0010] Furthermore, the display control device includes one or more of the following: microcontroller, graphics processing unit (GPU), mobile phone, tablet computer, laptop computer, ultra-mobile personal computer (UMPC), handheld computer, netbook, personal digital assistant (PDA), and wearable device.

[0011] Furthermore, the image acquisition device includes a camera for acquiring speckle images; preferably, the camera is a color camera.

[0012] Secondly, the present invention also provides a DIC strain detection method based on a display speckle image, comprising the following steps:

[0013] S101, the display control device controls the test sample to display a speckle image; the test sample is a display device capable of displaying images;

[0014] S102, calibrate the DIC strain detection device to accurately determine the internal and external parameters of the system;

[0015] S103, acquire image information of the entire deformation process of the sample under test using an image acquisition device, and store the image information using the image processing device;

[0016] S104, divide the computational region in the image of the sample before deformation into subsets, and set the values ​​of subsets and step size;

[0017] S105, for each subset, perform relevant calculations according to a predefined correlation function, find the corresponding subset in the deformed image, and determine the speckle strain;

[0018] S106 calculates all subsets, derives the strain field, and obtains the deformation and strain information of the entire field.

[0019] Furthermore, the displayed speckle image should meet the following requirements: the location of the speckle is random, the shape is random, the size, shape, and position are not repeated, the contrast has characteristics that can be distinguished by a computer program, and the speckle size is at least 3-5 pixels.

[0020] Furthermore, the display control device in step S101 is used to generate a display speckle image; the display control device has one or more computer programs, the one or more computer programs including instructions, which, when executed by the one or more control units, cause the test sample to have a display speckle image.

[0021] Further, in step S102, the image acquisition device is a camera, which captures images of the calibration board from different angles. A bundle adjustment algorithm is used to calculate the camera's internal parameters, including focal length, principal point, and distortion parameters, as well as its external parameters, including translation vectors and rotation matrices. The camera is positioned to capture the entire measurement area, ensuring sufficient field of view for the expected movement of the display device. After the camera is positioned, the lens focal length is adjusted to achieve a clear focus on the display device. Before analysis, size calibration is performed using images with known actual distances. Subsequent analytical data includes both pixel units and millimeter units.

[0022] Furthermore, the subset mentioned in step S104 includes a subset and a step size. The value of the subset controls the size of the DIC matching region, and the step controls the density of the analysis data. Preferably, the region of interest (crease) in the image before deformation of the electronic screen is divided into several subsets, and each subset is treated as rigid motion. Specifically, in the digital image of the display device before deformation, a region of interest is selected, and an appropriate subset and step size are selected for the region of interest. This image is used as the reference image by default, and all strains and displacements will be based on this image.

[0023] Further, step S105 includes the following steps: using a software program to analyze a custom computational region in the image, which contains a set of computational or measurement points; in each subset, each computational point is located at the center of the subset; correlation calculations are performed on the subsets for each image from the reference image to the subsequent deformable device; the correlation calculations first approximate the speckle image in the subset using an interpolation function, wherein the interpolation function is a subset-based shape function; and the subsets in the reference image are matched with the corresponding subsets in the deformable image using a matching criterion that incorporates subset weights.

[0024] Furthermore, the strain analysis uses data from the image processing device to calculate the deformation of the sample under test.

[0025] Furthermore, the apparatus and testing method provided by this invention can be applied to strain testing of flexible displays. The flexible display includes rollable displays or foldable displays.

[0026] Furthermore, the sample to be tested is loaded onto a stage, which drives the sample, which has already displayed a speckle image, to bend according to a preset motion pattern. This ensures that the image acquisition device accurately captures the crease of the sample and acquires a series of digital images of the entire bending process of the sample. These digital images are then stored by the image processing device.

[0027] The present invention has the following beneficial effects:

[0028] (1) The present invention uses speckle images as the measurement basis, so the measurement range can be set according to actual needs. It has a wide range of applications and can measure small deformations. It is especially suitable for flexible displays, such as roll-up displays and folding displays.

[0029] The DIC strain detection device used in this invention can realize online analysis and provide rapid real-time measurement. When dealing with materials with large deformation, it is necessary to readjust the camera position and the region of interest in the image to ensure that the deformation area can be completely captured by the image acquisition device and processed and analyzed by the image processing device.

[0030] (2) The present invention provides a DIC strain analysis device and method for display devices. The method of displaying speckle images on the display device eliminates the need for physical coating methods such as spraying and dyeing, thus avoiding speckle peeling during the test. In particular, the speckle will not peel off when bending, and multiple repeated tests can be performed. On the other hand, compared with physical coating, the method of displaying speckle images can analyze not only the surface stress of the display device, but also the stress of the inner layer of the screen. Attached Figure Description

[0031] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram of the process for DIC strain detection of a display screen based on speckle images provided in an embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of the structure of the display screen DIC strain detection device based on speckle images provided in an embodiment of the present invention.

[0034] Figure 3 This is a schematic diagram of the folding process of a flexible display screen provided in an embodiment of the present invention.

[0035] Figure label:

[0036] 10-LED light source, 11-camera, 12-sample to be tested, 13-stage, 14-computer. Detailed Implementation

[0037] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0038] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0039] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0040] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0041] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0042] The terms "first" and "second" used in the embodiments are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permissible. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein.

[0043] Using flexible displays as the test sample, such as Figures 1 to 3 The embodiments of the present invention will be described in detail below.

[0044] Example 1, please refer to Figure 2 and Figure 3 The first embodiment of the present invention provides a DIC strain detection device, including a light source emitting device, a display control device, an image acquisition device, and an image processing device; wherein, the light source emitting device includes an LED light source 10, which is symmetrically placed on both sides of the center normal of the test sample 12, and illuminates the test sample 12 at different angles in the device, providing full-field illumination for the detection device. The test sample 12 is a flexible display screen.

[0045] The image acquisition device includes a high-speed camera 11 for acquiring a series of digital images of the entire bending motion of the test sample 12. Preferably, the high-speed camera 11 is a color camera, capable of capturing motion images of the test sample 12 during its bending motion at a frame rate exceeding 250 frames per second and recording them in a storage medium.

[0046] The image processing device is a computer 14, which includes storage elements for storing digital images acquired by the high-speed camera 11 and software programs for strain analysis.

[0047] In this embodiment, a stage 13 for loading the test sample 12 is also included. The stage 13 is connected to the test sample 12 by gluing and is constrained to move in a preset manner to detect the creases of the test sample 12. It is positioned perpendicular to the high-speed camera 11 to ensure that the high-speed camera 11 can accurately capture the creases of the test sample 12. The test sample 12 displays a speckle image, and the stage 13 drives the test sample 12, which already displays the speckle image, to perform an angular bending motion to detect strain at the creases.

[0048] Example 2, please refer to Figure 1 Taking flexible displays as an example, this invention provides a DIC strain detection method based on display speckle images, which can be executed by a DIC strain detection device based on display speckle images, such as... Figure 2 As shown.

[0049] S101, The display control device controls the display of speckle image on the test sample: A microcontroller (not shown in the figure) is used as the display control device. Through the assembly and communication of the microcontroller, the required speckle image is designed on the test sample 12, and then it is fixed on the stage 13 that can be bent. The microcontroller can also be replaced by a programmable controller such as a PLC or mobile phone.

[0050] In existing DIC (Discrete Indication) technology, the surface of the test sample must be covered with a random speckle pattern, also known as speckle. The speckle deforms along with the surface of the test sample, thus serving as a carrier of deformation information. In this embodiment, a microcontroller (MCU) is used to implement and maintain communication with the test sample 12. That is, the speckle image is displayed on a flexible display screen, which serves as the test sample 12. This replaces traditional physical coating methods such as spraying. The display method can solve the problem of physical coating peeling and has a better detection effect on both the surface and inner layers of the test sample 12.

[0051] The displayed speckle image should meet the following requirements: 1. Randomness: the location and shape of the speckles are random, and they can be round dots or irregularly shaped spots; 2. Non-repetitiveness: including non-repetition in size, shape, and position; 3. Isotropicity: biased towards the size of the speckles; 4. High contrast: the greater the difference in RGB values, the better. Preferably, white speckles on a black background or black speckles on a white background are effective. It should be noted that the displayed speckle image is not random black and white noise, but refers to features that can be distinguished by a computer program.

[0052] In addition to the above requirements, the speckle size should not be too large or too small, with a minimum of approximately 3-5 pixels. If it is too large, a subset of the image will only contain black or white, resulting in poor image matching. If it is too small, the camera's resolution will not be able to accurately distinguish the speckle.

[0053] The displayed speckle image can be static or refreshed at a certain frequency.

[0054] S102, Parameter calibration of DIC strain detection device: Calibrate the DIC strain detection device to accurately determine the internal and external parameters of the system; wherein, the DIC strain detection device includes a light source emitting device, an image acquisition device and a computer 14.

[0055] In this example, to obtain better accuracy results, careful alignment of the plane of the sample 12 with the parallelism of the sensor plane of the camera 11 is necessary. Images of the calibration board are captured from different viewpoints, and the internal parameters (focal length, principal point, distortion parameters) and external parameters (translation vector and rotation matrix) of the camera 11 are calculated using a bundle adjustment algorithm. The camera 11 is positioned to capture the entire measurement area, ensuring sufficient field of view for the expected movement of the sample 12. After the camera 11 is positioned, the lens focal length is adjusted to achieve a clear focus on the sample 12. Before analysis, dimensional calibration is performed using images with known actual distances. Subsequent analytical data includes both pixel and millimeter units. This invention utilizes a single camera for image acquisition and correlation matching.

[0056] If the plane of the sample 12 being measured is not parallel and aligned with the camera, or if the sample 12 is displaced in the outward direction, an error will occur. This error is equal to W / Z, where W is the displacement in the outward direction and Z is the distance between the camera and the sample 12 being measured.

[0057] Because the sample 12 moves relative to the camera in the direction of the plane, the image captured by the camera will have the problem of near objects appearing larger and far objects appearing smaller. When DIC processes the correlation, it will treat the magnified image as if the sample 12 has undergone a stretching motion and obtain the stretching strain; it will treat the reduced image as if the sample 12 has undergone a compressing motion and obtain the compressing strain.

[0058] Furthermore, this error can be minimized by reducing W or increasing Z. Optimizing the measurement setup and process can reduce W, while choosing a telephoto lens can increase Z. Once the distance is determined, the camera needs to be carefully placed directly opposite the sample 12 and at the same height as the sample 12. The camera height and alignment can be precisely checked by carefully observing along the camera or by using a ruler or level.

[0059] S103, Imaging and Acquisition: Acquire a series of digital images of the bending action of the sample 12 using an image acquisition device.

[0060] In this example, camera 11 acquires a series of digital images of the entire bending process of the sample 12 under test, and the digital images are stored in computer 14. For continuous bending tests, multiple sets of images need to be captured, usually at set time intervals. Ideally, the images should be captured smoothly and continuously, rather than sporadically. Existing DIC testing technology allows the speckle pattern to be directly applied to the surface of the sample, making it easy for the camera to acquire clear digital images.

[0061] S104, Divide the sub-region to record the basic speckle map: Divide the area to be measured of the sample 12 into a grid, that is, divide the region of interest in the image of the sample 12 before deformation, such as the crease, into several subsets, and treat each subset as rigid motion.

[0062] In this embodiment, the image of the tested sample 12 before deformation is used as the reference image. The region of interest (ROI) is selected using the square tool in the software program. An appropriate subset and step size are chosen for the selected ROI, and their values ​​are set to control the optimal data and analysis time. Based on the specific working environment and accuracy requirements, the appropriate subset and step size are set with the goal of achieving the most suitable analysis results and analysis time. The values ​​of the subset and step size depend on the pixel size of the speckle pattern in the ROI of the acquired digital image.

[0063] S105, Image Matching Calculation: For each subset in step S104, perform correlation calculations according to a predefined correlation function to find the corresponding subset in the deformed image, in order to prepare for the displacement and strain calculation of speckle.

[0064] The key to achieving non-contact deformation measurement is identifying the measurement points and tracking their coordinate positions before and after deformation. The basic principle of image matching is to determine if two related images are identical by comparing the correlation of pixels at measurement points in subsets. The correlation function reaches its peak value when the correlation between the two subsets is at its best. Commonly used correlation functions include the direct correlation function, covariance correlation function, absolute difference correlation function, minimum sum of squared differences correlation function, regularized cross-correlation function, standardized covariance function, and sum of squared differences method. By comparing the anti-interference capabilities of various functions, the standardized covariance function is preferred.

[0065] S106 calculates all subsets, derives the strain field, and obtains the deformation and strain information of the entire field.

[0066] In this embodiment, steps S103 to S106 involve using software to analyze a custom region of interest (ROI) in the image, which contains a set of calculated or measured points. Each subset includes a calculated point, and each calculated point is located at the center of the subset. Correlation calculations are performed on the subsets from the reference image to each subsequent deformed image. This correlation calculation refers to: firstly, using an interpolation function to calculate the approximation of the speckle image within the subset, where the interpolation function is a subset-based shape function; and then using a matching criterion incorporating subset weights to match the subsets in the reference image with the corresponding subsets in the deformed image.

[0067] For example, taking a 5-inch foldable screen as the test sample 12, such as... Figure 3 As shown, a microcontroller is used to set several speckle images on the foldable screen, each speckle image being 3-5 pixels in size. A 1mm*1mm area at the crease in the middle of the foldable screen is selected and divided into 100 square subsets, each subset being 100*100 micrometers in size. The step size is set to 3 time units, and each subset contains 5*5 pixels (25 pixels in total), meaning each subset contains 5-8 speckles. After 200,000 folds using the above method, a deformation greater than 1µm was measured. Therefore, this invention can achieve full-field deformation measurement with a high spatial sampling rate (reaching 0.1mm) and high measurement sensitivity (reaching 1µm), dynamically measuring the deformation of flexible screens during the folding process (time sampling rate reaching 10Hz).

[0068] Comparative Example: When using a speckle spraying method, the flexible display screen has limited bending cycles or is affected by the angle, making it prone to cracking or even peeling of the speckle coating, and making it impossible to obtain continuous images of speckle changes. Using a 5-inch foldable screen as the test sample, cracks appeared in the coating after only 1-2 bends using the speckle spraying method, affecting subsequent testing. Therefore, strain analysis data for the comparative example cannot be provided. The DIC strain detection method using speckle images in this invention can effectively avoid coating peeling.

[0069] In summary, this invention eliminates the need for physical coating methods such as spraying and dyeing. By displaying a speckle pattern on the surface of the display device, strain detection is performed on its surface and inner layers, achieving the same effect as traditional physical coating. Furthermore, when performing bending operations, there is no need to consider speckle shedding, allowing for repeated testing and making it more convenient.

[0070] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0071] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A DIC strain detection device based on display speckle images, characterized in that: This includes display control equipment, image acquisition equipment, and image processing equipment; among which, The display control device controls the display of a speckle image on the test sample; the test sample is a display device capable of displaying images; the displayed speckle image should meet the following requirements: the location of the speckle is random, the shape is random, the size, shape, and position are non-repeating, it has a contrast that can be distinguished by a computer program, and the speckle size is at least 3. 5 pixels; The image acquisition device is used to acquire image information of the entire deformation process of the sample under test. The image processing device stores the image information acquired by the image acquisition device and performs strain analysis on the image information; It also includes a light source emitting device that is set on the sample to provide full-field illumination for the detection device; The light source emitting device is an LED light source, and the LED light source illuminates the sample under test at different angles to provide a uniform light field; It also includes a stage for loading the sample to be tested, which drives the sample, which has been shown with speckle images, to bend according to a preset motion pattern and is in a perpendicular position to the image acquisition device, so as to ensure that the image acquisition device accurately captures the crease of the sample and realizes strain detection at the crease.

2. The DIC strain detection device based on display speckle images according to claim 1, characterized in that, The display control device includes one or more of the following: microcontroller, graphics processor, mobile phone, tablet computer, laptop computer, super mobile personal computer, handheld computer, netbook, and personal digital assistant.

3. The DIC strain detection device based on display speckle images according to claim 1, characterized in that, The image acquisition device is a color camera.

4. A DIC strain detection method based on display speckle images, characterized in that, Using the DIC strain detection device based on display speckle images as described in claim 1, the method includes the following steps: S101, The display control device controls the display of speckle images on the tested sample; S102, calibrate the DIC strain detection device to accurately determine the internal and external parameters of the system; S103, acquire image information of the entire deformation process of the sample under test using an image acquisition device, and store the image information using the image processing device; S104, divide the computational region in the image of the sample before deformation into subsets, and set the values ​​of subsets and step size; S105, for each subset, perform relevant calculations according to a predefined correlation function, find the corresponding subset in the deformed image, and determine the speckle strain; S106 calculates all subsets, derives the strain field, and obtains the deformation and strain information of the entire field.

5. The DIC strain detection method based on display speckle images according to claim 4, characterized in that, The display control device in step S101 is used to generate a display speckle image; the display control device has one or more computer programs, the one or more computer programs including instructions, which, when executed by the one or more control units, cause the test sample to have a display speckle image.

6. The DIC strain detection method based on display speckle images according to claim 4, characterized in that, In step S102, the image acquisition device is a camera, which captures images of the calibration board from different perspectives, and uses a bundle adjustment algorithm to calculate the camera's internal parameters, including focal length, principal point, and distortion parameters, as well as its external parameters, including translation vector and rotation matrix.

7. The DIC strain detection method based on display speckle images according to claim 4, characterized in that, The sample to be tested is loaded onto a stage, which drives the sample, which has been shown with speckle images, to bend in a preset motion pattern. This ensures that the image acquisition device accurately captures the creases of the sample and acquires a series of digital images of the entire bending process of the sample. These digital images are then stored by the image processing device.

8. The application of the DIC strain detection method based on display speckle images according to any one of claims 4 to 7 in the strain testing of flexible display screens.

9. The application according to claim 8, characterized in that, The flexible display screen includes a rollable display screen or a foldable display screen.

Images