Synchronous measurement method and system for three-dimensional displacement field of transparent soil based on binocular vision

Abstract

This application discloses a method and system for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision. This application improves upon current displacement field measurements that require full-field matching. First, a local comparison window is defined for the current speckle grid. The similarity of the speckle grid in the local comparison window at time t+1 is compared with the current speckle grid to determine its position. Then, the displacement vector of the speckle grid can be determined by directly assigning the vectors of other speckle grids with similar speckle density within the local comparison window to this displacement vector. Thus, the displacement vectors of multiple speckle grids can be obtained after calculating the displacement vector of only one speckle grid, effectively reducing the amount of data processing.

Description

Technical Field

[0001] This invention relates to the field of transparent soil displacement measurement, and in particular to a method and system for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this application set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.

[0003] In soil deformation measurement, displacement field measurement based on image technology needs to meet the real-time monitoring requirements in dynamic scenarios. Traditional measurement methods that rely on discrete tracer markers have problems such as difficulty in defining interference and poor adaptability. Although particle image velocity measurement technology does not require tracer markers, the traditional full-field matching mode requires traversing the entire image for each speckle grid to find matching targets, which involves a huge amount of computation. This results in deformation analysis and measurement not being synchronized, limiting the real-time capture of dynamic processes such as pile driving, soil displacement, and dam seepage failure. Summary of the Invention

[0004] One objective of this application is to provide a method for synchronous measurement of three-dimensional displacement fields in transparent soil based on binocular vision. This scheme improves single-frame analysis efficiency without reducing accuracy by optimizing the matching logic, enabling image acquisition, analysis, and displacement output to be completed within a single frame, thus achieving synchronization between measurement and analysis. Another objective of this application is to provide a system for synchronous measurement of three-dimensional displacement fields in transparent soil based on binocular vision.

[0005] To achieve the above objectives, this application discloses a method for synchronous measurement of the three-dimensional displacement field of transparent soil based on binocular vision. This method is applied to one of three speckle images being measured. The three speckle images are orthogonally decomposed images of the three-dimensional speckle image of transparent soil after laser illumination, acquired by a binocular vision system; including:

[0006] The speckle images at times t and t+1 are divided into grids to obtain a first image including multiple first speckle grids and a second image including multiple second speckle grids.

[0007] Based on the projection position of the currently selected first speckle grid in the first and second images, the corresponding first local comparison window and second local comparison window are delineated in the first and second images;

[0008] Based on the similarity between the currently selected first speckle grid and each second speckle grid in the second local comparison window, a second speckle grid matching the currently selected first speckle grid is determined. The displacement vector of the currently selected first speckle grid is determined according to the position of the currently matched first and second speckle grids. The displacement vectors of speckle grids whose speckle density difference with the currently selected first speckle grid is less than a set threshold are all determined as the displacement vectors of the currently selected first speckle grid.

[0009] One of the remaining first speckle grids in the first local comparison window is selected again as the currently selected first speckle grid, until all the first speckle grids in the first image have determined their displacement vectors. Based on the displacement vector of each first speckle grid, a three-dimensional displacement field of transparent soil at time t is formed to synchronously measure the three-dimensional displacement field of transparent soil.

[0010] Optionally, the step of defining the corresponding first local comparison window and second local comparison window in the first image and the second image based on the projection position of the currently selected first speckle grid in the first image and the second image includes:

[0011] Based on the center position of the currently selected first speckle grid in the first image, the same number of grids are extended in both the width and height directions of the image to form the first local comparison window.

[0012] The position of the currently selected first speckle grid in the first image is taken as its projection position in the second image. Based on the center of the projection position, the same number of grids as the first local comparison window are extended in the width and height directions of the image to form the second local comparison window.

[0013] Optionally, determining the second speckle grid matching the currently selected first speckle grid based on the similarity between the currently selected first speckle grid and each second speckle grid within the second local comparison window includes:

[0014] Extract the grayscale matrix of the currently selected first speckle grid and the grayscale matrix of each second speckle grid within the second local comparison window, respectively.

[0015] The cross-correlation coefficient method is used to multiply the pixel gray values ​​at corresponding positions in the two gray matrices one by one, and sum all the multiplication results to obtain the correlation coefficient between the currently selected first speckle grid and each second speckle grid. The correlation coefficient is used to characterize the similarity between the two.

[0016] Optionally, determining the second speckle grid that matches the currently selected first speckle grid specifically includes:

[0017] A preset similarity threshold is used to filter out second speckle grids within the second local comparison window whose correlation coefficient is greater than or equal to the similarity threshold;

[0018] If multiple second speckle grids exist after filtering, the second speckle grid with the highest correlation coefficient is selected as the matching target; if only one second speckle grid exists after filtering, that second speckle grid is directly selected as the matching target.

[0019] Optionally, determining the displacement vectors of all speckle grids whose speckle density difference with the currently selected first speckle grid is less than a set threshold as the displacement vectors of the currently selected first speckle grid includes:

[0020] Calculate the difference between the speckle density of each other first speckle grid within the first local comparison window and the speckle density of the currently selected first speckle grid;

[0021] The first speckle grid with a difference value lower than a set density threshold is selected to form a density-similar grid group;

[0022] The displacement vector of the currently selected first speckle grid is assigned one by one to each grid in the density similar grid group, and these grids are marked as grids with determined displacement vectors.

[0023] Optionally, calculating the difference between the speckle density of each other first speckle grid within the first local comparison window and the speckle density of the currently selected first speckle grid includes:

[0024] If the speckle field of transparent soil is a natural speckle field or an artificial speckle field with clear particle boundaries, the threshold segmentation algorithm is used to extract the speckle region of each first speckle grid within the first local comparison window, and the number of speckles in each speckle region is counted. The number of speckles is used as the speckle density of the corresponding first speckle grid.

[0025] If the speckle field of the transparent soil is a speckle field with uniform gray distribution, calculate the standard deviation of gray values ​​of all pixels in each first speckle grid within the first local comparison window, and use the standard deviation of gray values ​​as the speckle density of the corresponding first speckle grid.

[0026] Optionally, selecting one of the remaining first speckle grids in the first local comparison window as the currently selected first speckle grid includes:

[0027] According to the grid arrangement order in the image width or height direction, traverse all the first speckle grids in the first local comparison window one by one;

[0028] Check the displacement vector determination status of each first speckle grid, select the first first speckle grid that has not been marked as having a determined displacement vector, and re-select it as the currently selected first speckle grid.

[0029] Optionally, determining the displacement vector of the currently selected first speckle grid based on the positions of the currently matched first and second speckle grids includes:

[0030] Obtain the width and height coordinates of the currently selected first speckle grid in the first image;

[0031] Obtain the width and height coordinates of the matching second speckle grid in the second image;

[0032] Calculate the coordinate difference between the two grids in the width direction and the coordinate difference in the height direction, and combine the coordinate difference in the width direction and the coordinate difference in the height direction to form the displacement vector of the currently selected first speckle grid.

[0033] Optionally, the displacement vector based on each first speckle grid includes:

[0034] An interpolation algorithm is used to optimize the displacement vector of each first speckle grid with sub-pixel accuracy;

[0035] Calculate the displacement gradient between two adjacent speckle grids with determined displacement vectors. If the displacement gradient of a certain grid exceeds the preset gradient threshold, the grid is reselected as the currently selected first speckle grid. Repeat the steps of similarity calculation, target determination and displacement vector assignment to correct its displacement vector.

[0036] Outliers in all displacement vectors were eliminated using statistical criteria.

[0037] The displacement vectors of all the corrected first speckle grids are integrated according to the order of their positions in the first image to form a complete three-dimensional displacement field of the transparent soil at time t.

[0038] Another aspect of this application discloses a synchronous measurement system for the three-dimensional displacement field of transparent soil based on binocular vision. The synchronous measurement method for the three-dimensional displacement field of transparent soil is applied to one of three speckle images currently being measured. The three speckle images are orthogonally decomposed images of the three-dimensional speckle image of transparent soil after laser illumination, acquired by the binocular vision system; including:

[0039] Image acquisition module: The speckle images at times t and t+1 are divided into grids to obtain a first image including multiple first speckle grids and a second image including multiple second speckle grids. Based on the projection position of the currently selected first speckle grid in the first image and the second image, the corresponding first local comparison window and second local comparison window are delineated in the first image and the second image.

[0040] Similarity segmentation module: Based on the similarity between the currently selected first speckle grid and each second speckle grid in the second local comparison window, determine the second speckle grid that matches the currently selected first speckle grid. Determine the displacement vector of the currently selected first speckle grid according to the position of the currently matched first and second speckle grids. Determine the displacement vector of speckle grids whose speckle density difference with the currently selected first speckle grid is less than a set threshold as the displacement vector of the currently selected first speckle grid. Displacement vector module: Select one of the remaining first speckle grids in the first local comparison window as the currently selected first speckle grid until all first speckle grids in the first image have determined displacement vectors. Based on the displacement vector of each first speckle grid, form a three-dimensional displacement field of transparent soil at time t, so as to synchronously measure the three-dimensional displacement field of transparent soil.

[0041] The beneficial effects of this application are as follows:

[0042] This application first discovers that in transparent soil deformation scenarios, similar speckle density and location in speckle images indicate that the corresponding transparent soils are in similar stress environments and locations, thus their deformations are similar. Based on this discovery, this application improves the current displacement field measurement that requires full-field matching. First, a local comparison window is defined for the current speckle grid. The similarity of the current speckle grid with the local comparison window at time t+1 is compared. After determining the position of the speckle grid at time t+1, the displacement vector of that speckle grid can be determined. Then, the displacement vector of the speckle grid is determined. The vectors of other speckle grids with similar speckle density in the comparison window are directly assigned to the displacement vector. Thus, after calculating the displacement vector of only one speckle grid, the displacement vectors of multiple speckle grids can be obtained. Then, the process is repeated until the displacement vectors of all speckle grids are determined. In this way, global matching is not required, which effectively reduces the amount of data processing and shortens the analysis time. While maintaining accuracy, the time difference between measurement and analysis is shortened, and the measurement and analysis are synchronized. This provides a synchronous measurement scheme for the three-dimensional displacement field of transparent soil based on binocular vision. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0044] Figure 1 This illustration shows one of the flowcharts of the synchronous measurement method for three-dimensional displacement field of transparent soil based on binocular vision, according to an embodiment of this application.

[0045] Figure 2 This is the second flowchart illustrating the synchronous measurement method for three-dimensional displacement field of transparent soil based on binocular vision, according to an embodiment of this application.

[0046] Figure 3(a) shows one of the schematic diagrams of the method for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision according to an embodiment of this application;

[0047] Figure 3(b) shows a second schematic diagram of the method for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision according to an embodiment of this application;

[0048] Figure 4 This is the third flowchart illustrating the synchronous measurement method for three-dimensional displacement field of transparent soil based on binocular vision, according to an embodiment of this application.

[0049] Figure 5 The fourth flowchart illustrates the synchronous measurement method for three-dimensional displacement field of transparent soil based on binocular vision, according to an embodiment of this application.

[0050] Figure 6 The fifth illustration shows a flowchart of the synchronous measurement method for three-dimensional displacement field of transparent soil based on binocular vision, according to an embodiment of this application.

[0051] Figure 7 The sixth illustration shows a flowchart of the synchronous measurement method for three-dimensional displacement field of transparent soil based on binocular vision, according to an embodiment of this application.

[0052] Figure 8 The third schematic diagram of the method for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision, according to an embodiment of this application, is shown.

[0053] Figure 9 The seventh flowchart illustrates the synchronous measurement method for three-dimensional displacement field of transparent soil based on binocular vision, according to an embodiment of this application.

[0054] Figure 10 This is illustrated as the eighth flowchart of the synchronous measurement method for three-dimensional displacement field of transparent soil based on binocular vision, according to an embodiment of this application.

[0055] Figure 11 The ninth illustration shows a flowchart of the synchronous measurement method for three-dimensional displacement field of transparent soil based on binocular vision, according to an embodiment of this application.

[0056] Figure 12(a) shows one specific embodiment of the synchronous measurement method for three-dimensional displacement field of transparent soil based on binocular vision according to the present application;

[0057] Figure 12(b) shows a second specific embodiment of the synchronous measurement method for three-dimensional displacement field of transparent soil based on binocular vision according to the present application.

[0058] Figure 13 A schematic diagram of the structure of the transparent soil three-dimensional displacement field synchronous measurement system based on binocular vision, according to an embodiment of this application, is shown. Detailed Implementation

[0059] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0060] It should be understood that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0061] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0062] In order to solve at least one of the problems existing in the prior art, according to one aspect of this application, such as Figure 1 As shown, this embodiment discloses a method for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision, including:

[0063] S100: The speckle images at times t and t+1 are divided into grids to obtain a first image including multiple first speckle grids and a second image including multiple second speckle grids;

[0064] S200: Based on the projection position of the currently selected first speckle grid in the first image and the second image, delineate the corresponding first local comparison window and the second local comparison window in the first image and the second image.

[0065] In the above scheme, it should be noted that the binocular camera in this application displays three-dimensional information. Therefore, all images involved in this application are three-dimensional images, and the meshes are all three-dimensional meshes. When calculating the displacement, each plane can be calculated separately. Then, the three orthogonal planes (xy-axis plane, yz-axis plane, and xz-axis plane) are calculated repeatedly and then vectored together. That is, the three-dimensional speckle image is first converted into three planar speckle images, and then the processing of this application is performed on the three planar speckle images respectively. It can be understood that the synchronous measurement method of the three-dimensional displacement field of transparent soil in this application is applied to one of the three speckle images being measured. The three speckle images are the orthogonally decomposed images of the three-dimensional speckle image of transparent soil after laser irradiation acquired by the binocular vision system. It should be further noted that each sampling frame in this application needs to repeat the specific processing of the three speckle images to obtain the three-dimensional displacement vector. The embodiments of this application are described in detail below.

[0066] The acquired continuous speckle image is divided into a first speckle grid and a second speckle grid according to a preset rule. Since continuous speckle images contain a large amount of information, performing displacement analysis on the entire image would result in excessive computation. However, by discretizing the image into multiple small units through grid partitioning, each unit can be analyzed independently. In this embodiment, the partitioning method can be uniform, that is, dividing the image at equal intervals in both the width and height directions, or adaptive partitioning based on the speckle distribution density, for example, using finer grids in dense speckle areas and coarser grids in sparse speckle areas.

[0067] Next, corresponding first and second local comparison windows are defined. Since the displacement of the same speckle pattern at adjacent time points will not exceed a certain range, it is unnecessary to search for a matching speckle grid in the entire image; only a local area needs to be defined around the currently selected speckle grid. Specifically, the first local comparison window is the region centered on the currently selected speckle grid in the image at time t, and the second local comparison window is the region centered on the projection position of the currently selected speckle grid at time t+1 in the image. Limiting the comparison range reduces the number of similarity calculations, thereby improving computational efficiency. The shapes of the first and second local comparison windows are typically rectangular, but other shapes such as circles can also be used depending on the actual situation; this application is not limited to these.

[0068] It should be noted that transparent soil itself possesses certain optical properties. When laser light shines on the surface or interior of transparent soil, the laser beam is reflected and scattered by the particles, pores, and other structures within the soil, forming a series of irregular speckle patterns. The speckle distribution at different times is captured using image acquisition equipment to obtain speckle images at time t and t+1.

[0069] S300: Based on the similarity between the currently selected first speckle grid and each second speckle grid in the second local comparison window, determine the second speckle grid that matches the currently selected first speckle grid, and determine the displacement vector of the currently selected first speckle grid according to the position of the currently matched first and second speckle grids. All the displacement vectors of speckle grids whose speckle density difference with the currently selected first speckle grid is lower than a set threshold are determined as the displacement vector of the currently selected first speckle grid.

[0070] In the above scheme, the speckle distribution within different speckle grids can be quantified by density parameters, and grids with similar speckle densities will produce similar displacements. This application's embodiment uses a threshold segmentation algorithm to extract speckle regions. A grayscale threshold is set, and pixels with grayscale values ​​greater than the threshold are identified as speckle regions, while pixels with grayscale values ​​less than the threshold are identified as background regions. The speckle density is obtained by counting the number of speckles within each speckle region. For speckle fields with uniform grayscale distribution, the grayscale difference between speckles and the background is small, and the boundaries are unclear. The speckle density can be characterized by calculating the standard deviation of grayscale values ​​for all pixels in each speckle grid. The larger the standard deviation, the greater the fluctuation of grayscale values ​​within the grid, and the denser the speckle distribution.

[0071] The displacement vector represents the magnitude and direction of the displacement of the currently selected speckle mesh from time t to time t+1. First, the width and height coordinates of the currently selected speckle mesh in the image at time t are obtained. Then, the width and height coordinates of the matched speckle mesh in the image at time t+1 are obtained. The differences between the two coordinates in the width and height directions are then calculated, where the difference in the width direction represents the horizontal displacement, and the difference in the height direction represents the vertical displacement. Transparent soil regions corresponding to meshes with similar speckle density have similar physical and mechanical properties. Under the same external force, the displacement patterns of these regions will remain consistent. Therefore, the determined displacement vector can be assigned to meshes with similar density without the need for separate similarity matching calculations, thus further reducing the computational load.

[0072] S400: Select one of the remaining first speckle grids in the first local comparison window as the currently selected first speckle grid, until all the first speckle grids in the first image have determined their displacement vectors. Based on the displacement vector of each first speckle grid, form a three-dimensional displacement field of transparent soil at time t, so as to synchronously measure the three-dimensional displacement field of transparent soil.

[0073] In the above scheme, by processing each grid with an undetermined displacement vector one by one, all speckle grids in the image can obtain their corresponding displacement vectors, thus forming a complete displacement field. Once the displacement vectors of all grids within a local comparison window have been determined, new grids are selected from adjacent unprocessed areas, and the local comparison window is redefined. After the displacement vectors of all speckle grids are determined, these discrete displacement vectors are integrated according to the positions of their corresponding grids in the image.

[0074] In alternative implementations, such as Figure 2 As shown, the step of defining the corresponding first local comparison window and second local comparison window in the first image and the second image based on the projection position of the currently selected first speckle grid in the first image and the second image includes:

[0075] S110: Using the center position of the currently selected first speckle grid in the first image as a reference, extend the same number of grids in both the width and height directions of the image to form the first local comparison window.

[0076] In the above scheme, each speckle grid has its corresponding spatial range. The start and end coordinates of the grid in the image width direction are obtained, and the average of the two is the center coordinate in the width direction. Similarly, the start and end coordinates of the grid in the height direction are obtained, and the average of the two is the center coordinate in the height direction. The center coordinates in both directions together constitute the center position of the grid. Using this center position as a reference, the same number of grids are extended in both the image width and height directions to define the first local comparison window.

[0077] In a specific example, as shown in Figure 3(a), A is the first image at time t, and a is the first speckle grid. This forms the first local comparison window. Find the start and end coordinates of 'a' in the width direction of the first image A, and take their average; this is the center of 'a' in the width direction. Then find the start and end coordinates of 'a' in the height direction, and take their average; this is the center of 'a' in the height direction. Combining the centers of 'a' in the width and height directions, we obtain the center position of the speckle grid 'a'. Using the center position of 'a' obtained above as a reference, extend the same number of grids in both the width and height directions of the first image A. The area covered by this extension forms the first local comparison window. .

[0078] It should be noted that the number of extended grids can be adjusted according to the expected displacement of the transparent soil. When the expected displacement is large, the number of extensions can be increased appropriately, and when the expected displacement is small, the number of extensions can be decreased appropriately. This application is not limited to this.

[0079] S120: The position of the currently selected first speckle grid in the first image is taken as its projection position in the second image. Based on the center of the projection position, the same number of grids as the first local comparison window are extended in the width and height directions of the image to form the second local comparison window.

[0080] In the above scheme, based on the assumption of continuity of speckle displacement between adjacent time points, that is, the position of the speckle grid at time t+1 will not differ too much from its position at time t, the grid position at time t is used as its estimated position at time t+1, which is the projected position. A second local comparison window is defined with the center of the projected position as the reference, and the matching range is limited by the estimated position, thereby quickly finding the corresponding matching grid.

[0081] In a specific example, as shown in Figures 3(a) and 3(b), A is the first image at time t, and a is the first speckle grid. B is the first local comparison window, and B is the second image at time t+1. The second local comparison window. The position of 'a' in the first image at time t is taken as its projection position in the second image B at time t+1. Based on the same principle, the average of the start and end coordinates of this projection position in the width and height directions is taken to obtain the center of this projection position. Using this center as a reference, the first local comparison window is extended in both the width and height directions. With the same number of grids, the extended area becomes the second local comparison window. .

[0082] In alternative implementations, such as Figure 4 As shown, determining the second speckle grid that matches the currently selected first speckle grid based on the similarity between the currently selected first speckle grid and each second speckle grid within the second local comparison window includes:

[0083] S210: Extract the grayscale matrix of the currently selected first speckle grid and the grayscale matrix of each second speckle grid in the second local comparison window.

[0084] In the above scheme, the pixel range of each speckle grid is first determined, that is, the range of pixel coordinates in the width and height directions of the grid in the image. Then, the grayscale value of each pixel within this range is read one by one in order from left to right and from top to bottom. These grayscale values ​​are then arranged into a two-dimensional matrix in the order of reading. The grayscale matrix can represent the grayscale distribution of the speckle grid, thereby enabling similarity comparison through calculation.

[0085] S220: Using the cross-correlation coefficient method, the pixel gray values ​​at corresponding positions in the two gray matrices are multiplied one by one, and all multiplication results are summed to obtain the correlation coefficient between the currently selected first speckle grid and each second speckle grid. The correlation coefficient is used to characterize the similarity between the two.

[0086] In the above scheme, for speckle grids formed by the same speckle at different times, the pixel gray values ​​at the same position in the corresponding gray-level matrix will be correlated. The pixel gray values ​​at corresponding positions in the two gray-level matrices are multiplied one by one. Positions with both large or both small pixel gray values ​​have larger product values, thus highlighting positions with strong correlation. Positions with one strong and one weak pixel gray value have smaller product values, thus weakening positions with weak correlation. The sum of all multiplication results is then calculated; the larger the sum, the stronger the correlation between the two gray-level matrices, i.e., the higher the similarity between the two speckle grids. The correlation coefficient quantifies the similarity of the gray-level distributions of the two speckle grids. The correlation coefficient ranges from [-1, 1]. The closer the correlation coefficient is to 1, the more similar the gray-level distributions of the two grids are, and the higher the probability of corresponding to the same speckle. When the correlation coefficient is close to 0 or negative, it indicates that the gray-level distributions of the two grids are unrelated, and the possibility of matching can be directly ruled out.

[0087] In alternative implementations, such as Figure 5 As shown, determining the second speckle grid that matches the currently selected first speckle grid specifically includes:

[0088] S230: Set a preset similarity threshold to filter out second speckle grids within the second local comparison window whose correlation coefficient is greater than or equal to the similarity threshold.

[0089] In the above scheme, the correlation coefficients of all second speckle grids in the second local comparison window are extracted one by one. The relationship between each correlation coefficient and the preset similarity threshold is compared one by one. All second speckle grids with correlation coefficients ≥ similarity threshold are retained, and grids with correlation coefficients < similarity threshold are removed.

[0090] It should be noted that a preset similarity threshold is used to distinguish between similar and dissimilar speckle grids. The preset similarity threshold is determined by a large number of transparent soil speckle matching experiments and data analysis combined with the characteristics of different speckle field types. At the same time, the preset similarity threshold can be adjusted by the actual speckle image quality, which is not limited in this application.

[0091] S240: If multiple second speckle grids exist after filtering, select the second speckle grid with the highest correlation coefficient as the matching target; if only one second speckle grid exists after filtering, directly use that second speckle grid as the matching target.

[0092] In the above scheme, when there are multiple grids in the candidate matching grid set, the second speckle grid with the highest correlation coefficient is selected as the matching target. The magnitude of the correlation coefficient reflects the similarity of the gray-level distribution. The grid with the highest correlation coefficient is closest to the gray-level characteristics of the currently selected first speckle grid, and the probability of the same speckle being imaged at time t+1 is the highest. When there is only one second speckle grid in the candidate matching grid set, this second speckle grid has already met the condition of high similarity, and there are no other second speckle networks, so this second speckle grid is directly used as the matching target.

[0093] In alternative implementations, such as Figure 6 As shown, determining the displacement vectors of all speckle grids whose speckle density difference with the currently selected first speckle grid is less than a set threshold as the displacement vectors of the currently selected first speckle grid includes:

[0094] S270: Calculate the difference between the speckle density of each other first speckle grid within the first local comparison window and the speckle density of the currently selected first speckle grid.

[0095] In the above scheme, the speckle density of the currently selected speckle grid is first obtained. Then, the speckle density of each other speckle grid within the first local comparison window is obtained. The speckle density difference is then obtained by subtracting the speckle density of the currently selected grid from the speckle density of the other grids and taking the absolute value. The degree of difference in speckle density between different grids is determined by the density difference. The smaller the difference, the more similar the speckle distribution characteristics between the grids.

[0096] S280: Select the first speckle grid with a difference value lower than the set density threshold to form a density similar grid group.

[0097] In the above scheme, by analyzing the displacement characteristics of transparent soil, a critical value for density difference is determined. When the difference in speckle density between two grids is lower than this critical value, the transparent soil regions corresponding to these two grids are deemed to have similar displacement characteristics, and the displacement vector can be reused. When the density difference is higher than this critical value, the displacement characteristics of the transparent soil regions corresponding to these two grids are deemed to be significantly different, and the displacement vector cannot be reused. This application embodiment avoids displacement errors caused by excessive density differences by setting a density threshold.

[0098] It should be noted that the density threshold can be adjusted according to factors such as the type of speckled field and the characteristics of transparent soil, and this application is not limited to this.

[0099] S290: Assign the displacement vector of the currently selected first speckle grid to each grid in the density similar grid group, and mark these grids as grids with determined displacement vectors.

[0100] In the above scheme, speckle grids with density differences below a set density threshold are selected to form density-similar grid groups. Grids with similar displacement characteristics are grouped together, facilitating unified displacement vector assignment. The determined displacement vectors of the currently selected speckle grids are assigned one by one to each grid in the density-similar grid group, eliminating the need for separate similarity matching and displacement calculations for these grids, allowing them to directly obtain their displacement vectors. Simultaneously, these grids are marked as having determined displacement vectors to avoid subsequent redundant processing. By reusing displacement vectors, redundant calculations can be reduced without compromising measurement accuracy.

[0101] In a specific example, such as Figure 7 As shown, A is the first image at time t, and a is the first speckle grid. This is the first local comparison window, c is... A grid with a determined displacement vector and similar density within the range satisfies the condition that the difference in speckle density between it and the first speckle grid a is less than a set threshold. Therefore, the displacement vector of the first speckle grid a has already been assigned, so there is no need to calculate the displacement of c separately.

[0102] In alternative implementations, such as Figure 8 As shown, calculating the difference between the speckle density of each other first speckle grid within the first local comparison window and the speckle density of the currently selected first speckle grid includes:

[0103] S250: If the speckle field of transparent soil is a natural speckle field or an artificial speckle field with clear particle boundaries, a threshold segmentation algorithm is used to extract the speckle region of each first speckle grid within the first local comparison window, and the number of speckles in each speckle region is counted. The number of speckles is used as the speckle density of the corresponding first speckle grid.

[0104] In the above scheme, the natural speckle field is formed naturally by the particle and pore structure of the transparent soil under laser irradiation. The grayscale difference between the boundaries of the speckle particles and the background is significant and clearly distinguishable. The artificial speckle field is formed by adding specific speckle materials to the transparent soil, and the boundaries of the speckle particles also have high clarity. For this type of speckle field, a threshold segmentation algorithm is used to separate the speckle region from the background region. Utilizing the grayscale difference between the speckle region and the background region, a suitable grayscale threshold is set. Pixels with grayscale values ​​higher than the threshold are classified as speckle regions, and pixels with grayscale values ​​lower than the threshold are classified as background regions. The number of speckle spots within a speckle region is obtained by counting the number of pixels within that region. A higher number of speckle spots indicates a denser speckle distribution within the grid, and consequently, a denser distribution of transparent soil particles.

[0105] S260: If the speckle field of the transparent soil is a speckle field with uniform gray distribution, calculate the standard deviation of gray values ​​of all pixels in each first speckle grid within the first local comparison window, and use the standard deviation of gray values ​​as the speckle density of the corresponding first speckle grid.

[0106] In the above scheme, the speckle field with uniform grayscale distribution suffers from small and uniform particle size of the transparent soil, or factors such as laser irradiation angle and intensity, resulting in small grayscale differences between the speckles and the background. The boundaries of the speckle particles are unclear, making it impossible to effectively separate the speckle region from the background region using a threshold segmentation algorithm. For this type of speckle field, the denser the speckle distribution, the more drastic the change in pixel grayscale values ​​and the larger the standard deviation; conversely, the sparser the speckle distribution, the smoother the change in pixel grayscale values ​​and the smaller the standard deviation. Therefore, the density of the speckles can be reflected by calculating the standard deviation of the grayscale values ​​of all pixels in each speckle grid.

[0107] In alternative implementations, such as Figure 9 As shown, selecting one of the remaining first speckle grids in the first local comparison window as the currently selected first speckle grid includes:

[0108] S310: According to the grid arrangement order in the image width or height direction, traverse all the first speckle grids in the first local comparison window one by one.

[0109] In the above scheme, after the grid is divided, all speckle grids present a regular arrangement in the image, with the width direction being the horizontal direction and the height direction being the vertical direction. The grid is traversed according to the grid arrangement order in the width or height direction of the image, so that each grid is processed in turn to avoid omissions.

[0110] It should be noted that the above-mentioned traversal order and selection rules are used in the embodiments of this application. This application is not limited to this. Other traversal orders such as diagonal direction and spiral direction can also be used, as long as it can ensure that all grids are processed.

[0111] S320: Check the displacement vector determination status of each first speckle grid, select the first first speckle grid that has not been marked as having a determined displacement vector, and re-select it as the currently selected first speckle grid.

[0112] In the above scheme, the displacement vector determination status of each speckle mesh is checked to distinguish between processed and unprocessed meshes. Meshes with determined displacement vectors are marked, while unmarked meshes are unprocessed. The first unmarked mesh is selected as the new currently selected mesh. When all meshes within the local comparison window have determined displacement vectors, the system searches for adjacent speckle mesh regions that contain undetermined displacement vectors. If an undetermined displacement vector exists within an adjacent region, an unprocessed mesh from that region is selected as the new currently selected mesh.

[0113] In a specific example, such as Figure 7 As shown, A is the first image at time t, and a is the first speckle grid. c is the first local comparison window. A first local comparison window of meshes with similar density and determined displacement vectors within a given range. Within the range, apart from the meshes c with displacement vectors that have been determined to have similar densities, the remaining meshes are unprocessed meshes. The first unmarked mesh in the unprocessed meshes is selected as the new currently selected mesh.

[0114] Referring to Figure 3(a), in the first image A, firstly determine the first local window in the first speckle grid a. Then at time t+1, as shown in Figure 3(b), from the second image B, extending in the width and height directions from the projection position of the first speckle grid a in the second image B, and intersecting with the first local window. The same number of grids form a second local window. Then in the second local window In the second local window, the second speckle grid corresponding to the first speckle grid a is matched. Each second speckle grid within the first image A is matched for similarity with the first speckle grid a in the first image A. The speckle grid with a similarity higher than a set threshold or the highest similarity is selected and used as the second speckle grid b after the displacement of the first speckle grid a at time t+1. The coordinate difference between the selected second speckle grid b and the first speckle grid a is the displacement vector x of the first speckle grid a (as shown in Figure 3(b), where the displacement vector of the first speckle grid a is a grid cell). Then, the displacement vectors of other speckle grids with low speckle density differences from the first speckle grid a are directly determined as the displacement vector x of the first speckle grid a, i.e., within the first local window. The displacement vector of other speckle grids with similar speckle density within the grid is x. Finally, combined with... Figure 7 As shown, the comparison window is re-established from the first local comparison window. In the range, apart from the grid c with a similar density and a determined displacement vector, one of the remaining first speckle grids is selected again as the currently selected first speckle grid, until all the first speckle grids in the first image A at time t have determined their displacement vectors.

[0115] It should be noted that, in the embodiments of this application, the evaluation indicators of similarity include speckle density, speckle distribution, and the shape of each speckle, and this application does not impose any limitations on these.

[0116] This application discovers that in transparent soil deformation scenarios, similar speckle density and location in speckle images indicate that the corresponding transparent soils are in similar stress environments and locations, resulting in similar deformation. Therefore, through the above method, this application does not require extensive calculation of the matching degree of each grid, nor does it require global matching of each grid (i.e., similarity matching between each grid and all grids). By defining a local comparison window for the current speckle grid, comparing the local comparison window at time t+1 with the current speckle grid for similarity matching, and determining the position of the speckle grid at time t+1, the displacement vector of that speckle grid can be determined. Then, the vectors of other speckle grids with similar speckle density in the local comparison window of the speckle grid are directly assigned as the displacement vector. Thus, after calculating the displacement vector of only one speckle grid, the displacement vectors of multiple speckle grids can be obtained, effectively reducing the amount of data processing.

[0117] In alternative implementations, such as Figure 10 As shown, determining the displacement vector of the currently selected first speckle grid based on the positions of the currently matched first and second speckle grids includes:

[0118] S330: Obtain the width and height coordinates of the currently selected first speckle grid in the first image, and obtain the width and height coordinates of the matched second speckle grid in the second image.

[0119] In the above scheme, by determining the pixel range of the matching grid in the second image, the coordinates of the center pixel are calculated to obtain the width and height coordinates of the second speckle grid in the second image. The width and height coordinates in the second image are compared with the coordinates in the first image to determine the positional changes of the grid at the two time points.

[0120] S340: Calculate the coordinate difference between the two grids in the width direction and the coordinate difference in the height direction, and combine the coordinate difference in the width direction and the coordinate difference in the height direction to form the displacement vector of the currently selected first speckle grid.

[0121] In the above scheme, the coordinate values ​​in the second image are subtracted from the coordinate values ​​in the first image. If the difference is positive, it indicates that the mesh has shifted in the positive direction. If the difference is negative, it indicates that the mesh has shifted in the negative direction. If the difference is zero, it indicates that the mesh has not shifted in the direction.

[0122] In alternative implementations, such as Figure 11 As shown, the displacement vector based on each first speckle grid includes:

[0123] S350: The displacement vector of each first speckle grid is optimized with sub-pixel accuracy using an interpolation algorithm.

[0124] In the above scheme, the displacement of the speckle grid is a sub-pixel displacement between two pixels. Calculating the displacement vector using only integer pixel coordinates would result in insufficient displacement accuracy. By calculating the sub-pixel displacement value using displacement data from adjacent grids, the accuracy of displacement measurement is improved. Based on the continuity of displacement, interpolation is performed between discrete displacement data points to obtain the displacement value at intermediate positions.

[0125] It should be noted that, in the embodiments of this application, the interpolation algorithm can be linear interpolation, bilinear interpolation, cubic spline interpolation, etc., and this application is not limited to these, as long as it can achieve sub-pixel accuracy optimization.

[0126] S360: Calculate the displacement gradient between two adjacent speckle grids with determined displacement vectors. If the displacement gradient of a certain grid exceeds the preset gradient threshold, the grid is reselected as the currently selected first speckle grid. Repeat the steps of similarity calculation, target determination, and displacement vector assignment to correct its displacement vector.

[0127] In the above scheme, the displacement vectors of two adjacent speckle grids differ, and the rate of change of this difference is the displacement gradient. The displacement vectors of two adjacent speckle grids are obtained by dividing the difference in displacement vectors between the two grids. Anomalies in the displacement field are detected through displacement gradient detection. Because the displacement of normal transparent soil is continuous and gradual, the displacement gradients of adjacent grids will not exceed a certain range. If the displacement gradient of a certain grid exceeds a preset gradient threshold, the displacement vector of that grid may have an error and needs to be corrected. The correction process involves reusing that grid as the currently selected speckle grid, repeating the similarity calculation, matching target determination, and displacement vector assignment steps, and re-matching and calculating to eliminate possible matching errors from the past, obtaining a more accurate displacement vector, thereby restoring the displacement gradient to the normal range.

[0128] S370: Using statistical criteria to remove coarse errors in all displacement vectors, the displacement vectors of all the corrected first speckle grids are integrated according to the order of their positions in the first image to form a complete three-dimensional displacement field of transparent soil at time t.

[0129] In the above scheme, outliers are anomalous data points in the displacement vector that significantly deviate from the normal distribution range, caused by factors such as image noise, matching errors, and calculation errors. Normal displacement data exhibits a concentrated distribution, while outliers deviate significantly from this distribution.

[0130] In the above scheme, based on the coordinate position of each speckle grid in the first image, the corresponding corrected displacement vectors are arranged in coordinate order to form a two-dimensional displacement data matrix corresponding to the image pixel size. This matrix is ​​the complete three-dimensional displacement field of transparent soil at time t.

[0131] In a specific example, Figure 12(a) shows the completed transparent soil model, and Figure 12(b) shows the original speckle image produced after laser irradiation of the transparent soil model.

[0132] Another aspect of this application discloses a synchronous measurement system for three-dimensional displacement fields of transparent soil based on binocular vision, such as... Figure 13 As shown, it includes:

[0133] Image acquisition module 11: performs grid division on the speckle images at times t and t+1 respectively to obtain a first image including multiple first speckle grids and a second image including multiple second speckle grids. Based on the projection position of the currently selected first speckle grid in the first image and the second image, it delineates the corresponding first local comparison window and second local comparison window in the first image and the second image.

[0134] Similarity segmentation module 12: Based on the similarity between the currently selected first speckle grid and each second speckle grid in the second local comparison window, determine the second speckle grid that matches the currently selected first speckle grid, determine the displacement vector of the currently selected first speckle grid according to the position of the currently matched first and second speckle grids, and determine the displacement vector of the speckle grid whose speckle density difference with the currently selected first speckle grid is lower than a set threshold as the displacement vector of the currently selected first speckle grid;

[0135] Displacement vector module 13: Select one of the remaining first speckle grids in the first local comparison window as the currently selected first speckle grid, until all first speckle grids in the first image have determined displacement vectors, and form a three-dimensional displacement field of transparent soil at time t based on the displacement vector of each first speckle grid, so as to synchronously measure the three-dimensional displacement field of transparent soil.

[0136] Since the principle behind this system's problem-solving is similar to the methods described above, the implementation of this system can be found in the implementation of the methods, and will not be repeated here.

[0137] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision, characterized in that, The method for synchronous measurement of the three-dimensional displacement field of transparent soil is applied to one of the three speckle images currently being measured. The three speckle images are the three-dimensional speckle images of transparent soil after laser irradiation acquired by the binocular vision system and decomposed orthogonally. The method for synchronous measurement of the three-dimensional displacement field of transparent soil includes: The speckle images at times t and t+1 are divided into grids to obtain a first image including multiple first speckle grids and a second image including multiple second speckle grids. Based on the projection position of the currently selected first speckle grid in the first and second images, the corresponding first local comparison window and second local comparison window are delineated in the first and second images; Extract the grayscale matrix of the currently selected first speckle grid and the grayscale matrix of each second speckle grid in the second local comparison window respectively; use the cross-correlation coefficient method to multiply the pixel grayscale values ​​at corresponding positions in the two grayscale matrices one by one, and sum all the multiplication results to obtain the correlation coefficient between the currently selected first speckle grid and each second speckle grid. Based on the correlation coefficient, a second speckle grid matching the currently selected first speckle grid is determined. The displacement vector of the currently selected first speckle grid is determined according to the positions of the currently matched first and second speckle grids. The displacement vectors of speckle grids whose speckle density difference with the currently selected first speckle grid is less than a set threshold are all determined as the displacement vectors of the currently selected first speckle grid. One of the remaining first speckle grids in the first local comparison window is selected again as the currently selected first speckle grid, until all the first speckle grids in the first image have determined their displacement vectors. Based on the displacement vector of each first speckle grid, a three-dimensional displacement field of transparent soil at time t is formed to synchronously measure the three-dimensional displacement field of transparent soil.

2. The method for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision according to claim 1, characterized in that, The step of defining the corresponding first local comparison window and second local comparison window in the first image and the second image based on the projection position of the currently selected first speckle grid in the first image and the second image includes: Based on the center position of the currently selected first speckle grid in the first image, the same number of grids are extended in both the width and height directions of the image to form the first local comparison window. The position of the currently selected first speckle grid in the first image is taken as its projection position in the second image. Based on the center of the projection position, the same number of grids as the first local comparison window are extended in the width and height directions of the image to form the second local comparison window.

3. The method for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision according to claim 1, characterized in that, The determination of the second speckle grid that matches the currently selected first speckle grid includes: A preset similarity threshold is used to filter out second speckle grids within the second local comparison window whose correlation coefficient is greater than or equal to the similarity threshold; If multiple second speckle grids exist after filtering, the second speckle grid with the highest correlation coefficient is selected as the matching target; if only one second speckle grid exists after filtering, that second speckle grid is directly selected as the matching target.

4. The method for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision according to claim 1, characterized in that, The step of determining the displacement vectors of all speckle grids whose speckle density difference with the currently selected first speckle grid is less than a set threshold as the displacement vectors of the currently selected first speckle grid includes: Calculate the difference between the speckle density of each other first speckle grid within the first local comparison window and the speckle density of the currently selected first speckle grid; The first speckle grid with a difference value lower than a set density threshold is selected to form a density-similar grid group; The displacement vector of the currently selected first speckle grid is assigned one by one to each grid in the density similar grid group, and these grids are marked as grids with determined displacement vectors.

5. The method for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision according to claim 4, characterized in that, The calculation of the difference between the speckle density of each other first speckle grid within the first local comparison window and the speckle density of the currently selected first speckle grid includes: If the speckle field of transparent soil is a natural speckle field or an artificial speckle field with clear particle boundaries, the threshold segmentation algorithm is used to extract the speckle region of each first speckle grid within the first local comparison window, and the number of speckles in each speckle region is counted. The number of speckles is used as the speckle density of the corresponding first speckle grid. If the speckle field of the transparent soil is a speckle field with uniform gray distribution, calculate the standard deviation of gray values ​​of all pixels in each first speckle grid within the first local comparison window, and use the standard deviation of gray values ​​as the speckle density of the corresponding first speckle grid.

6. The method for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision according to claim 1, characterized in that, The step of selecting a new first speckle grid from the remaining first speckle grids in the first local comparison window as the currently selected first speckle grid includes: According to the grid arrangement order in the image width or height direction, traverse all the first speckle grids in the first local comparison window one by one; Check the displacement vector determination status of each first speckle grid, select the first first speckle grid that has not been marked as having a determined displacement vector, and re-select it as the currently selected first speckle grid.

7. The method for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision according to claim 1, characterized in that, The step of determining the displacement vector of the currently selected first speckle grid based on the positions of the currently matched first and second speckle grids includes: Obtain the width and height coordinates of the currently selected first speckle mesh in the first image, and obtain the width and height coordinates of the matched second speckle mesh in the second image; Calculate the coordinate difference between the two grids in the width direction and the coordinate difference in the height direction, and combine the coordinate difference in the width direction and the coordinate difference in the height direction to form the displacement vector of the currently selected first speckle grid.

8. The method for synchronous measurement of three-dimensional displacement field of transparent soil based on binocular vision according to claim 1, characterized in that, The process of forming the transparent soil three-dimensional displacement field at time t based on the displacement vector of each first speckle grid includes: An interpolation algorithm is used to optimize the displacement vector of each first speckle grid with sub-pixel accuracy; Calculate the displacement gradient between two adjacent speckle grids with determined displacement vectors. If the displacement gradient of a certain grid exceeds the preset gradient threshold, the grid is reselected as the currently selected first speckle grid. Repeat the steps of similarity calculation, target determination and displacement vector assignment to correct its displacement vector. By using statistical criteria to remove coarse errors in all displacement vectors, the displacement vectors of all the corrected first speckle grids are integrated according to the order of their positions in the first image to form a complete three-dimensional displacement field of transparent soil at time t.

9. A synchronous measurement system for three-dimensional displacement field of transparent soil based on binocular vision, characterized in that, include: Image acquisition module: Applied to one of the three speckle images currently being measured. The three speckle images are three-dimensional speckle images of transparent soil after laser illumination acquired by the binocular vision system. The images are orthogonally decomposed. The speckle images at times t and t+1 are divided into grids to obtain a first image including multiple first speckle grids and a second image including multiple second speckle grids. Based on the projection position of the currently selected first speckle grid in the first and second images, the corresponding first local comparison window and second local comparison window are delineated in the first and second images. Similarity segmentation module: Extract the grayscale matrix of the currently selected first speckle grid and the grayscale matrix of each second speckle grid in the second local comparison window respectively. Using the cross-correlation coefficient method, multiply the pixel grayscale values ​​at corresponding positions in the two grayscale matrices one by one, and sum all the multiplication results to obtain the correlation coefficient between the currently selected first speckle grid and each second speckle grid. Based on the correlation coefficient, determine the second speckle grid that matches the currently selected first speckle grid. Determine the displacement vector of the currently selected first speckle grid according to the position of the currently matched first and second speckle grids. Determine the displacement vector of the speckle grid whose speckle density difference with the currently selected first speckle grid is less than a set threshold as the displacement vector of the currently selected first speckle grid. Displacement vector module: Select one of the remaining first speckle grids in the first local comparison window as the currently selected first speckle grid, until all first speckle grids in the first image have determined displacement vectors. Based on the displacement vector of each first speckle grid, form a three-dimensional displacement field of transparent soil at time t, so as to synchronously measure the three-dimensional displacement field of transparent soil.

Images