A method and related device for reconstructing the surface of wear particles based on multi-focus images

Abstract

The present application discloses a method and related device for reconstructing the surface of wear particles based on multi-focus images. The method includes obtaining a multi-focus image sequence of the wear particles to be reconstructed; using a differential evolution algorithm to determine the partitioning parameters corresponding to the multi-focus image sequence, and partitioning each focus image in the multi-focus image sequence based on the partitioning parameters to obtain a plurality of image blocks corresponding to each focus image; calculating the focus degree values of the partitioned image blocks, and fusing the multi-focus image sequence based on the focus degree values of the image blocks to obtain a fused image; determining the depth values of each pixel point in the fused image to obtain a reconstructed image corresponding to the wear particles to be reconstructed. In this implementation, the differential evolution algorithm is used to determine the partitioning parameters corresponding to the multi-focus image sequence, and the fused image blocks are selected based on the focus degree values of the partitioned image blocks, so as to improve the focus degree of the fused image blocks, and thus improve the clarity of the reconstructed image.

Description

Technical Field

[0001] This application relates to the technical field of bio-friction and wear, and particularly relates to a method and related device for reconstructing the surface of wear particles based on multi-focus images. Background Art

[0002] Wear particles include ferromagnetic abrasive particles, non-ferrous metal abrasive particles, ceramic chips, etc., and are common in fields such as mechanical friction and tribology. The research on wear particles is common in these fields. For example, in the field of mechanical friction, online sensors are usually installed on equipment to monitor the oil condition in the continuous circulation lubrication system in real time. The detection of wear particles in the oil is of great significance for judging mechanical failures and abnormal wear conditions. In the field of tribology, there are also frequent reports on the research of wear particles. For example, in the application research of artificial joints, the wear particles generated during the movement of the joint friction pair, since the wear particles are the products peeled off from the friction surface, the wear particles contain a large amount of detailed and important tribological information (such as abrasive particle size, quantity, composition, and morphological characteristics, etc.). Through tribological information, the wear form and mechanism of the replacement joint can be reflected. Thus, by analyzing the corresponding relationship between wear particles and the microscopic wear state, the change law in the complex friction and wear trend of joint materials in the human body environment can be sought, which has important practical significance for the monitoring of the use state of artificial joints and non-destructive fault diagnosis.

[0003] Currently, the research on wear particles mainly focuses on three-dimensional reconstruction of wear particles through images. Among them, three-dimensional reconstruction can include a motion-based three-dimensional reconstruction method and a contour-based three-dimensional reconstruction method that do not require an auxiliary light source, as well as a three-dimensional reconstruction method based on the brightness change of a single image that requires an auxiliary light source. The motion-based three-dimensional reconstruction method (structure from motion, SFM) reconstructs the target by the relative motion between the camera and the target object. This method reconstructs the object by collecting multiple views of the target object and realizing the three-dimensional reconstruction of the object through the knowledge of the epipolar geometry. Its basic process is camera calibration, feature point detection and matching, triangulation, and generating a three-dimensional point cloud. Due to the large optical distortion of the microscopic lens, it will affect the accuracy of the calibration result during the camera calibration process, resulting in inaccurate generation of the point cloud.

[0004] The contour-based three-dimensional reconstruction method (shape-from-silhouette, SFS) realizes the three-dimensional reconstruction of an object by obtaining the contour information of multiple views and calculating the spatial relationship between the obtained contour information. The contour-based three-dimensional reconstruction requires the determination of the rotation axis and the judgment of the rotation relationship between different views. In a complex fluid environment, errors are often introduced due to inaccurate judgment.

[0005] The three-dimensional reconstruction method based on the brightness change of a single image restores the three-dimensional shape through the brightness change of a single image (shape-from-shading, SFS), which can be regarded as the inverse process of the imaging process. However, due to the small amount of information in a single image, a large number of constraint conditions need to be added, which in turn limits the scope of application of the three-dimensional reconstruction method based on the brightness change of a single image.

[0006] Therefore, the existing technology still needs to be improved. Summary of the Invention

[0007] The technical problem to be solved by this application is to provide a method and related device for reconstructing the surface of wear particles based on multi-focus images in view of the deficiencies of the existing technology.

[0008] To solve the above technical problem, in the first aspect of the embodiments of this application, a method for reconstructing the surface of wear particles based on multi-focus images is provided. The method includes:

[0009] Obtain a multi-focus image sequence of the wear particles to be reconstructed;

[0010] Use the differential evolution algorithm to determine the partitioning parameters corresponding to the multi-focus image sequence, and based on the partitioning parameters, partition each focus image in the multi-focus image sequence to obtain several image blocks corresponding to each focus image;

[0011] Calculate the focus degree values of the partitioned image blocks, and fuse the multi-focus image sequence based on the focus degree values of the image blocks to obtain a fused image;

[0012] Determine the depth values of each pixel point in the fused image to obtain the reconstructed image corresponding to the wear particles to be reconstructed.

[0013] In the method for reconstructing the surface of wear particles based on multi-focus images, the multi-focus image sequence is obtained by a multi-focus image acquisition device. The multi-focus image acquisition device includes a workbench provided with a stage, a column provided on the workbench, and an acquisition component and a driving mechanism connected to the column. The driving mechanism is connected to the acquisition component and drives the acquisition component to move along the column in a direction close to or away from the stage; the acquisition component includes a camera, an annular light source, and a lens arranged along the extending direction of the column, and the stage is located below the lens.

[0014] In the method for reconstructing the surface of wear particles based on multi-focus images, after obtaining the multi-focus image sequence of the wear particles to be reconstructed, the method further includes:

[0015] For any two adjacent focused images in a multi-focused image sequence, detect the SIFT feature points of the first focused image and the SIFT feature points of the second focused image in the two adjacent focused images;

[0016] Use the KLT algorithm to match the SIFT feature points of the first focused image with the SIFT feature points of the second focused image, and determine the affine transformation matrix between the first focused image and the second focused image based on each matched SIFT feature point;

[0017] Take the first frame of the focused image in the multi-focused image sequence as the reference image, and perform motion compensation on each focused image in the multi-focused image sequence based on the affine transformation matrix between adjacent focused images to register the multi-focused image sequence.

[0018] The method for reconstructing the surface of wear particles based on multi-focused images, wherein the specific steps of using the differential evolution algorithm to determine the partitioning parameters corresponding to the multi-focused image sequence include:

[0019] Randomly generate an initial population, where the initial population includes a number of initial partitioning parameters;

[0020] For each initial partitioning parameter in the initial population, perform region partitioning on each focused image in the multi-focused image sequence to obtain a number of image blocks corresponding to each focused image; calculate the focus degree values of the partitioned image blocks, and fuse the multi-focused image sequence based on the focus degree values of the image blocks to obtain a fused image;

[0021] Calculate the global focus degree values of the fused images respectively, and select the target partitioning parameters in the initial population based on the global focus degree values of the fused images;

[0022] Form an intermediate population through crossover and mutation operations based on the target partitioning parameters, and determine a new generation population based on the intermediate population and the initial population;

[0023] Take the new generation population as the initial population and continue to execute the step of performing region partitioning on each focused image in the multi-focused image sequence for each initial partitioning parameter in the initial population until the iteration termination condition is met, so as to obtain the partitioning parameters corresponding to the multi-focused image sequence.

[0024] The method for reconstructing the surface of wear particles based on multi-focused images, wherein the specific steps of calculating the focus degree values of the partitioned image blocks include:

[0025] Perform a non - subsampled wavelet transform on each focused image in the multi - focused image sequence to obtain the total high - frequency coefficients of each pixel point in each focused image;

[0026] Calculate the sum of the total high - frequency coefficients of each pixel point in each image block to obtain the focus degree value of each image block.

[0027] The method for reconstructing the surface of wear particles based on multi - focused images, wherein the fusion of the multi - focused image sequence based on the focus degree values of each image block to obtain a fused image specifically includes:

[0028] For several image blocks corresponding to the same regional position, select the image block with the largest focus degree value among the several image blocks as the target image block to obtain the target image block corresponding to each regional position;

[0029] Stitch together the target image blocks corresponding to each regional position to obtain a fused image.

[0030] The method for reconstructing the surface of wear particles based on multi - focused images, wherein the determination of the depth value of each pixel point in the fused image to obtain the reconstructed image corresponding to the wear particle to be reconstructed specifically includes:

[0031] For each pixel position in the fused image, select the candidate pixel point corresponding to the pixel position in each focused image in the multi - focused image sequence and determine the candidate focus degree value of each candidate pixel point;

[0032] Perform Gaussian fitting on the candidate focus degree values of each candidate pixel point to obtain a focus degree value curve, and select the peak value of the focus degree value curve as the target focus degree value corresponding to the pixel position;

[0033] Determine the depth value corresponding to the pixel position based on the target focus degree value to obtain the reconstructed image corresponding to the wear particle to be reconstructed.

[0034] The method for reconstructing the surface of wear particles based on multi - focused images, wherein after the determination of the depth value of each pixel point in the fused image to obtain the reconstructed image corresponding to the wear particle to be reconstructed, the method includes:

[0035] Use a kernel regression algorithm to smooth the reconstructed image, and use the smoothed reconstructed image as the surface reconstructed image of the wear particle to be reconstructed.

[0036] In the second aspect of the embodiments of the present application, a multi-focus image acquisition device is provided. The device includes a workbench provided with a stage, a column provided on the workbench, and an acquisition component and a driving mechanism connected to the column. The driving mechanism is connected to the acquisition component and drives the acquisition component to move along the column in a direction approaching or departing from the stage; the acquisition component includes a camera, an annular light source, and a lens arranged along the extending direction of the column, and the stage is located below the lens.

[0037] In the third aspect of the embodiments of the present application, a terminal device is provided, which includes: a processor, a memory, and a communication bus; a computer-readable program executable by the processor is stored on the memory;

[0038] The communication bus realizes the connection and communication between the processor and the memory;

[0039] When the processor executes the computer-readable program, the steps in any of the above-mentioned methods for reconstructing the surface of wear particles based on multi-focus images are realized.

[0040] Beneficial effects: Compared with the prior art, the present application provides a method for reconstructing the surface of wear particles based on multi-focus images and related devices. The method includes obtaining a multi-focus image sequence of the wear particles to be reconstructed; using a differential evolution algorithm to determine the partitioning parameters corresponding to the multi-focus image sequence, and based on the partitioning parameters, partitioning each focus image in the multi-focus image sequence to obtain a plurality of image blocks corresponding to each focus image; calculating the focus degree values of the partitioned image blocks, and fusing the multi-focus image sequence based on the focus degree values of the image blocks to obtain a fused image; determining the depth values of each pixel point in the fused image to obtain a reconstructed image corresponding to the wear particles to be reconstructed. In this embodiment, the differential evolution algorithm is used to determine the partitioning parameters corresponding to the multi-focus image sequence, and the fused image blocks are selected based on the focus degree values of the partitioned image blocks, so that the focus degree of the fused image blocks can be improved, and thus the clarity of the reconstructed image can be improved. Description of the Drawings

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative labor, other drawings can be obtained based on these drawings.

[0042] Figure 1 It is a schematic structural diagram of a perspective of the multi-focus image acquisition device provided by the present application.

[0043] Figure 2Schematic structural diagram of a multi-focus image acquisition device provided by this application from a perspective

[0044] Figure 3 Schematic structural diagram of a multi-focus image acquisition device provided by this application from a perspective

[0045] Figure 4 Flow chart of a wear particle surface reconstruction method based on multi-focus images provided by this application.

[0046] Figure 5 Schematic diagram of the generation process of the difference of Gaussian scale space in the wear particle surface reconstruction method based on multi-focus images provided by this application.

[0047] Figure 6 Schematic diagram of the pole determination process in the wear particle surface reconstruction method based on multi-focus images provided by this application.

[0048] Figure 7 Schematic diagram of the main gradient direction determination process in the wear particle surface reconstruction method based on multi-focus images provided by this application.

[0049] Figure 8 Schematic diagram of the key point descriptor determination process in the wear particle surface reconstruction method based on multi-focus images provided by this application.

[0050] Figure 9 Schematic diagram of fixed-frame motion compensation in the wear particle surface reconstruction method based on multi-focus images provided by this application.

[0051] Figure 10 Flow schematic diagram of the division parameter determination process in the wear particle surface reconstruction method based on multi-focus images provided by this application.

[0052] Figure 11 Schematic diagram of the high-frequency component coefficient map in the wear particle surface reconstruction method based on multi-focus images provided by this application.

[0053] Figure 12 Schematic diagram showing that the focus value reflects the focused area in the wear particle surface reconstruction method based on multi-focus images provided by this application.

[0054] Figure 13 Schematic diagram of the focus value area in the wear particle surface reconstruction method based on multi-focus images provided by this application.

[0055] Figure 14 Set of comparison schematic diagrams of the reconstructed image and the laser confocal image determined by the wear particle surface reconstruction method based on multi-focus images provided by this application.

[0056] Figure 15A set of comparison schematic diagrams of the reconstructed image and the confocal laser image are determined for the wear particle surface reconstruction method based on multi-focus images provided by this application.

[0057] Figure 16 The structural schematic diagram of the terminal device provided by this application. Detailed implementation manners

[0058] This application provides a wear particle surface reconstruction method and related devices based on multi-focus images. To make the objectives, technical solutions and effects of this application clearer and more definite, the following further describes this application in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific examples described herein are only used to explain this application and are not used to limit this application.

[0059] Those skilled in the art of this technology can understand that unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "including" used in the specification of this application means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used here may include wireless connection or wireless coupling. The phrase "and / or" used here includes all or any unit and all combinations of one or more related listed items.

[0060] Those skilled in the art of this technology can understand that unless otherwise defined, all terms (including technical terms and scientific terms) used here have the same meaning as the general understanding of those of ordinary skill in the art to which this application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as here.

[0061] It should be understood that the sequence numbers and magnitudes of the steps in this embodiment do not mean the order of execution. The order of execution of each process is determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0062] The inventor has discovered through research that wear particles include ferromagnetic abrasive particles, non-ferrous metal abrasive particles, ceramic chips, etc., which are commonly found in fields such as mechanical friction and tribology. The research on wear particles is common in these fields. For example, in the field of mechanical friction, online sensors are usually installed on equipment to monitor the oil condition in a continuous circulation lubrication system in real time. The detection of wear particles in the oil is of great significance for judging mechanical faults and abnormal wear conditions. In the field of tribology, research on wear particles is also often reported. For example, in the application research of artificial joints, the wear particles generated during the movement of the joint friction pair, since the wear particles are the products peeled off from the friction surface, thus the wear particles contain a large amount of detailed and important tribological information (such as abrasive particle size, quantity, composition, and morphological characteristics, etc.). Through tribological information, the wear form and mechanism of the replacement joint can be reflected. Therefore, by analyzing the corresponding relationship between wear particles and the microscopic wear state, the change law in the complex friction and wear trend of joint materials in the human body environment can be sought, which has important practical significance for the monitoring of the use state of artificial joints and non-destructive fault diagnosis.

[0063] Currently, the research on wear particles mainly focuses on the three-dimensional reconstruction of wear particles through image methods. Among them, the three-dimensional reconstruction can include a motion-based three-dimensional reconstruction method and a contour-based three-dimensional reconstruction method that do not require an auxiliary light source, as well as a three-dimensional reconstruction method based on the brightness change of a single image that requires an auxiliary light source. The motion-based three-dimensional reconstruction method (structure from motion, SFM) reconstructs the target by the relative motion between the camera and the target object. This method reconstructs the object in three dimensions by collecting multiple views of the target object and using the knowledge of epipolar geometry. Its basic process includes camera calibration, feature point detection and matching, triangulation, and generation of a three-dimensional point cloud. Due to the large optical distortion of the microscopic lens, it will affect the accuracy of the calibration result during the camera calibration process, resulting in inaccurate generation of the point cloud.

[0064] The contour-based three-dimensional reconstruction method (shape-from-silhouette, SFS) realizes the three-dimensional reconstruction of an object by obtaining the contour information of multiple views and calculating the spatial relationship between the obtained contour information. The contour-based three-dimensional reconstruction requires the determination of the rotation axis and the judgment of the rotation relationship between different views. In a complex fluid environment, errors are often introduced due to inaccurate judgment.

[0065] The three-dimensional reconstruction method based on the brightness change of a single image restores the three-dimensional shape (shape-from-shading, SFS) through the brightness change of a single image, which can be regarded as the inverse process of the imaging process. However, due to the small amount of information in a single image, a large number of constraint conditions need to be added, which in turn limits the applicable range of the three-dimensional reconstruction method based on the brightness change of a single image.

[0066] To solve the above problems, in the embodiments of the present application, a multi-focus image sequence of the wear particles to be reconstructed is acquired; a differential evolution algorithm is used to determine the partitioning parameters corresponding to the multi-focus image sequence, and based on the partitioning parameters, each focus image in the multi-focus image sequence is partitioned into regions to obtain a plurality of image blocks corresponding to each focus image; the focus degree values of the partitioned image blocks are calculated, and the multi-focus image sequence is fused based on the focus degree values of the image blocks to obtain a fused image; the depth values of the pixel points in the fused image are determined to obtain the reconstructed image corresponding to the wear particles to be reconstructed. In this embodiment, the differential evolution algorithm is used to determine the partitioning parameters corresponding to the multi-focus image sequence, and the fused image blocks are selected based on the focus degree values of the partitioned image blocks, so that the focus degree of the fused image blocks can be improved, and thus the clarity of the reconstructed image can be improved.

[0067] The following further describes the content of the application by describing the embodiments in conjunction with the accompanying drawings.

[0068] The method for reconstructing the surface of wear particles based on multi-focus images provided in this embodiment needs to acquire a multi-focus image sequence. Therefore, in order to acquire the multi-focus image sequence, this embodiment provides a multi-focus image acquisition device, such as Figure 1 such as Figure 2 and Figure 3As shown in the figure, the device includes a workbench 1, on which a loading platform 2 and a column 3 are arranged. One end of the column 3 is connected to the workbench 1, and the other end extends away from the workbench 1. A collection component 7 and a driving mechanism 4 are arranged on the column 3. The driving mechanism 4 is connected to the collection component 7 and drives the collection component 7 to move along the column 5 in a direction close to or away from the loading platform 2. Among them, the collection component 7 includes a camera 71, an annular light source 72 and a lens 73 arranged along the extension direction of the column. The loading platform 2 is located below the lens 73, and the annular light source 72 is used to provide light for the camera 71. The camera 71 can be connected to an external device and transmit the images it collects to the external device through a connecting line. The driving mechanism 4 can adopt a servo motor. A scale is arranged on the column 3 along its extension direction, and the moving distance of the collection component 7 is reflected by the scale value. In addition, the collection component 7 is connected to the column 3 through a connecting component, and the connecting component includes a sliding table 5 and a connecting piece 6. The sliding table 5 is slidably connected to the column 3 and is connected to the driving mechanism 4. One end of the connecting piece 6 is connected to the sliding table 5, and the other end extends in a direction close to the loading platform 2 and is connected to the collection component 7. In this embodiment, during the process of the camera being driven by the driving mechanism to slide along the column, a multi-focus image sequence is collected. In this way, multiple focus images with different focuses can be captured by one camera, thus avoiding the problem that the image accuracy of the reconstructed image is affected by the optical distortion of the microscopic lens.

[0069] The working process of the device is as follows: Control the servo motor to rotate to drive the sliding table along the column through the servo motor. The sliding table drives the collection component to move through the connecting piece, and the camera performs image acquisition or video recording during the movement to obtain a multi-focus image sequence. Among them, the camera parameters of the camera can be set according to actual needs, and the camera parameters can include contrast, exposure time, white balance, video frame rate, file storage location, etc. That is to say, by setting the moving speed and moving stroke of the collection mechanism along the column, and simultaneously starting the servo motor and the collection component, a multi-focus image sequence of the measured object placed on the loading platform can be obtained.

[0070] In one implementation, the moving speed range of the sliding table is 0.0025 mm / s - 30 cm / s, the stroke of the table is 340 mm, and the acquisition frequency of the collection component is 30 frames / s. For example, if the moving speed of the sliding table is 2.5 um / s, it takes 40 s for the sliding table to run 100 um at a speed of 2.5 um / s, and 1200 frames can be captured in 40 s. In addition, after determining the moving speed of the sliding table and the number of captured images, the distance between two frames can be determined according to the exposure method.

[0071] This embodiment provides a method for reconstructing the surface of wear particles based on multi-focus images, as Figure 4 shown, the method includes:

[0072] S10. Obtain a multi-focus image sequence of the wear particles to be reconstructed.

[0073] Specifically, the multi-focus image sequence includes a plurality of focused images. Each of the plurality of focused images includes the wear particles to be reconstructed. And the plurality of focused images are of the same scene captured, and each of the plurality of focused images focuses on a different position. In one implementation, the multi-focus image sequence can be obtained by the above-mentioned multi-focus image acquisition device. The specific structure and working process of this device can refer to the above description and will not be elaborated here.

[0074] In one implementation of this embodiment, after obtaining the multi-focus image sequence of the wear particles to be reconstructed, the method further includes:

[0075] For any two adjacent focused images in the multi-focus image sequence, detect the SIFT feature points of the first focused image in the two adjacent focused images and the SIFT feature points of the second focused image in the two adjacent focused images;

[0076] Use the KLT algorithm to match the SIFT feature points of the first focused image with the SIFT feature points of the second focused image, and determine the affine transformation matrix between the first focused image and the second focused image based on the matched SIFT feature points;

[0077] Take the first-frame focused image in the multi-focus image sequence as the reference image, and perform motion compensation on each focused image in the multi-focus image sequence based on the affine transformation matrices between adjacent focused images to register the multi-focus image sequence.

[0078] Specifically, both the first focused image and the second focused image are focused images in the multi-focus image sequence, and the first focused image and the second focused image are two adjacent image frames. Here, adjacent image frames refer to two image frames that are adjacent in the acquisition order. For example, after the first focused image is captured in the image acquisition order, the second focused image is captured, and there is no other focused image between the first focused image and the second focused image. In addition, before detecting the SIFT feature points of the first focused image and the SIFT feature points of the second focused image in the two adjacent focused images, the first focused image and the second focused image can be preprocessed, where the preprocessing can include histogram equalization and image filtering enhancement, etc.

[0079] The process of obtaining SIFT feature points specifically includes scale space and extreme point detection, key point localization, determination of its main direction, and generation of key point descriptors. Here, taking the process of SIFT feature points of the first focused image as an example, the process of obtaining SIFT feature points is described.

[0080] 1. Scale space and extreme point detection

[0081] Obtain the Gaussian scale space of the first focused image I(x, y), and its Gaussian scale space can be expressed as:

[0082] L(x, y, σ) = G(x, y, σ)I(x, y)

[0083] Generate the Difference of Gaussian (DOG) scale space according to the Gaussian scale space of the first focused image. As Figure 5 shown, the DOG scale space can be expressed as:

[0084] D(x, y, σ) = (G(x, y, kσ) - G(x, y, σ))I(x, y)

[0085]

[0086] Among them, G(x, y, σ) is the Gaussian kernel function, σ is the scale space factor, and k is the reciprocal of the total number of layers within a group.

[0087] After determining the DOG scale space, as Figure 6 shown, in the DOG scale space, compare the pixel value of each pixel with the pixel values of all its corresponding adjacent pixels to obtain extreme points.

[0088] 2. Key point localization and determination of the main direction

[0089] Key points include local extreme points in the DOG scale space. First, use the fitting relationship to determine the exact position and scale of the key points, and calculate the gradients of each key point. Then, use the histogram to statistically analyze the gradients and directions of the pixels in the neighborhood to obtain the gradient direction histogram of the pixels in the neighborhood of the key points, and take the peak direction in the gradient direction histogram as the main direction of the key points. For example, as Figure 7 shown, the direction of the fifth histogram in the figure is the main gradient direction.

[0090] 3. Generation of key point descriptors;

[0091] As Figure 8As shown in the figure, move the coordinate axes to the main direction of the key point, take an 8×8 window centered on the key point, calculate the gradient of each pixel in the image, then establish an equally spaced gradient histogram at 45°, calculate the cumulative values in different directions, and finally use the normalized cumulative value as a descriptor.

[0092] The KLT algorithm is a matching algorithm that uses the sum of squared differences (SSD) of the gray levels between two adjacent frames of the video in the window to be tracked as a metric. Assume that a feature window W contains feature texture information. Assume that the video frame corresponding to time t is represented by I(x, y, t), and the video frame corresponding to time t+τ is represented by I(x, y, t+τ). Then the corresponding position satisfies the following equation:

[0093] I(x, y, t+τ) = I(x-Δx, y-Δy, t)

[0094] where Δx and Δy are the offsets of the feature point X(x, y), and each pixel point of I(x, y, t+τ) can be obtained by translating the corresponding pixel point of I(x, y, t) by d(Δx, Δy).

[0095] The first focused image is denoted as I, the second focused image is denoted as J, and the error ε between the first focused image and the second focused image can be expressed as:

[0096]

[0097] where W is a given feature window, and ω(X) is a weight function that is usually set to 1.

[0098] Within the W window, I is centered at and J is centered at with a radius of Thus, J(X)-I(X-d) is changed to a symmetric form Correspondingly, ε can be expressed as:

[0099]

[0100] Expand using Taylor expansion, delete the highest-order term and only retain the first two terms, and take the derivative with respect to d to obtain:

[0101]

[0102] where g is the first-order Taylor coefficient of the Taylor expansion, and g x and g y represent the first-order Taylor coefficients in the x and y directions respectively.

[0103] Performing Newton iteration on each pixel point can obtain:

[0104]

[0105] Among them, d represents the translation of the center of the feature window W, and d k represents the value of d obtained in the k-th iteration. Usually, during the iteration process, the initial estimated value d0 needs to be calculated, where d0 = 0.

[0106] The affine transformation matrix is used to reflect the global motion between the first focused image and the second focused image. The affine transformation matrix can be determined by using the RANSAC algorithm. Among them, the mathematical model of the RANSAC algorithm is:

[0107]

[0108] Among them, represents the affine transformation matrix, a 11 , a 12 , a 21 , a 22 represents the rotation parameter and the scaling parameter; a 13 , a 23 represents the translation parameter.

[0109] After obtaining the affine transformation matrix between adjacent focused images, it is used to collect the original camera path C of the multi-focused image sequence t can be expressed as:

[0110] C t = C t-1 H t-1 , t = 1, 2,..., N

[0111] Among them, t represents the t-th frame of the focused image, and H t-1 represents the affine transformation H between the (t - 1)-th frame of the focused image and the t-th frame of the focused image t , and N represents the total number of frames of the video.

[0112] In one implementation of this embodiment, when using the first frame of the focused image in the multi-focused image sequence as the reference image and performing motion compensation on each focused image in the multi-focused image sequence based on the affine transformation matrix between adjacent focused images, the fixed-frame compensation method can be used. Among them, as Figure 9 shown, the fixed-frame compensation method is to select a reference image, estimate the motion vector between the reference image and each current frame through an existing global motion estimation algorithm, and directly perform motion compensation on the current frame. By adopting the fixed-frame compensation method for motion compensation in this embodiment, the stability of each focused image in the multi-focused image sequence can be improved, and the interference caused by the random jitter of the imaging system can be removed.

[0113] The process of the fixed-frame compensation method can be:

[0114] For the current frame p k with pixel coordinates [x, y] k perform an affine transformation to obtain new coordinates [x′, y′] T , where the calculation formula for [x′, y′] T can be:

[0115]

[0116] Assign the pixel value p k at the original coordinate position in p k [x, y] to the new coordinate p k '[x', y'], then the compensated image is obtained.

[0117] S20. Use the differential evolution algorithm to determine the partitioning parameters corresponding to the multi-focus image sequence, and perform region partitioning on each focus image in the multi-focus image sequence based on the partitioning parameters to obtain several image blocks corresponding to each focus image respectively.

[0118] Specifically, the partitioning parameters are determined based on the differential evolution algorithm, and the partitioning parameters include the block size of a single image block. The number of several image blocks corresponding to each focus image is the same, and for an image block a in a focus image, there is an image block b in several image blocks corresponding to other focus images. The regional positions of the image blocks in their respective corresponding focus images are the same as the regional position of the image block a in its corresponding focus image. That is to say, after registration of each focus image in the multi-focus image sequence, the image sizes of each focus image are the same, and the image edges of each focus image are the same.

[0119] In an implementation manner of this embodiment, as Figure 10 shown, the use of the differential evolution algorithm to determine the partitioning parameters corresponding to the multi-focus image sequence specifically includes:

[0120] Randomly generate an initial population, where the initial population includes several initial partitioning parameters;

[0121] For each initial partitioning parameter in the initial population, perform region partitioning on each focus image in the multi-focus image sequence based on the initial partitioning parameter to obtain several image blocks corresponding to each focus image respectively; calculate the focus degree values of the partitioned image blocks, and fuse the multi-focus image sequence based on the focus degree values of the image blocks to obtain a fused image;

[0122] Calculate the global focus degree values of the fused images respectively, and select the target partitioning parameters in the initial population based on the global focus degree values of the fused images;

[0123] Form an intermediate population through crossover and mutation operations based on the target division parameters, and determine a new generation of population based on the intermediate population and the initial population;

[0124] Use the new generation of population as the initial population and continue to execute the step of dividing each focused image in the multi-focus image sequence based on the initial division parameter until the iteration termination condition is met, so as to obtain the division parameter corresponding to the multi-focus image sequence.

[0125] Specifically, before randomly generating the initial population, it is necessary to preset the control parameters of the differential evolution algorithm. Among them, the control parameters can include crossover probability, mutation probability, population size, iteration termination condition, etc. The initial population is randomly generated, and the initial population includes several initial division parameters. Among them, each initial division parameter in the several initial division parameters is different from each other.

[0126] After obtaining the initial population, for each initial division parameter in the initial population, the multi-focus image sequence is divided into regions according to the initial division parameter, and the focus degree values of the divided image blocks are calculated. And the multi-focus image sequence is fused based on the focus degree values of the image blocks to obtain a fused image. Among them, the process of dividing the multi-focus image sequence based on each initial division parameter is the same as the process of dividing the multi-focus image sequence based on the division parameter above, and will not be described here.

[0127] In an implementation manner of this embodiment, the calculation of the focus degree values of the divided image blocks specifically includes:

[0128] Perform a non-downsampling wavelet transform on each focused image in the multi-focus image sequence to obtain the total high-frequency coefficients of each pixel point in each focused image;

[0129] Calculate the sum of the total high-frequency coefficients of each pixel point in each image block to obtain the focus degree value of each image block.

[0130] Specifically, the non-downsampling wavelet transform is to apply a filter bank Process the signal c0 to obtain the set W = {ω1,…,ω J ,c J}, where ω j represents the wavelet coefficient at the j scale, and c J represents the wavelet coefficient at the coarsest resolution. The decomposition applies the porous algorithm:

[0131]

[0132] In addition, when l / 2 jWhen it is an integer or 0, there is h (j) [l] = h[l], such as

[0133] h (1) =(…, h[-2], 0, h[-1], 0, h[0], 0, h[1], 0, h[2], …)

[0134] The reconstruction can be obtained through the following formula

[0135]

[0136] For the filter bank, only the reconstruction condition needs to be verified:

[0137]

[0138] Therefore, there is a higher degree of freedom in synthesizing the prototype filter bank.

[0139] Furthermore, when performing the undecimated wavelet transform on each focused image in the multi-focus image sequence, each focused image can correspondingly obtain three high-frequency component coefficient maps and one low-frequency coefficient map. Among them, the image sizes of the three high-frequency component coefficient maps and the one low-frequency coefficient map are the same as the image size of the focused image, and the components corresponding to each high-frequency component coefficient map among the three high-frequency component coefficient maps are different. In one implementation, the three high-frequency component coefficient maps respectively correspond to the horizontal component, the vertical component, and the diagonal component. For example, as Figure 11 shown, the focused images of the wear particles in the focused far-field and the focused near-field respectively obtain three high-frequency component coefficient maps with the same image size as the focused image after the undecimated wavelet transform, namely the high-frequency horizontal component image, the high-frequency vertical component image, and the high-frequency diagonal component image.

[0140] In one implementation of this embodiment, the focus degree value of each pixel point in the focused image is determined based on the high-frequency component coefficient map. And after obtaining the high-frequency component coefficient maps corresponding to each focused image through the undecimated wavelet transform, the high-frequency component coefficients of each pixel point can be calculated by template convolution to expand the high-frequency coefficient gap between the focused pixel points and the defocused pixel points. In addition, since each focused image can obtain three high-frequency component coefficient maps, each pixel point correspondingly has three high-frequency component coefficients. Thus, after obtaining the three high-frequency component coefficient maps corresponding to each focused image, the total high-frequency coefficient of the pixel point can be determined based on the three high-frequency component coefficients corresponding to each pixel point, where the total high-frequency coefficient can be equal to the sum of the absolute values of the high-frequency component coefficients.

[0141] For example, the three high-frequency component coefficients corresponding to each pixel point in the focused image are expressed as:

[0142]

[0143]

[0144]

[0145] Among them, the superscripts H, V, and D respectively represent the high-frequency horizontal component, the high-frequency vertical component, and the high-frequency diagonal component, and the subscript I represents the I-th focused image in the image sequence.

[0146] Then, the total high-frequency coefficient D I (x, y) of each pixel can be expressed as:

[0147]

[0148] Furthermore, after obtaining the total high-frequency coefficient, the total high-frequency coefficient corresponding to each pixel is used as the focusing degree of each pixel, and then based on the focusing degree of each pixel, the focusing degree value of each image block is calculated. Among them, the focusing degree value of each image block can be:

[0149]

[0150] Among them, W n×n is a square neighborhood centered at (x, y) with a size of n×n, n×n is the initial partitioning parameter, and (i, j) is the position in the square neighborhood.

[0151] When using the focusing degree value FM NSWT (x, y) to reflect the focusing degree of the image block, as Figure 12 shown, the larger the focusing degree value, the higher the focusing degree of the image block; conversely, the smaller the focusing degree value, the smaller the focusing degree of the image block. Therefore, when fusing the multi-focus image sequence based on the focusing degree value of each image block, for each regional position, the image block with the largest focusing degree value can be selected as the target image block, which can improve the focusing degree of each image block in the fused image obtained by fusion, and thus improve the image clarity of the fused image.

[0152] In an implementation manner of this embodiment, the fusion of the multi-focus image sequence based on the focusing degree value of each image block to obtain a fused image specifically includes:

[0153] For several image blocks corresponding to the same regional position, select the image block with the largest focusing degree value among the several image blocks as the target image block to obtain the target image block corresponding to each regional position;

[0154] Stitch the target image blocks corresponding to each regional position to obtain a fused image.

[0155] Specifically, the image blocks are obtained by dividing the focused image, so that each image block is an image region in the focused image, and thus each image block corresponds to a region position in the focused image. It can be understood that the region position refers to the position where the image block is located in its corresponding focused image. In addition, since the image sizes of the focused images are the same and are divided according to the same initial division parameters, the image blocks obtained by dividing each focused image correspond one by one. Thus, for each region position, there is an image block corresponding to this region position among the several image blocks obtained by dividing each focused image, that is to say, several image blocks can be obtained for each region position. In addition, after obtaining the several image blocks corresponding to each region position, select the image block with the largest focus value among the several image blocks corresponding to this region position as the target image block, and then splice the target image blocks corresponding to each region position to obtain a fused image, where the image size of the fused image is the same as that of the focused image.

[0156] The global focus value of the fused image is used to reflect the focus degree of the fused image, where the global focus value can be equal to the sum of the focus values of the image blocks in the fused image. That is to say, after obtaining the fused image, calculate the sum of the focus values of the image blocks in the fused image, and use the calculated sum as the global focus value corresponding to the fused image. Thus, each initial division parameter corresponds to a global focus value, and then the target division parameter can be selected from several initial division parameters based on the global focus value, where the target division parameter can be the initial division parameter corresponding to the largest global focus value among several initial division parameters.

[0157] After obtaining the target division parameter, an intermediate population can be formed by performing crossover and mutation operations on the target division parameter. Among them, the mutation operation can use a mutation operator to expand the search space, combine the division parameters randomly selected from the current population of the G-th generation, where the mutated division parameter v i，G+1 can be expressed as:

[0158] v i，G+1 = x r1，G + F(x r2，G - x r3，G )

[0159] x i，G = x i(L) + rand i [0, 1](x i(H) - x i(L) ), i = r1, r2, r3

[0160] Among them, i, r1, r2, r3 are not equal to each other, and their values are from {1, 2,..., NP}, where NP represents the population size, G represents the G-th generation, G + 1 represents the (G + 1)-th generation, x i(H) and x i(L) are respectively the upper and lower limits of the population individual elements, and the subscript i in x i，G represents the i-th element of the d-dimensional division parameter.

[0161] F is a non-negative weight factor in [0, 2]. A small F value will bring faster convergence, and a large F value will bring more diversity. At the beginning of the iteration, the F value obtains the maximum total high-frequency coefficient. As the number of iterations increases, the F value will gradually decrease.

[0162] The crossover operation adopts the DE crossover operation. The trial division parameter u i，G+1 is generated from the mutant division parameter v i，G+1 and the elements of the target division parameter x i，G ; among them, the expression of the trial division parameter u i，G+1 can be:

[0163]

[0164] S30. Calculate the sharpness values of the divided image blocks, and fuse the multi-focus image sequence based on the sharpness values of the image blocks to obtain a fused image.

[0165] Specifically, the calculation process of the sharpness values of the image blocks and the process of fusing the multi-focus image sequence based on the sharpness values of the image blocks to obtain a fused image are the same as the calculation process in the above process of determining the division parameters. For details, please refer to the above description and will not be elaborated here.

[0166] S40. Determine the depth values of the pixel points in the fused image to obtain the reconstructed image corresponding to the to-be-reconstructed wear particle.

[0167] Specifically, each pixel point in the reconstructed image includes a depth value. That is to say, the reconstructed image is formed by configuring depth values for the pixel points in the fused image, so that the reconstructed image is a three-dimensional reconstruction image of the surface of the to-be-reconstructed wear particle.

[0168] In one implementation, the determining the depth values of the pixel points in the fused image to obtain the reconstructed image corresponding to the to-be-reconstructed wear particle specifically includes:

[0169] For each pixel position in the fused image, select the candidate pixel points corresponding to the pixel position in each focused image in the multi-focus image sequence, and determine the candidate sharpness values of the candidate pixel points;

[0170] Perform Gaussian fitting on the candidate focus values of each candidate pixel to obtain a focus value curve, and select the peak value of the focus value curve as the target focus value corresponding to the pixel position;

[0171] Determine the depth value corresponding to the pixel position based on the target focus value to obtain the reconstructed image corresponding to the wear particle to be reconstructed.

[0172] Specifically, for each pixel position in the fused image, there is a candidate pixel in each focused image in the multi-focus image sequence corresponding to this pixel position. That is to say, the pixel position of the candidate pixel in the focused image where it is located is the pixel position. Thus, the number of candidate pixels corresponding to the pixel position is equal to the number of focused images in the multi-focus image sequence. For example, if the multi-focus image sequence includes 20 focused images, then the number of candidate pixels is 20.

[0173] After obtaining each candidate pixel, determine the candidate focus value of each candidate pixel, where the candidate focus value is equal to the focus value of the candidate image block centered on the candidate pixel, and the image size of the candidate image block is equal to the image size of the image block divided based on the division parameter. For example, if the division parameter is n×n, then the image size of the image block divided based on the division parameter is n×n. Correspondingly, the image size of the candidate image block is n×n, and the focus value of the candidate pixel is the sum of the total high-frequency coefficients of all pixels in the n×n candidate image block. Thus, the process of determining the candidate focus values of each candidate pixel can be: for each candidate pixel, select a candidate image block centered on the candidate pixel with the division parameter as the region parameter in the focused image where the candidate pixel is located, and calculate the sum of the total high-frequency coefficients of all pixels in the candidate image block to obtain the candidate focus value of the candidate pixel.

[0174] After obtaining the candidate focus values of each candidate pixel, since the focused images in the multi-focus image sequence are discretely acquired, correspondingly, each candidate focus value is a discrete point. Also, due to the property that the focus value curve has a Gaussian distribution at the peak, the candidate focus values can be Gaussian-fitted to obtain a focus value curve, as Figure 13 shown, the focus value curve is a function curve of the focus depth and the focus value of each focused image. In addition, after obtaining the focus value curve, take the peak value of the focus value curve as the target focus value corresponding to the pixel position, then based on the focus value curve, the focus depth corresponding to the peak can be determined, and the focus depth corresponding to the peak is used as the depth value corresponding to the pixel position to obtain the reconstructed image corresponding to the wear particle to be reconstructed.

[0175] In one implementation of this embodiment, in order to improve the clarity of the reconstructed image, after obtaining the reconstructed image, the reconstructed image can also be smoothed to highlight the detailed information therein and overcome problems such as unevenness and blurred edges in the initial depth map. Based on this, after determining the depth values of the pixel points in the fused image to obtain the reconstructed image corresponding to the to-be-reconstructed worn particle, the method includes:

[0176] Use the kernel regression algorithm to smooth the reconstructed image, and use the smoothed reconstructed image as the surface reconstructed image of the to-be-reconstructed worn particle.

[0177] Specifically, the kernel regression algorithm belongs to local regression, which has the advantages of being applicable to unnormalized data, having no fixed form of the regression function, and not requiring corresponding preprocessing operations on the data before use; being completely data-driven, having a wide application range, high stability, and high accuracy of the model. Among them, the regression model adopted by the kernel regression algorithm is:

[0178]

[0179] where x i =[x 1i ,x 2i T represents the pixel point at the pixel position (x 1i ,x 2i ) in the reconstructed image, where i = 1, 2,..., P, and P represents the total number of pixel points in a local window. describes the initial depth value at the i-th pixel point x i , and ε i is an independently and identically distributed interference signal.

[0180] The goal of the kernel regression algorithm is to recover the unknown regression function t(x ) through the pre-estimated value i . To predict the value of the function t(x i ) at a certain pixel point x, perform N-order local smoothing on t(x i ). Assume that x is a sampling point near x i , then t(x i ​The N - th order Taylor series expansion of () is used to obtain the N - th order Taylor expansion formula. Then, local fitting of the N - th order Taylor expansion formula is performed at x by the weighted least - squares method, and the value of the fitted polynomial at x is used as the estimated value t(x) of the regression function at x. Furthermore, the N - th order Taylor expansion formula is converted into a weighted least - squares optimization problem by using the least - squares method to optimize the parameters and the control function of the weights of other pixels around the center point within the local window. The smoothed depth value of each pixel point is obtained by solving the least - squares optimization problem, thereby obtaining the smoothed reconstructed image. Among them, the Gaussian kernel is used as the kernel function for the least - squares optimization problem.

[0181] To illustrate the effect of the reconstructed image obtained by the wear particle surface reconstruction method based on multi - focus images provided by this embodiment, this embodiment provides the method provided by this embodiment and the surface map of the wear particle collected by a laser confocal microscope. As Figure 14 and Figure 15 shown, the reconstructed image obtained by this method can vividly and completely display the surface of the wear particle.

[0182] In summary, this embodiment provides a wear particle surface reconstruction method based on multi - focus images. The method includes obtaining a multi - focus image sequence of the wear particle to be reconstructed; using the differential evolution algorithm to determine the division parameters corresponding to the multi - focus image sequence, and performing regional division on each focus image in the multi - focus image sequence based on the division parameters to obtain several image blocks corresponding to each focus image; calculating the focus degree values of the divided image blocks, and fusing the multi - focus image sequence based on the focus degree values of the image blocks to obtain a fused image; determining the depth values of each pixel point in the fused image to obtain the reconstructed image corresponding to the wear particle to be reconstructed. In this embodiment, the differential evolution algorithm is used to determine the division parameters corresponding to the multi - focus image sequence, and the fused image blocks are selected based on the focus degree values of the divided image blocks, which can improve the focus degree of the fused image blocks, thereby improving the clarity of the reconstructed image. In addition, after obtaining the multi - focus image sequence, this embodiment also performs registration operations between the focus images in the multi - focus image sequence, and performs smoothing through the kernel regression algorithm after obtaining the reconstructed image, which can further improve the clarity of the reconstructed image.

[0183] Based on the above - mentioned wear particle surface reconstruction method based on multi - focus images, this embodiment provides a computer - readable storage medium. The computer - readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the wear particle surface reconstruction method based on multi - focus images as described in the above - mentioned embodiment.

[0184] Based on the above - mentioned wear particle surface reconstruction method based on multi - focus images, this application also provides a terminal device, as Figure 16As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communications interface 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22, and the communications interface 23 can complete mutual communication through the bus 24. The display screen 21 is set to display a user guidance interface preset in the initial setting mode. The communications interface 23 can transmit information. The processor 20 can call the logical instructions in the memory 22 to execute the methods in the above embodiments.

[0185] In addition, when the logical instructions in the above-mentioned memory 22 can be implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0186] The memory 22, as a computer-readable storage medium, can be set to store software programs and computer-executable programs, such as the program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, that is, to implement the methods in the above embodiments.

[0187] The memory 22 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs can also be transient storage media.

[0188] In addition, the specific processes of loading and executing multiple instructions by the above-mentioned storage medium and the instruction processor in the terminal device have been described in detail in the above methods and will not be repeated here one by one.

[0189] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for reconstructing the surface of wear particles based on multi-focus images, characterized in that The method includes: Obtaining a multi-focus image sequence of the worn particles to be reconstructed; Using a differential evolution algorithm to determine the partitioning parameters corresponding to the multi-focus image sequence, and partitioning each focus image in the multi-focus image sequence based on the partitioning parameters to obtain a number of image blocks corresponding to each focus image; Calculating the focus degree values of the partitioned image blocks, and fusing the multi-focus image sequence based on the focus degree values of the image blocks to obtain a fused image; Determining the depth values of the pixel points in the fused image to obtain a reconstructed image corresponding to the worn particles to be reconstructed; Among them, the specific calculation of the focus degree values of the partitioned image blocks includes: Performing a non-subsampled wavelet transform on each focus image in the multi-focus image sequence to obtain the total high-frequency coefficients of the pixel points in each focus image. Among them, each focus image corresponds to three high-frequency component coefficient maps, and the total high-frequency coefficient of each pixel point is equal to the sum of the absolute values of the high-frequency horizontal component image, the high-frequency vertical component image, and the high-frequency diagonal component image corresponding to the pixel point; Calculating the sum of the total high-frequency coefficients of the pixel points in each image block to obtain the focus degree value of each image block; The specific determination of the depth values of the pixel points in the fused image to obtain a reconstructed image corresponding to the worn particles to be reconstructed includes: For each pixel position in the fused image, selecting candidate pixel points corresponding to the pixel position in each focus image in the multi-focus image sequence, and determining the candidate focus degree values of the candidate pixel points; Performing Gaussian fitting on the candidate focus degree values of the candidate pixel points to obtain a focus degree value curve, and selecting the peak value of the focus degree value curve as the target focus degree value corresponding to the pixel position; Determining the depth value corresponding to the pixel position based on the target focus degree value to obtain a reconstructed image corresponding to the worn particles to be reconstructed.

2. The method for reconstructing the surface of wear particles based on multi-focus images according to claim 1, wherein The multi-focus image sequence is obtained by a multi-focus image acquisition device. Among them, the multi-focus image acquisition device includes a workbench provided with a stage, a column provided on the workbench, and an acquisition component and a driving mechanism connected to the column. The driving mechanism is connected to the acquisition component and drives the acquisition component to move along the column in a direction close to or away from the stage; the acquisition component includes a camera, an annular light source, and a lens arranged along the extending direction of the column, and the stage is located below the lens.

3. The method for reconstructing the surface of wear particles based on multi-focus images according to claim 1, wherein After obtaining the multi-focus image sequence of the worn particles to be reconstructed, the method further includes: For any two adjacent focus images in the multi-focus image sequence, detecting the SIFT feature points of the first focus image in the two adjacent focus images and the SIFT feature points of the second focus image in the two adjacent focus images; Using the KLT algorithm to match the SIFT feature points of the first focus image with the SIFT feature points of the second focus image, and determining the affine transformation matrix between the first focus image and the second focus image based on the matched SIFT feature points; Using the first-frame focused image in the multi-focus image sequence as the reference image, and based on the affine transformation matrix between adjacent focused images, perform motion compensation on each focused image in the multi-focus image sequence to register the multi-focus image sequence.

4. The method for reconstructing the surface of wear particles based on multi-focus images according to claim 1, wherein, The specific steps of using the differential evolution algorithm to determine the partitioning parameters corresponding to the multi-focus image sequence include: Randomly generate an initial population, where the initial population includes several initial partitioning parameters; For each initial partitioning parameter in the initial population, perform regional partitioning on each focused image in the multi-focus image sequence based on the initial partitioning parameter to obtain several image blocks corresponding to each focused image; calculate the focus degree values of the partitioned image blocks, and fuse the multi-focus image sequence based on the focus degree values of the image blocks to obtain a fused image; Calculate the global focus degree values of each fused image respectively, and select the target partitioning parameter in the initial population based on the global focus degree values of each fused image; Form an intermediate population through crossover and mutation operations based on the target partitioning parameter, and determine a new generation population based on the intermediate population and the initial population; Use the new generation population as the initial population and continue to execute the step of performing regional partitioning on each focused image in the multi-focus image sequence based on the initial partitioning parameter until the iteration termination condition is met, so as to obtain the partitioning parameters corresponding to the multi-focus image sequence.

5. The method for reconstructing the surface of wear particles based on multi-focus images according to claim 1, wherein The specific steps of fusing the multi-focus image sequence based on the focus degree values of the image blocks to obtain a fused image include: For several image blocks corresponding to the same regional position, select the image block with the largest focus degree value among the several image blocks as the target image block, so as to obtain the target image block corresponding to each regional position; Stitch the target image blocks corresponding to each regional position to obtain a fused image.

6. The method for reconstructing the surface of wear particles based on multi-focus images according to claim 1, wherein After determining the depth values of each pixel point in the fused image to obtain the reconstructed image corresponding to the to-be-reconstructed wear particle, the method includes: Use the kernel regression algorithm to smooth the reconstructed image, and use the smoothed reconstructed image as the surface reconstructed image of the to-be-reconstructed wear particle.

7. A terminal device, characterized in that, Including: A processor, a memory, and a communication bus; A computer-readable program executable by the processor is stored on the memory; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, it realizes the steps in the method for reconstructing the surface of wear particles based on multi-focus images according to any one of claims 1-6.

Images