Data processing method for pavement disease image fusion based on three-dimensional digital holography

Abstract

The application discloses a kind of data processing methods of road surface disease image fusion based on three-dimensional digital holography, and original graph is obtained by generating digital interference fringe image and projecting to road surface through laser interferometer and collecting road surface;Computer simulation restoration reconstruction is obtained three-dimensional holographic disease graph;Two-dimensional disease graph is obtained using Fourier transform;Three-dimensional holographic disease graph and two-dimensional disease graph are fused to generate fusion disease graph;Using specified data, the final training model is obtained by testing simulation correction to fusion model.This application can obtain three-dimensional disease graph and two-dimensional disease graph according to original hologram, and through least square method, left and right and before and after disease graph are spliced to form multi-size disease database, the combination of two-dimensional and three-dimensional disease graph data not only greatly improves data precision, but also improves image quality, and disease detection type covers various diseases of road surface, which can facilitate equipment to automatically identify road surface disease.

Description

Technical Field

[0001] This invention relates to the field of road maintenance data processing, specifically to a data processing method based on three-dimensional digital holography for fusion of road damage images. Background Technology

[0002] Since the advent of the first rapid pavement detection system in the 1970s, many scholars and researchers both domestically and internationally have been dedicated to the research of pavement distress image processing and automatic identification. This approach generally targets two-dimensional pavement images acquired by detection equipment, performing preprocessing such as image enhancement, then segmenting and extracting target features using methods such as thresholding, and finally classifying the distress. Image analysis methods can provide specific quantitative analysis of the segmented distress targets; however, because pavement distress image data is a non-linear detection target, image analysis methods face several challenges: complex and variable backgrounds, strong speckle noise, low target signal-to-noise ratio, low contrast between target and background, and poor spatial continuity of target pixels. Although researchers have proposed many different processing methods, a universal and effective method for automatically identifying various pavement distresses (cracks and deformations) has yet to be found. Currently, manual identification of distress images is the primary method, which is expensive and inefficient. The massive amount of three-dimensional point cloud data generated by three-dimensional laser detection technology makes intelligent identification difficult. Existing deep learning networks, due to the large amount and accuracy of distress data, struggle to achieve uniformity through multi-data fusion during training.

[0003] Therefore, how to combine new technologies to solve the problems of image quality, accuracy and automatic identification of road surface defects, and to establish a multi-size defect database has become an urgent technical problem to be solved. Summary of the Invention

[0004] To address the aforementioned problems, this invention aims to provide a data processing method for road surface distress image fusion based on three-dimensional digital holography.

[0005] To achieve this technical objective, the present invention provides a data processing method for road surface distress image fusion based on three-dimensional digital holography, the specific steps of which are as follows:

[0006] S1. A digital interference fringe image is generated by the laser interferometer in the three-dimensional digital holographic vehicle-mounted acquisition device and projected onto the road surface. The original image is obtained by acquiring the road surface with at least two industrial cameras. The original image is then cropped according to the effective acquisition range of the industrial cameras to obtain the original hologram.

[0007] S2. Use computer simulation of optical diffraction process to realize holographic reproduction and reconstruction of road surface, and obtain a three-dimensional holographic disease map with road surface depth data;

[0008] S3. Fourier multilevel matrix transformation and inverse Fourier multilevel matrix transformation are used to remove interference fringe information from the original hologram to obtain a two-dimensional defect map;

[0009] S4. Combine the three-dimensional holographic disease map and the two-dimensional disease map generated from the same original hologram to generate a fused disease map;

[0010] S5. The least squares fitting method is used to perform multi-size fusion of the three-dimensional holographic disease map and the two-dimensional disease map obtained from at least two industrial cameras to obtain a multi-size road surface dataset; the small-size road surface data generated in the first step is subjected to noise reduction processing to remove edge noise.

[0011] S6: Multi-size road surface datasets are fused to generate a 3D holographic disease database, a 2D disease database, and a fused disease database. These are then used to train a neural network model to obtain a fused model. The fused model is then tested and simulated using specified data to obtain the final training model.

[0012] Preferably, in step S2, the original hologram contains light intensity information and phase information of light waves with digital interference fringes on the road surface. Based on the relationship between light intensity and electromagnetic equations, a three-dimensional reconstruction is performed using Fourier fast transform to obtain the phase difference. The continuous phase difference changes are calculated to restore the overall depth information of the original hologram of the road surface, thus obtaining a three-dimensional holographic disease map with road surface depth data.

[0013] The specific digital holographic 3D reconstruction process is as follows:

[0014] S21. The electromagnetic energy equation for any pixel (x, y) in the original hologram is:

[0015] E(x,y)=A(x,y)exp(iφ(x,y))

[0016] Where φ(x,y) represents the phase information of any pixel (x,y), and A(x,y) represents the light intensity information of any pixel (x,y).

[0017] S22. The light intensity equation for any pixel (x, y) in the original hologram is:

[0018] I(x, y) = A(x, y) 2 =I(x,y)=|E(x,y)| 2 =E(x,y)E(x,y) * ;

[0019] S23, Fresnel digital diffraction transformation formula:

[0020]

[0021]

[0022] Where: Γ(ξ, β) is the light intensity and phase of the original hologram, λ is the wavelength of the light source, i represents a complex number, d is the vertical distance from the laser interferometer to the camera aperture plane, and ρ is the diffraction distance from the camera aperture to the laser interferometer plane;

[0023] S24. Fourier transform is equivalent to Fresnel transform. The formula transformation yields the 3D holographic phase information of any pixel (x, y) in the original hologram. Continuous Fourier transforms provide the overall phase difference of the original hologram, i.e., the depth information of all pixels in the original hologram, thus allowing for the 3D reconstruction of the road surface's holographic damage map.

[0024] I(x, y) = |Γ(ξ, β)| 2

[0025] .

[0026] Preferably, in step S3, the road surface information of the original hologram itself is obtained, and the Fourier multilevel formula of the light intensity A(x,y) of any pixel (x,y) in the original hologram is:

[0027] ;

[0028] According to the Fourier multilevel formula, remove the zeroth and first-order Fourier terms a0+a1(x,y). 1 The data is then processed by performing a multi-level inverse Fourier transform to obtain the two-dimensional pixels (x, y) of any pixel in the original hologram with the interference fringe information eliminated.

[0029] By performing Fourier multilevel matrix transformation and inverse Fourier multilevel matrix transformation on all pixels of the original hologram, digital interference fringes are eliminated to obtain a two-dimensional pavement defect map.

[0030] Preferably, in step S4, the three-dimensional holographic disease map and the two-dimensional disease map generated from the same original hologram are fused at the pixel level.

[0031] The specific data fusion process is as follows: The original hologram image itself is represented as matrix data. The three-dimensional holographic disease map generates matrix A, and the two-dimensional disease map generates matrix B. Under the optimization criterion, the matrix can be represented as the superposition of a low-rank matrix and a sparse matrix. Matrix C is obtained by adding matrix A and matrix B respectively. The augmented Lagrange multiplier method is used to solve for the low-rank matrix E and the sparse matrix F of matrix C. E and F are superimposed and transformed to generate the image, and finally the fused disease map is obtained.

[0032] Preferably, in step S5, the three-dimensional holographic disease images, two-dimensional disease images, and fused disease images obtained from different industrial cameras are stitched together to obtain a road surface map;

[0033] The least squares method is used to find the optimal fitting plane ax+by+cz+d=0 for the road surface map to be stitched. Using the fitting plane of one industrial camera as the reference, the road surface map of the other industrial camera is rotated so that the road surface maps on the left and right sides are completely on the same reference plane, and seamless stitching is performed to generate the left and right stitched defect map.

[0034] Similarly, by stitching together the left-right images of the road damage from two different industrial cameras, the deviation angle between the two planes is obtained using the spatial data of the fitted plane. The previous left-right stitched image is selected as the reference plane, and the images to be stitched are rotated in three-dimensional space. Finally, the three-dimensional data matrices to be stitched are spatially aligned to achieve seamless stitching. The front-to-back stitching can generate multi-size road maps.

[0035] Preferably, in step S6, feature engineering is performed on the three-dimensional holographic disease images and two-dimensional disease images of different sizes respectively;

[0036] First, some data is labeled. Then, the data is preprocessed by normalization and binarization to generate a 3D holographic disease image database, a 2D disease image database, a 3D hologram, and a fused disease database. The three databases are then segmented to generate three corresponding training sets, validation sets, and test sets. These sets are then fed into the pre-built neural network model for iterative training to obtain three training models. Two fusion models are obtained by weighted voting fusion and averaging fusion of the training models, respectively. These fusion models are then imported and tested on the three test sets. The model that meets the requirements is selected and exported as the training model.

[0037] The beneficial effects of this invention are that the digital holographic vehicle-mounted inspection equipment provides a rapid technology for large-scale road surface inspection. Based on the acquired original hologram, a three-dimensional defect map is obtained through reconstruction, and a two-dimensional defect map is obtained after removing interference fringes. By using the least squares method, the left and right and front and rear defect maps are stitched together to form a multi-size defect database. The combination of two-dimensional and three-dimensional defect map data not only greatly improves the data accuracy but also improves the image quality. The defect detection types cover various road surface defects (cracks and deformation, etc.), which can facilitate the automatic identification of road surface defects by the equipment. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart of the application;

[0040] Figure 2 This is the original hologram of the road surface (2*1 meter) captured by a single industrial camera in this application;

[0041] Figure 3 This is a 3D holographic image of the defect (2*1 meter) reconstructed by a single industrial camera in this application;

[0042] Figure 4 A two-dimensional defect image (2*1 meter) obtained by removing stripes from a single industrial camera in this application;

[0043] Figure 5 This is a fusion image of defects from a single industrial camera in this application (2*1 meters);

[0044] Figure 6 This is a multi-size stitched image (4*1 meter) of the three-dimensional holographic defect images from the left and right industrial cameras in this application;

[0045] Figure 7 This application presents a multi-size stitched image (4*1 meter) of defects created by fusing images from left and right industrial cameras.

[0046] Figure 8 This is a multi-size stitched image (4*1 meter) of two-dimensional defect images (de-striped) from the left and right industrial cameras in this application.

[0047] Figure 9 This is a stitched image of the three-dimensional holographic defect images from the left and right industrial cameras in this application (4*10 meters);

[0048] Figure 10 This is a 3D holographic data splicing display diagram (4*10 meters) of the left and right industrial cameras in this application;

[0049] Figure 11 This is a stitched image of the 3D point cloud data of the left and right industrial cameras in this application (4*10 meters);

[0050] Figure 12 This is a display of the disease identification results in this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will be described in conjunction with the appendices in the embodiments of the present invention. Figure 1-12The technical solutions in the embodiments of the present invention are clearly and completely described herein. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] To better illustrate the working process of this invention, the principle of its implementation will be explained below.

[0053] Two-dimensional image stitching is easily affected by ambient light and road surface features, resulting in noticeable seams and uneven brightness in the stitched image. Furthermore, two-dimensional images lack depth information, making it difficult to identify road surface deformation-related defects such as potholes, subsidence, and bulges. Common methods for three-dimensional point cloud stitching include those relying on external feature points and those relying on overlapping data from multiple measurements, but neither is suitable for detecting road surface defects, which requires rapid detection by vehicle-mounted equipment. Additionally, large-scale stitching of point cloud data is difficult; generally, only sparse point cloud data can be used for display, significantly reducing data accuracy and quality. The fusion of two-dimensional and three-dimensional data, due to their different sources, typically involves only simple data overlay or feature fusion. However, road surfaces themselves rarely provide reference features, and even with defective surfaces, feature extraction is a crucial problem to solve.

[0054] Therefore, in order to solve the problem of the common source of two-dimensional and three-dimensional data, improve the quality and accuracy of road surface distress acquisition images, reduce data volume and solve the problem of automatic identification, the main purpose of this invention is to establish a multi-scale distress database and multi-model fusion for automatic identification of road surface distress.

[0055] Example 1:

[0056] Step 1: To ensure accurate acquisition of single-lane width data (3.75 meters), a digital interference fringe image is generated using a laser interferometer and projected onto the road surface (covering a width of 4.2 meters). At least two industrial cameras are used to capture images of the road surface to obtain the original image. The original image is then cropped according to the effective acquisition range of the two industrial cameras to obtain the original hologram. Figure 2 These are the original holograms of the road surface (2*1 meters) captured by the left and right single cameras respectively.

[0057] Step Two: To obtain high-precision three-dimensional information, computer simulation of the optical diffraction process is used to achieve holographic reconstruction of the road surface and to restore and reconstruct the original hologram. (Reference) Figure 3The three-dimensional holographic damage map (2*1 meter) is reconstructed by a single camera on both sides. The original hologram contains light intensity information and phase information of the light wave with digital interference fringes on the road surface. Based on the relationship between light intensity and electromagnetic equations, a three-dimensional reconstruction is performed using Fourier fast transform to obtain the phase difference. The continuous phase difference changes are calculated to reconstruct the overall depth information of the original road surface hologram, thus obtaining a three-dimensional holographic damage map with road surface depth data. The specific digital holographic three-dimensional reconstruction process is as follows:

[0058] The electromagnetic energy equation for any pixel (x, y) in the original hologram is:

[0059] E(x,y)=A(x,y)exp(iφ(x,y))

[0060] Where φ(x,y) represents the phase information of any pixel (x,y), and A(x,y) represents the light intensity information of any pixel (x,y).

[0061] The light intensity equation for any pixel (x, y) in the original hologram is:

[0062] I(x, y) = A(x, y) 2 =I(x,y)=|E(x,y)| 2 =E(x,y)E(x,y) * ;

[0063] Fresnel digital diffraction transform formula:

[0064]

[0065]

[0066] Where: Γ(ξ, β) is the light intensity and phase of the original hologram, λ is the wavelength of the light source, i represents a complex number, d is the vertical distance from the laser interferometer to the camera aperture plane, and ρ is the diffraction distance from the camera aperture to the laser interferometer plane;

[0067] Fourier transform is equivalent to Fresnel transform. Through formula transformation, the 3D holographic phase information of any pixel (x, y) in the original hologram is obtained. Through continuous Fourier transforms, the overall phase difference of the original hologram is obtained, which represents the depth information of all pixels in the original hologram. Thus, a 3D holographic image of road surface defects is reconstructed.

[0068] I(x, y) = |Γ(ξ, β)| 2

[0069] .

[0070] Step 3: The original hologram itself contains road surface information. Removing interference fringes yields a two-dimensional map of the road surface defects. (Refer to...) Figure 4 Two-dimensional disease images of stripes removed by single cameras on the left and right sides.

[0071] The Fourier multilevel formula for the light intensity A(x,y) at any pixel (x,y) in the original hologram:

[0072]

[0073] Remove the zeroth and first-order Fourier transforms a0+a1(x,y). 1 The data is then processed, and a multi-level Fourier inverse transform is performed to obtain two-dimensional pixels (x, y) of any pixel in the original hologram with the interference fringe information eliminated. By performing multi-level Fourier matrix transformation and inverse Fourier matrix transformation on all pixels in the original hologram, digital interference fringes are eliminated to obtain a two-dimensional road surface defect map.

[0074] Step 4: The 3D holographic disease map contains depth information, which can be used to analyze deformation-type road surface defects; the 2D disease map has road surface texture features, which can be used to analyze crack-type road surface defects. However, both have missing information. To extract more detailed disease features, it is necessary to perform pixel-level matrix data fusion on these two sources of data: the 3D holographic disease map and the 2D disease map. Therefore, we first generate matrix A from the 3D holographic disease map and matrix B from the 2D disease map. Under the optimization criterion, the matrix can be represented as the superposition of a low-rank matrix and a sparse matrix. By adding matrices A and B respectively, we obtain matrix C. We then use the augmented Lagrange multiplier method to solve for the low-rank matrix E and the sparse matrix F of matrix C. We then superimpose and transform E and F respectively to generate the final fused disease map. The results are shown in the reference. Figure 5 .

[0075] Step 5: Because some road surface defects cover a large area, exceeding the effective shooting range of a single camera, it is necessary to stitch photos taken by different cameras side-by-side and front-to-back to generate a larger-sized stitched defect image in order to identify larger defects. Since the 3D digital holographic vehicle-mounted acquisition equipment uses a simultaneous shooting method with different industrial cameras, firstly, the small-sized original holographic road surface data generated in the first step is denoised, and edge noise is removed. Secondly, the least squares method is used to find the optimal fitting plane ax+by+cz+d=0 for the road surface images to be stitched. Using the fitting plane of one industrial camera as a reference, the road surface image from the other industrial camera is rotated so that the left and right road surface images are completely on the same reference plane, allowing for seamless stitching and generating a left-right stitched defect image (results reference). Figure 6 The image shows a multi-size stitched image of a defect from an industrial camera (4*1 meter) and a multi-size stitched image of a defect from an industrial camera (4*1 meter). Figure 8Multi-size stitched images (4*1 meter) of 2D road surface defects obtained from left and right industrial cameras (with stripe removal); similarly, left and right stitched defect images from two different industrial cameras are stitched together front and back. The deviation angle between the two planes is obtained using spatial data from the fitted plane. The previous left and right stitched image is selected as the reference plane. The images to be stitched are rotated in 3D space. Finally, the 3D data matrices to be stitched are spatially aligned to achieve seamless stitching. Front and back stitching can generate multi-size road surface maps (results reference). Figure 9 A stitched image of the road surface damage from left and right cameras (4*10 meters) is generated, resulting in a road surface dataset of multiple sizes. Additionally, for demonstration purposes, a 3D display of the 3D hologram and the stitched image can be provided (see reference). Figure 10 and Figure 11 .

[0076] Step Six: Establish multi-size pavement distress datasets, and perform feature engineering on distress databases of different sizes. First, label some data, then preprocess the data through normalization and binarization to generate a 3D holographic distress image database, a 2D distress image database, a 3D hologram, and a fused distress database. Divide the data into the three databases to generate three corresponding training sets, validation sets, and test sets. Substitute these sets into the pre-built neural network model and perform multiple iterations to obtain three training models. Two fusion models are obtained through weighted voting fusion and averaging fusion methods, respectively. Import the fusion models and test them on the three test sets. The model that meets the requirements is exported as the training model.

[0077] To verify the effectiveness and authenticity of the multi-scale disease training results, we performed feature extraction and model fusion based on existing training data, derived the training model, and tested it on disease maps at different scales. (Reference) Figure 12 The disease identification results show that, on test disease images at different scales (12a: 3D hologram from the left camera (2*1 meter), 12b: fused disease image from the left camera (2*1 meter), 12c: multi-size stitched image of 3D holograms from both cameras (4*1 meter), and 12d: multi-size stitched image of fused disease images from both cameras (4*1 meter)), the cracks (located within the effective range of the left camera) were all well identified.

[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any minor modifications, equivalent substitutions, and improvements made to the above embodiments based on the technical essence of the present invention should be included within the protection scope of the present invention.

Claims

1. A data processing method for road surface distress image fusion based on three-dimensional digital holography, characterized in that, The specific steps are as follows: S1. A digital interference fringe image is generated by the laser interferometer in the three-dimensional digital holographic vehicle-mounted acquisition device and projected onto the road surface. The original image is obtained by acquiring the road surface with at least two industrial cameras. The original image is then cropped according to the effective acquisition range of the industrial cameras to obtain the original hologram. S2. Use computer simulation of optical diffraction process to realize holographic reproduction and reconstruction of road surface, and obtain a three-dimensional holographic disease map with road surface depth data; S3. Fourier multilevel matrix transformation and inverse Fourier multilevel matrix transformation are used to remove interference fringe information from the original hologram, thereby obtaining a two-dimensional defect map; S4. Combine the three-dimensional holographic disease map and the two-dimensional disease map generated from the same original hologram to generate a fused disease map; S5. The least squares fitting method is used to fuse the three-dimensional holographic disease images and two-dimensional disease images obtained from at least two industrial cameras into multi-size data sets. The small-size road data generated in the first step is denoised by removing edge noise. In step S5, the three-dimensional holographic disease images, two-dimensional disease images, and fused disease images acquired by different industrial cameras are stitched together to obtain a road surface map; The least squares method is used to find the optimal fitting plane ax+by+cz+d = 0 for the road surface maps to be stitched. Using the fitting plane of one industrial camera as a reference, the road surface map of the other industrial camera is rotated so that the left and right road surface maps are completely on the same reference plane, and seamless stitching is performed to generate the left and right stitched defect map. The left-right stitched images of the road damage from two different industrial cameras are stitched together front-to-back. The deviation angle between the two planes is obtained by fitting the spatial data of the plane. The previous left-right stitched image is selected as the reference plane. The images to be stitched are rotated in three-dimensional space. Finally, the three-dimensional data matrix to be stitched is spatially docked to achieve seamless stitching. The front-to-back stitching can generate multi-size road maps. S6: Multi-size road surface datasets are fused to generate a 3D holographic disease database, a 2D disease database, and a fused disease database. These are then used to train a neural network model to obtain a fused model. The fused model is then tested and simulated using specified data to obtain the final training model.

2. The data processing method for road surface distress image fusion based on three-dimensional digital holography according to claim 1, characterized in that: In step S2, the original hologram contains light intensity information and phase information of light waves with digital interference fringes on the road surface. Based on the relationship between light intensity and electromagnetic equations, a three-dimensional reconstruction is performed using Fourier fast transform to obtain the phase difference. The continuous phase difference changes are calculated to restore the overall depth information of the original hologram of the road surface, thus obtaining a three-dimensional holographic disease map with road surface depth data. The specific digital holographic 3D reconstruction process is as follows: S21. Any pixel in the original hologram The electromagnetic energy equation is: ; in For any pixel (x,y), The light intensity information of any pixel (x, y); S22, Arbitrary pixels of the original hologram The equation for light intensity is: ； S23, Fresnel digital diffraction transformation formula: in: The light intensity and phase of the original hologram. The wavelength of the light source, Let d represent a complex number, and d be the perpendicular distance from the laser interferometer to the camera aperture plane. The diffraction distance from the camera aperture to the plane of the laser interferometer; S24. The Fourier transform is equivalent to the Fresnel transform; arbitrary pixels of the original hologram can be obtained through formula transformation. Three-dimensional holographic phase information is obtained by continuously transforming the phase difference of the original hologram, which is the depth information of all pixels in the original hologram. This allows for the three-dimensional reconstruction of the road surface into a three-dimensional holographic image of road damage. 。 3. The data processing method for road surface distress image fusion based on three-dimensional digital holography according to claim 1, characterized in that: In step S3, the road surface information of the original hologram is obtained, and the light intensity of any pixel (x,y) in the original hologram is... Fourier multilevel formula: ； According to the Fourier multilevel formula, remove the zeroth and first-order Fourier terms. The data is then processed by performing a multi-level inverse Fourier transform to obtain any pixel in the original hologram with interference fringe information eliminated. Two-dimensional pixels; By performing Fourier multilevel matrix transformation and inverse Fourier multilevel matrix transformation on all pixels of the original hologram, digital interference fringes are eliminated to obtain a two-dimensional pavement defect map.

4. The data processing method for road surface distress image fusion based on three-dimensional digital holography according to claim 1, characterized in that: In step S4, the three-dimensional holographic disease map and the two-dimensional disease map generated from the same original hologram are fused at the pixel level. The specific data fusion process is as follows: The original hologram image itself is represented as matrix data. The three-dimensional holographic disease map generates matrix A, and the two-dimensional disease map generates matrix B. Under the optimization criterion, the matrix can be represented as the superposition of a low-rank matrix and a sparse matrix. Matrix C is obtained by adding matrix A and matrix B respectively. The augmented Lagrange multiplier method is used to solve for the low-rank matrix E and the sparse matrix F of matrix C. E and F are superimposed and transformed to generate the image, and finally the fused disease map is obtained.

5. The data processing method for road surface distress image fusion based on three-dimensional digital holography according to claim 1, characterized in that: In step S6, feature engineering is performed on the three-dimensional holographic disease images and two-dimensional disease images of different sizes respectively; First, some data is labeled. Then, the data is preprocessed by normalization and binarization to generate a 3D holographic disease image database, a 2D disease image database, a 3D hologram, and a fused disease database. The three databases are then segmented to generate three corresponding training sets, validation sets, and test sets. These sets are then fed into the pre-built neural network model for iterative training to obtain three training models. Two fusion models are obtained by weighted voting fusion and averaging fusion of the training models, respectively. These fusion models are then imported and tested on the three test sets. The model that meets the requirements is selected and exported as the training model.

Images