A road texture reconstruction method based on recursive super-resolution network

By adopting a road surface texture reconstruction method based on a recurrent super-resolution network, the problem of high-frequency information recovery of low-resolution texture data under high-speed driving of vehicle-mounted 3D line laser scanning equipment is solved. It realizes efficient unidirectional and stepwise super-resolution reconstruction, recovers high-resolution road surface texture, and is suitable for high-magnification super-resolution reconstruction of single-channel road surface texture.

CN118279150BActive Publication Date: 2024-11-19SOUTHWEST JIAOTONG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410435850.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-11
Publication Date
2024-11-19
Estimated Expiration
2044-04-11

AI Technical Summary

Technical Problem

Existing technologies struggle to effectively recover high-frequency information from low-resolution road texture data collected by vehicle-mounted 3D line laser scanning equipment at high speeds. Traditional image interpolation methods are ineffective, and deep learning-based methods are not suitable for unidirectional and hierarchical super-resolution reconstruction of single-channel road textures.

Method used

A road surface texture reconstruction method based on a recurrent super-resolution network is adopted. By normalizing and downsampling at different scaling levels, road surface texture data are obtained, a super-resolution network is constructed, and the network is trained using a recurrent method. The network parameters are optimized by combining image quality and texture parameter indicators, and finally, a 16x super-resolution reconstruction of low-resolution textures is achieved.

Benefits of technology

While maintaining a good average texture depth relative error, it achieves 16x super-resolution reconstruction of low-resolution textures, restores high-resolution road surface textures, adapts to the need for continuous high-precision data acquisition under high-speed driving, and the reconstruction quality is superior to traditional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118279150B_ABST
    Figure CN118279150B_ABST
Patent Text Reader

Abstract

The present invention discloses a road texture reconstruction method based on a cyclic recursive super-resolution network. The method comprises the following steps: obtaining road texture data with different resolutions at different zoom levels; pairing the same road textures with two times the resolution difference to obtain two times the resolution difference data pairs; constructing a super-resolution network, initializing and optimizing the super-resolution network parameters; using the super-resolution network after optimizing the parameters, reconstructing the road texture data with different resolutions at different zoom levels step by step to the initial resolution texture data, and obtaining the reconstruction quality evaluation result of the super-resolution network using the image quality index and the texture parameter index. The present invention can achieve up to 16 times super-resolution reconstruction of low-resolution textures while maintaining a good average texture depth relative error, thereby meeting the requirements of unidirectional super-resolution and step-by-step super-resolution reconstruction of single-channel road textures.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of road surface texture super-resolution, and in particular to a road surface texture reconstruction method based on a cyclic recursive super-resolution network. Background Art

[0002] As a characteristic of the road surface, pavement texture not only provides anti-skid performance for vehicle tires but also can quickly drain water from the road surface, and has a significant relationship with driving risks. Given the complexity and randomness of pavement texture distribution, it is necessary to collect a large amount of three-dimensional pavement texture data with an initial resolution (0.5 mm / pixel and below) to study the relationship between pavement texture and the surface function it provides. With the development of optical technology and equipment, three-dimensional texture data of the road surface can be collected at a certain driving speed through vehicle-mounted three-dimensional line laser scanning equipment. When the driving speed of the vehicle-mounted three-dimensional line laser scanning equipment increases, and the sampling rate of the equipment is fixed, the resolution of the collected three-dimensional texture of the road surface in the driving direction decreases. For example, when the driving speed is 80 km / h, the texture data resolution corresponding to the driving speed is only 3.2 mm / pixel. The texture data under this working condition loses a lot of high-frequency information, which makes it difficult to meet the data quality requirements required for studying the relationship between pavement texture and road surface characteristics. Improving accuracy by upgrading hardware equipment is limited by huge costs, and traditional image interpolation methods, such as nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation, are essentially unable to restore the lost high-frequency information. In particular, when restoring high-resolution images, the visual effect is poor. Using deep learning-based image super-resolution reconstruction methods to restore low-resolution images to high-resolution images has become a possible solution to this problem.

[0003] At present, the image super-resolution reconstruction method based on self-supervised deep learning restores high-resolution images by training paired high-low resolution image pairs, learning prior information and mapping features. Compared with traditional interpolation algorithms, this method can restore certain high-frequency information. However, most image super-resolution reconstruction algorithms based on deep learning only perform bilateral reconstruction on the RGB three-channel image set and can only achieve a single fixed magnification. It is not suitable for the needs of unidirectional super-resolution and step-by-step super-resolution reconstruction of single-channel road texture in this field. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a road texture reconstruction method based on a recursive super-resolution network, which can achieve up to 16 times super-resolution reconstruction of low-resolution texture while maintaining a good average texture depth relative error, thereby meeting the needs of unidirectional super-resolution and step-by-step super-resolution reconstruction of single-channel road texture.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0006] A road surface texture reconstruction method based on a recursive super-resolution network comprises the following steps:

[0007] S1, obtaining initial resolution road surface texture data, and performing normalization processing and unidirectional multi-level downsampling processing on the initial resolution road surface texture data in turn to obtain resolution road surface texture data at different zoom levels;

[0008] S2, according to the resolution road surface texture data at different zoom levels in step S1, pairing the same road surface texture with a two-fold resolution difference to obtain a two-fold resolution difference data pair;

[0009] S3, constructing a super-resolution network, initializing super-resolution network parameters, and training the super-resolution network with the double-resolution difference data in step S2 in ascending order by a cyclic recursive method to obtain a super-resolution network with optimized parameters;

[0010] S4. Using the super-resolution network after optimizing the parameters in step S3, the road texture data with different zoom levels are reconstructed step by step to the initial resolution texture data, and the reconstruction quality evaluation result of the super-resolution network is obtained using the image quality index and the texture parameter index.

[0011] Furthermore, in step S1, the initial resolution road surface texture data is normalized, including the following steps:

[0012] A1. Standardize the initial resolution road texture data, expressed as;

[0013]

[0014] Where: x1 is the normalized initial resolution road surface texture data, x0 is the initial resolution road surface texture data, μ is the average value of the initial resolution road surface texture data, σ is the standard deviation of the initial resolution road surface texture data;

[0015] A2. Map the range of the initial resolution road texture data normalized in step A1 to (-1,1), expressed as:

[0016]

[0017] Where: x2 is the initial resolution road surface texture data with a range mapped to (-1,1);

[0018] A3, mapping the value range of the initial resolution road surface texture data whose value range is mapped to (-1, 1) in step A2 to (0, 1), and obtaining the normalized initial resolution road surface texture data, which is expressed as:

[0019]

[0020] Where: x3 is the initial resolution road texture data after normalization.

[0021] Furthermore, in step S1, the normalized initial resolution road surface texture data set is subjected to unidirectional multi-level downsampling processing to obtain resolution road surface texture data at different zoom levels, specifically: according to the normalized initial resolution road surface texture data set, each piece of initial resolution road surface texture data is downsampled according to different zoom factors, and the downsampling zoom factors are 1 / 2, 1 / 4, 1 / 8, and 1 / 16, respectively, to obtain 4 groups of resolution road surface texture data at different zoom levels.

[0022] Furthermore, in step S2, the same road surface textures with twice the resolution difference are paired to obtain twice the resolution difference data pairs. The specific process is: for the same road surface texture, 1 / 16 level resolution texture is paired with 1 / 8 level resolution texture, 1 / 8 level resolution texture is paired with 1 / 4 level resolution texture, 1 / 4 level resolution texture is paired with 1 / 2 level resolution texture, 1 / 2 level resolution texture is paired with original high resolution texture, and 4 sets of twice the resolution difference data pairs are obtained.

[0023] Furthermore, in step S3, the constructed super-resolution network includes an input layer, a first convolutional layer, a first activation function layer, a residual module unit, a second convolutional layer, a third convolutional layer, a sub-pixel convolutional layer, a fourth convolutional layer, a second activation function layer, a fifth convolutional layer and a first output layer connected in sequence; the output of the first activation function layer is added to the output of the second convolutional layer in the channel direction.

[0024] Furthermore, the number of convolution kernels of the first and second convolution layers is 64, the size of the convolution kernel is 3×3, the direction step is (1,1), and the edge padding size is 1; the number of convolution kernels of the third convolution layer is 256, the size of the convolution kernel is 3×3, the direction step is (1,2), and the edge padding size is 1; the number of convolution kernels of the fourth convolution layer is 256, the size of the convolution kernel is 3×3, the direction step is (1,1), and the edge padding size is 1; the number of convolution kernels of the fifth convolution layer is 1, the size of the convolution kernel is 1×1, the direction step is (1,1), and the edge padding size is 1; the first activation function layer and the second activation function layer both use the ReLU activation function; the upsampling factor used in the sub-pixel convolution layer is 2, which is used to double the width and height of the feature layer; the first output layer directly outputs without any activation function.

[0025] Furthermore, the residual module unit includes 16 residual modules of the same structure; the residual module includes an input layer, a first deep convolution layer, a first deep activation layer, a second deep convolution, a residual connection and a second output layer; the number of convolution kernels of the first deep convolution layer and the second deep convolution layer are both 64, the convolution kernel size is 3×3, the direction step size is (1,1), and the edge padding size is 1; the first deep activation layer adopts the ReLU activation function; the output of the second deep convolution layer is added to the input layer in the channel direction; the second output layer directly outputs without any activation function.

[0026] Further, in step S3, the double-resolution difference data pairs in step S2 are sequentially used in ascending order to train the super-resolution network by a cyclic recursive method. The specific process is: first, the 1 / 16 level resolution texture data in the double-resolution difference data pair is used as input, and the super-resolution texture data outputted by the super-resolution network is 1 / 8 level, and the loss function is calculated with the 1 / 8 level resolution texture data in the double-resolution difference data pair, and the super-resolution network parameters are updated according to the back-propagation gradient of the optimizer; then, the 1 / 8 level resolution texture data in the double-resolution difference data pair is used as input, and the super-resolution texture data outputted by the super-resolution network is 1 / 4 level, and the loss function is calculated with the 1 / 4 level resolution texture data in the double-resolution difference data pair. The loss function is calculated based on the gradient backpropagation of the optimizer, and the super-resolution network parameters are updated according to the gradient backpropagation of the optimizer; the 1 / 4 level resolution texture data in the double resolution difference data pair is taken as input, and the output of the super-resolution network is 1 / 2 level super-resolution texture data, and the loss function is calculated with the 1 / 2 level resolution texture data in the double resolution difference data pair, and the super-resolution network parameters are updated according to the gradient backpropagation of the optimizer; finally, the 1 / 2 level resolution texture data in the double resolution difference data pair is taken as input, and the output of the super-resolution network is 1 / 2 level super-resolution texture data, and the loss function is calculated with the initial resolution road surface texture data in the double resolution difference data pair, and the super-resolution network parameters are updated according to the gradient backpropagation of the optimizer.

[0027] Furthermore, in step S3, a super-resolution network after optimizing parameters is obtained, and the specific process is as follows: during the iterative training of the super-resolution network, the loss function value of the corresponding resolution texture data between the super-resolution texture data and the twice-resolution difference data is calculated; the road surface texture super-resolution network is trained in two stages, the loss type of the first stage adopts the L1 norm, and the loss type of the second stage adopts the mean square error loss; the optimizer completes the gradient calculation and performs gradient back propagation according to the loss function value; the minimum loss function value that occurs during the road surface texture super-resolution training process is recorded, and the corresponding weight parameters are saved to obtain the super-resolution network after optimizing parameters.

[0028] Furthermore, in step S4, the image quality evaluation index adopts peak signal-to-noise ratio and structural similarity, and the texture parameter index adopts average texture depth relative error.

[0029] The beneficial effects of the present invention are:

[0030] (1) The present invention can achieve up to 16 times super-resolution reconstruction of low-resolution textures while maintaining a good average texture depth relative error, thereby meeting the requirements for unidirectional super-resolution and step-by-step super-resolution reconstruction of single-channel road textures;

[0031] (2) The present invention adaptively constructs a super-resolution network based on cyclic recursion to solve the problem that the resolution of road surface texture data in the driving direction decreases due to the increase in speed of the vehicle-mounted three-dimensional line laser scanning device during driving, so that the low-resolution road surface texture can be reconstructed into a super-resolution road surface texture step by step in the driving direction until the high-resolution road surface texture (initial resolution road surface texture) is restored;

[0032] (3) The present invention uses image quality indicators and texture parameter indicators to obtain the reconstruction quality evaluation results of the super-resolution network, and the reconstructed super-resolution road texture is superior to the traditional interpolation method in terms of evaluation indicators and visual restoration effects, especially high-multiple super-resolution reconstruction;

[0033] (4) The present invention can perform up to 16 times of super-resolution reconstruction of road surface texture step by step, which not only enables continuous acquisition of high-precision road surface texture under high-speed driving, but also the quality of the reconstructed super-resolution texture data can meet the data requirements required for actual related research. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of a road texture reconstruction method based on a recursive super-resolution network;

[0035] Figure 2 This is a structural diagram of a road surface texture super-resolution network according to the method of an embodiment of the present invention;

[0036] Figure 3 This is a structural diagram of a residual module in a road surface texture super-resolution network according to the method of an embodiment of the present invention;

[0037] Figure 4 This is a flowchart of a super-resolution network iteration according to an embodiment of the present invention;

[0038] Figure 5 The training peak signal-to-noise ratio and structural similarity curve of the 1 / 2 level resolution road surface texture described in the embodiment of the present invention;

[0039] Figure 6The comparison result diagram of the two-dimensional images of road surface textures at various levels after reconstruction shown in the embodiment of the present invention;

[0040] Figure 7 This is a comparison result diagram of the reconstructed three-dimensional images of road surface textures at various levels shown in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0042] like Figure 1 As shown, a road surface texture reconstruction method based on a recursive super-resolution network includes steps S1-S4, which are as follows:

[0043] S1. Acquire initial resolution road surface texture data, perform normalization processing and unidirectional multi-level downsampling processing on the initial resolution road surface texture data in sequence, and acquire resolution road surface texture data at different zoom levels.

[0044] In an optional embodiment of the present invention, the present invention obtains initial resolution road surface texture data, and sequentially performs normalization processing and unidirectional multi-level downsampling processing on the initial resolution road surface texture data to obtain four groups of resolution road surface texture data at different zoom levels of 1 / 2, 1 / 4, 1 / 8 and 1 / 16. The present invention obtains 1000 pieces of initial resolution road surface texture data, with a size of 512×512 and a resolution of 0.2 mm / pixel, corresponding to a driving speed of 5 km / h for the acquisition device.

[0045] The present invention performs normalization processing on initial resolution road surface texture data, comprising the following steps:

[0046] A1. Standardize the initial resolution road texture data, expressed as;

[0047]

[0048] Wherein: x1 is the normalized initial resolution road surface texture data, x0 is the initial resolution road surface texture data, μ is the average value of the initial resolution road surface texture data, and σ is the standard deviation of the initial resolution road surface texture data.

[0049] A2. Map the range of the initial resolution road texture data normalized in step A1 to (-1,1), expressed as:

[0050]

[0051] Where: x2 is the initial resolution road surface texture data with a range mapped to (-1,1).

[0052] A3, mapping the value range of the initial resolution road surface texture data whose value range is mapped to (-1, 1) in step A2 to (0, 1), and obtaining the normalized initial resolution road surface texture data, which is expressed as:

[0053]

[0054] Where: x3 is the initial resolution road texture data after normalization.

[0055] The present invention performs unidirectional multi-level downsampling processing on the normalized initial resolution road surface texture data set to obtain resolution road surface texture data at different zoom levels. Specifically, according to the normalized initial resolution road surface texture data set, each piece of initial resolution road surface texture data is downsampled according to different zoom factors, and the downsampling zoom factors are 1 / 2, 1 / 4, 1 / 8, and 1 / 16, respectively, to obtain 4 groups of resolution road surface texture data at different zoom levels.

[0056] Specifically, the resolutions of the road surface texture data of four groups with different zoom levels of 1 / 2, 1 / 4, 1 / 8 and 1 / 16 in the present invention are 0.4 mm / pixel, 0.8 mm / pixel, 1.6 mm / pixel and 3.2 mm / pixel respectively.

[0057] S2. Pair the same type of road surface textures with a two-fold resolution difference according to the resolution road surface texture data at different zoom levels in step S1 to obtain a two-fold resolution difference data pair.

[0058] In an optional embodiment of the present invention, the present invention pairs the same road surface textures with two times the resolution difference to obtain two times the resolution difference data pairs, and the specific process is: for the same road surface texture, 1 / 16 level resolution texture is paired with 1 / 8 level resolution texture, 1 / 8 level resolution texture is paired with 1 / 4 level resolution texture, 1 / 4 level resolution texture is paired with 1 / 2 level resolution texture, 1 / 2 level resolution texture is paired with original high resolution texture, and 4 groups of two times resolution difference data pairs are obtained.

[0059] S3, construct a super-resolution network, initialize the super-resolution network parameters, and train the super-resolution network in ascending order using the double-resolution difference data in step S2 by a cyclic recursive method to obtain a super-resolution network with optimized parameters.

[0060] In an optional embodiment of the present invention, the present invention constructs a super-resolution network, initializes super-resolution network parameters, and trains the super-resolution network in ascending order using a cyclic recursive method on the double-resolution difference data in step S2 to obtain a super-resolution network with optimized parameters.

[0061] like Figure 2 As shown, the super-resolution network constructed by the present invention includes an input layer, a first convolution layer, a first activation function layer, a residual module unit, a second convolution layer, a third convolution layer, a sub-pixel convolution layer, a fourth convolution layer, a second activation function layer, a fifth convolution layer and a first output layer connected in sequence; the output of the first activation function layer is added to the output of the second convolution layer in the channel direction to achieve the fusion of shallow features and deep features, which can avoid the risks of gradient diffusion and gradient explosion.

[0062] The number of convolution kernels of the first and second convolution layers are both 64, the convolution kernel size is 3×3, the direction stride is (1,1), and the edge padding size is 1; the number of convolution kernels of the third convolution layer is 256, the convolution kernel size is 3×3, the direction stride is (1,2), and the edge padding size is 1; the number of convolution kernels of the fourth convolution layer is 256, the convolution kernel size is 3×3, the direction stride is (1,1), and the edge padding size is 1; the number of convolution kernels of the fifth convolution layer is 1, the convolution kernel size is 1×1, the direction stride is (1,1), and the edge padding size is 1; the first activation function layer and the second activation function layer both use the ReLU activation function; the upsampling factor used in the sub-pixel convolution layer is 2, which is used to double the width and height of the feature layer; the first output layer directly outputs without any activation function.

[0063] The residual module unit includes 16 residual modules with the same structure. Figure 3 As shown in the figure, the residual module includes an input layer, a first deep convolution layer, a first deep activation layer, a second deep convolution, a residual connection and a second output layer; the number of convolution kernels in the first deep convolution layer and the second deep convolution layer are both 64, the convolution kernel size is 3×3, the direction step size is (1,1), and the edge padding size is 1; the first deep activation layer adopts the ReLU activation function; the output of the second deep convolution layer is added to the input layer in the channel direction; the second output layer directly outputs without any activation function.

[0064] The present invention initializes super-resolution network parameters, including weights of all convolutional layers in the super-resolution network, learning rate, batch size, number of training iterations, optimizer type, and loss function type.

[0065] The present invention trains a super-resolution network by a cyclic recursive method using the double-resolution difference data pairs in step S2 in ascending order. The specific process is: first, the 1 / 16-level resolution texture data in the double-resolution difference data pair is used as input, and the super-resolution texture data outputted by the super-resolution network is 1 / 8-level super-resolution texture data, and the loss function is calculated with the 1 / 8-level resolution texture data in the double-resolution difference data pair, and the super-resolution network parameters are updated according to the back-propagation gradient of the optimizer; then, the 1 / 8-level resolution texture data in the double-resolution difference data pair is used as input, and the super-resolution texture data outputted by the super-resolution network is 1 / 4-level super-resolution texture data, and the loss function is calculated with the 1 / 4-level resolution texture data in the double-resolution difference data pair. Function calculation, and update the super-resolution network parameters according to the back-propagation gradient of the optimizer; then take the 1 / 4 level resolution texture data in the double resolution difference data pair as input, output it as 1 / 2 level super-resolution texture data through the super-resolution network, and calculate the loss function with the 1 / 2 level resolution texture data in the double resolution difference data pair, and update the super-resolution network parameters according to the back-propagation gradient of the optimizer; finally, take the 1 / 2 level resolution texture data in the double resolution difference data pair as input, output it as initial super-resolution texture data through the super-resolution network, and calculate the loss function with the initial resolution road surface texture data in the double resolution difference data pair, and update the super-resolution network parameters according to the back-propagation gradient of the optimizer. The process of one iteration in training is as follows: Figure 4 shown.

[0066] In each iterative training, the former of the two-resolution difference data pair is input as relatively low-resolution texture data into the super-resolution network for 2-fold magnification reconstruction, and is output as super-resolution texture data through the super-resolution network, while the latter of the two-resolution difference data pair is used as relatively high-resolution texture data. The present invention randomly obtains two-resolution difference data pairs of a preset batch size, and after one iterative training is completed, a new batch of two-resolution difference data pairs of a preset batch size are randomly obtained to cyclically recursively train the super-resolution network according to the above steps until the set number of training iterations is completed.

[0067] The present invention obtains a super-resolution network after optimizing parameters, and the specific process is: in the iterative training process of the super-resolution network, the loss function value of the corresponding resolution texture data between the super-resolution texture data and the double resolution difference data is calculated; the road surface texture super-resolution network is trained in two stages, the loss type of the first stage adopts the L1 norm, and the loss type of the second stage adopts the mean square error loss; the optimizer completes the gradient calculation and performs the gradient back propagation according to the loss function value, so that a smaller loss function value is obtained when the network completes the next iteration, and the smaller the loss function value is, the smaller the error between the reconstructed super-resolution texture data and the relatively high-resolution texture is; the minimum loss function value appearing in the road surface texture super-resolution training process is recorded, and the corresponding weight parameters are saved to obtain the super-resolution network after optimizing parameters.

[0068] The loss type of the first stage in the present invention adopts the L1 norm, which is expressed as:

[0069]

[0070] Where: L1_Loss is the L1 norm loss of the first stage, N is the selected batch size, I HR i is the i-th relatively high-resolution texture data in the selected batch of double-resolution difference data pairs, I SR i is the i-th super-resolution texture data output by the super-resolution network in the selected batch of double-resolution difference data pairs, and ||·||1 is the L1 norm.

[0071] The loss type of the second stage in the present invention adopts mean square error loss, which is expressed as:

[0072]

[0073] Where: MSE_Loss is the mean square error loss of the second stage.

[0074] S4. Using the super-resolution network after optimizing the parameters in step S3, the road texture data with different zoom levels are reconstructed step by step to the initial resolution texture data, and the reconstruction quality evaluation result of the super-resolution network is obtained using the image quality index and the texture parameter index.

[0075] In an optional embodiment of the present invention, the present invention utilizes the super-resolution network after optimizing the parameters in step S3 to reconstruct the resolution road texture data of different zoom levels step by step to the initial resolution texture data, and uses the peak signal-to-noise ratio and structural similarity as image quality evaluation indicators, and uses the average texture depth relative error as the texture parameter indicator to obtain the reconstruction quality evaluation result of the super-resolution network.

[0076] The present invention uses image quality indicators and texture parameter indicators to obtain the reconstruction quality evaluation results of the super-resolution network. The specific process is: select at least one group of initial resolution road surface texture data sets that have not been trained by the super-resolution network, and perform normalization and unidirectional multi-level downsampling processing on them to obtain resolution road surface texture data at different zoom levels, and recursively pass the resolution road surface texture data at different zoom levels through the super-resolution network to obtain super-resolution texture data with the same size as the original high-resolution texture at the corresponding zoom level, and use the obtained super-resolution texture data and the initial resolution texture data as input to calculate the peak signal-to-noise ratio, structural similarity and average texture depth relative error, so as to obtain the reconstruction quality evaluation results of the super-resolution network. Figure 5 As shown in Figure 3, the peak signal-to-noise ratio and structural similarity of a certain 1 / 2 level super-resolution texture data and the initial resolution texture data vary with the number of iterations, which can reflect the reconstruction quality of the super-resolution network.

[0077] The present invention uses the super-resolution network after optimizing parameters to evaluate the image quality of super-resolution road surface texture according to peak signal-to-noise ratio and structural similarity, and obtains the reconstruction quality evaluation result of the super-resolution network, such as Figure 6 The present invention reversely converts the normalized two-dimensional super-resolution texture data into three-dimensional super-resolution texture data, and calculates the average texture depth relative error between the three-dimensional super-resolution texture data at different zoom levels and the initial resolution texture data, which is expressed as:

[0078]

[0079] Where: error is the relative error of the average texture depth, MTD hr is the average texture depth value of the initial resolution texture data, MTD sr is the average texture depth value of the super-resolution texture data.

[0080] The present invention calculates the average texture depth value, which is expressed as:

[0081]

[0082] Where: MTD is the average texture depth value, H is the size of the texture data along the driving direction, W is the size of the texture data along the non-driving direction, Z p (x, y) is the maximum value of the texture data, Z(x i ,y i ) is the elevation value of the texture at position (i, j).

[0083] The present invention evaluates the reconstruction quality of super-resolution texture from the perspective of texture parameters. Figure 7As shown, the three-dimensional image and average texture depth relative error index of a certain test data and super-resolution road texture data at different zoom levels.

[0084] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A road texture reconstruction method based on a recursive super-resolution network, characterized in that: The following steps are involved: S1, obtaining initial resolution road surface texture data, and performing normalization processing and unidirectional multi-level downsampling processing on the initial resolution road surface texture data in turn to obtain resolution road surface texture data at different zoom levels; The normalized initial resolution road surface texture data set is subjected to unidirectional multi-level downsampling processing to obtain resolution road surface texture data at different zoom levels. Specifically, according to the normalized initial resolution road surface texture data set, each piece of initial resolution road surface texture data is downsampled according to different zoom factors. The downsampling zoom factors are 1 / 2, 1 / 4, 1 / 8, and 1 / 16, respectively, to obtain 4 sets of resolution road surface texture data at different zoom levels. S2, according to the resolution road surface texture data at different zoom levels in step S1, pairing the same road surface texture with a two-fold resolution difference to obtain a two-fold resolution difference data pair; S3, constructing a super-resolution network, initializing super-resolution network parameters, and training the super-resolution network with the double-resolution difference data in step S2 in ascending order by a cyclic recursive method to obtain a super-resolution network with optimized parameters; The double-resolution difference data pairs in step S2 are sequentially trained in ascending order by a cyclic recursive method for super-resolution network training. The specific process is as follows: first, the 1 / 16 level resolution texture data in the double-resolution difference data pairs is used as input, and the super-resolution texture data outputted by the super-resolution network is 1 / 8 level, and the loss function is calculated with the 1 / 8 level resolution texture data in the double-resolution difference data pairs, and the super-resolution network parameters are updated according to the back-propagated gradient of the optimizer; Then, the 1 / 8 level resolution texture data in the double resolution difference data pair is used as input, and the super-resolution network outputs the 1 / 4 level super-resolution texture data. The loss function is calculated with the 1 / 4 level resolution texture data in the double resolution difference data pair, and the super-resolution network parameters are updated according to the back-propagated gradient of the optimizer; Then, the 1 / 4 level resolution texture data in the double resolution difference data pair is taken as input, and the super-resolution network outputs the 1 / 2 level super-resolution texture data, and the loss function is calculated with the 1 / 2 level resolution texture data in the double resolution difference data pair, and the super-resolution network parameters are updated according to the back-propagation gradient of the optimizer; finally, the 1 / 2 level resolution texture data in the double resolution difference data pair is taken as input, and the super-resolution network outputs the initial super-resolution texture data, and the loss function is calculated with the initial resolution road surface texture data in the double resolution difference data pair, and the super-resolution network parameters are updated according to the back-propagation gradient of the optimizer; S4. Using the super-resolution network after optimizing the parameters in step S3, the road texture data with different zoom levels are reconstructed step by step to the initial resolution texture data, and the reconstruction quality evaluation result of the super-resolution network is obtained using the image quality index and the texture parameter index.

2. A road surface texture reconstruction method based on a recursive super-resolution network according to claim 1, characterized in that: In step S1, the initial resolution road surface texture data is normalized, including the following steps: A1. Standardize the initial resolution road texture data, expressed as; Where: x1 is the normalized initial resolution road surface texture data, x0 is the initial resolution road surface texture data, μ is the average value of the initial resolution road surface texture data, σ is the standard deviation of the initial resolution road surface texture data; A2. Map the range of the initial resolution road texture data normalized in step A1 to (-1,1), expressed as: Where: x2 is the initial resolution road surface texture data with a range mapped to (-1,1); A3, mapping the value range of the initial resolution road surface texture data whose value range is mapped to (-1, 1) in step A2 to (0, 1), and obtaining the normalized initial resolution road surface texture data, which is expressed as: Where: x3 is the initial resolution road texture data after normalization.

3. The road surface texture reconstruction method based on a recursive super-resolution network according to claim 1 is characterized in that: In step S2, the same road surface textures with twice the resolution difference are paired to obtain twice the resolution difference data pairs. The specific process is: for the same road surface texture, 1 / 16 level resolution texture is paired with 1 / 8 level resolution texture, 1 / 8 level resolution texture is paired with 1 / 4 level resolution texture, 1 / 4 level resolution texture is paired with 1 / 2 level resolution texture, 1 / 2 level resolution texture is paired with original high resolution texture, and 4 groups of twice the resolution difference data pairs are obtained.

4. The road surface texture reconstruction method based on a recursive super-resolution network according to claim 1 is characterized in that: In step S3, the constructed super-resolution network includes an input layer, a first convolution layer, a first activation function layer, a residual module unit, a second convolution layer, a third convolution layer, a sub-pixel convolution layer, a fourth convolution layer, a second activation function layer, a fifth convolution layer and a first output layer connected in sequence; the output of the first activation function layer is added to the output of the second convolution layer in the channel direction.

5. The road surface texture reconstruction method based on a recursive super-resolution network according to claim 4 is characterized in that: The number of convolution kernels of the first and second convolution layers are both 64, the convolution kernel size is 3×3, the direction stride is (1,1), and the edge padding size is 1; the number of convolution kernels of the third convolution layer is 256, the convolution kernel size is 3×3, the direction stride is (1,2), and the edge padding size is 1; the number of convolution kernels of the fourth convolution layer is 256, the convolution kernel size is 3×3, the direction stride is (1,1), and the edge padding size is 1; the number of convolution kernels of the fifth convolution layer is 1, the convolution kernel size is 1×1, the direction stride is (1,1), and the edge padding size is 1; the first activation function layer and the second activation function layer both use the ReLU activation function; the upsampling factor used in the sub-pixel convolution layer is 2, which is used to double the width and height of the feature layer; the first output layer directly outputs without any activation function.

6. The road surface texture reconstruction method based on a recursive super-resolution network according to claim 4 is characterized in that: The residual module unit includes 16 residual modules with the same structure; the residual module includes an input layer, a first deep convolutional layer, a first deep activation layer, a second deep convolutional layer, a residual connection and a second output layer; the number of convolution kernels in the first deep convolutional layer and the second deep convolutional layer are both 64, the convolution kernel size is 3×3, the direction step size is (1,1), and the edge padding size is 1; the first deep activation layer uses the ReLU activation function; the output of the second deep convolutional layer is added to the input layer in the channel direction; the second output layer directly outputs without any activation function.

7. The road surface texture reconstruction method based on a recursive super-resolution network according to claim 1, characterized in that: In step S3, a super-resolution network after optimizing parameters is obtained. The specific process is as follows: during the iterative training of the super-resolution network, the loss function value of the corresponding resolution texture data in the super-resolution texture data and the double-resolution difference data is calculated; the road surface texture super-resolution network is trained in two stages, the loss type of the first stage adopts the L1 norm, and the loss type of the second stage adopts the mean square error loss; the optimizer completes the gradient calculation and performs gradient back propagation according to the loss function value; the minimum loss function value that appears during the road surface texture super-resolution training process is recorded, and the corresponding weight parameters are saved to obtain the super-resolution network after optimizing parameters.

8. The road surface texture reconstruction method based on a recursive super-resolution network according to claim 1, characterized in that: In step S4, the peak signal-to-noise ratio and structural similarity are used as image quality evaluation indicators, and the average texture depth relative error is used as the texture parameter indicator.