Method, device, equipment, medium and program product for improving accuracy of road flatness identification

Abstract

The present disclosure provides a method, device, equipment, medium and program product for improving the accuracy of road flatness identification, which comprises: obtaining image data and flatness data of a target road, and determining the correspondence between the image data and the flatness data; performing feature extraction on the flatness data to obtain flatness time sequence features, performing feature extraction on the image data corresponding to the flatness data to obtain image features, performing feature fusion on the flatness time sequence features and the image features to obtain fused features, identifying outliers in the flatness data based on the fused features, and obtaining outliers in the flatness data; and correcting the outliers in the flatness data to obtain corrected flatness data of the target road. The present disclosure automatically detects and corrects outliers in flatness data, thereby improving the accuracy of measured flatness data.

Description

Technical Field

[0001] This disclosure relates to the field of traffic data processing technology, and in particular to a method, apparatus, equipment, medium, and program product for improving the accuracy of road surface smoothness identification. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this disclosure as set forth in the claims. The description herein is not intended to be a prior art simply because it is included in this section.

[0003] In highway construction and maintenance, road surface smoothness is a core indicator for measuring driving comfort, safety, and road quality. Road surface smoothness is typically quantified and assessed using the International Roughness Index (IRI) or data from laser roughness meters.

[0004] In actual testing and subsequent data processing, vibration markings are a common and unique source of interference. Due to their raised or recessed design (such as horizontal strips or dotted protrusions), vibration markings cause the flatness testing equipment (such as laser profilers and inertial reference systems) to vibrate during operation, resulting in abnormal data fluctuations and distorted overall flatness evaluation. Summary of the Invention

[0005] In view of this, the purpose of this disclosure is to provide a method, apparatus, equipment, medium and program product for improving the accuracy of road surface smoothness identification, and to at least to some extent solve one of the technical problems in the related art.

[0006] To achieve the above objectives, the first aspect of this exemplary embodiment provides a method for improving the accuracy of road surface smoothness identification, comprising:

[0007] Acquire image data and smoothness data of the target road surface, and determine the correspondence between the image data and the smoothness data;

[0008] The planarity data is subjected to feature extraction to obtain planarity temporal features. The planarity data corresponding to the planarity data is subjected to feature extraction to obtain image features. The planarity temporal features and the image features are fused to obtain fused features. Based on the fused features, the planarity data is subjected to outlier identification to obtain outliers in the planarity data.

[0009] The outliers in the smoothness data are corrected to obtain the corrected smoothness data of the target road surface.

[0010] In some exemplary embodiments, acquiring image data and smoothness data of the target road surface, and determining the correspondence between the image data and the smoothness data, includes:

[0011] For each preset collection distance traveled on the target road surface, a camera device would capture an image of the front, and a smoothness data would be collected using a laser. The collection distance is the distance between the side of the camera device closest to the laser and the laser itself.

[0012] Based on the side of the camera device closest to the laser within its acquisition range and the acquisition distance, the image data is determined from the front image;

[0013] The current flatness data is compared with the previously described image data.

[0014] In some exemplary embodiments, the steps of extracting features from the flatness data to obtain flatness temporal features, extracting features from the image data corresponding to the flatness data to obtain image features, fusing the flatness temporal features and the image features to obtain fused features, and identifying outliers in the flatness data based on the fused features to obtain outliers in the flatness data include:

[0015] The target smoothness data at the target time and the smoothness data at a preset number of times before and after the target time are used to form a smoothness data sequence. Features are extracted from the smoothness data sequence to obtain smoothness time-series features.

[0016] Determine the target image data corresponding to the target flatness data, and perform feature extraction on the target image data to obtain image features;

[0017] Based on the cross-attention mechanism, the temporal features of flatness and the image features are fused to obtain fused features;

[0018] The target flatness data is classified based on the fusion features to determine whether the target flatness data is an outlier in the flatness data.

[0019] In some exemplary embodiments, the method further includes:

[0020] By combining the fused features to decode the image features, a vibration mark recognition result is obtained, which includes the vibration mark type, vibration mark location, and vibration mark size.

[0021] The loss function used to decode the image features by combining the fused features includes a regression loss function and a Focalloss loss function.

[0022] The regression loss function is configured to determine the position and size of the vibration mark;

[0023] The Focalloss loss function is configured to determine the vibration mark type.

[0024] In some exemplary embodiments, correcting outliers in the smoothness data to obtain corrected smoothness data for the target road surface includes at least one of the following:

[0025] Replace the outlier value with zero;

[0026] or,

[0027] Replace the outliers with the local average value of the flatness data.

[0028] In some exemplary embodiments, the determination of the local average value includes:

[0029] Centered on the outlier value, a subset of flatness data is obtained from the flatness data within a preset range, and the average value of the flatness data subset is calculated to obtain the local average value.

[0030] Based on the same inventive concept, a second aspect of the exemplary embodiments of this disclosure provides an apparatus for improving the accuracy of road surface smoothness identification, comprising:

[0031] The image data and smoothness data association module is configured to acquire image data and smoothness data of the target road surface, and determine the correspondence between the image data and the smoothness data;

[0032] The image data and flatness data recognition module is configured to extract features from the flatness data to obtain flatness time-series features, extract features from the image data corresponding to the flatness data to obtain image features, fuse the flatness time-series features and the image features to obtain fused features, and identify outliers in the flatness data based on the fused features to obtain outliers in the flatness data.

[0033] The smoothness data correction module is configured to correct outliers in the smoothness data to obtain corrected smoothness data of the target road surface.

[0034] Based on the same inventive concept, a third aspect of the exemplary embodiments of this disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method as described in the first aspect.

[0035] Based on the same inventive concept, a fourth aspect of the exemplary embodiments of this disclosure provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method as described in the first aspect.

[0036] Based on the same inventive concept, a fifth aspect of the exemplary embodiments of this disclosure provides a computer program product including computer program instructions that, when run on a computer, cause the computer to perform the method as described in the first aspect.

[0037] As can be seen from the above description, the method, apparatus, device, medium, and program products for improving the accuracy of road surface smoothness identification provided in this disclosure include: acquiring image data and smoothness data of a target road surface, and determining the correspondence between the image data and the smoothness data; extracting features from the smoothness data to obtain smoothness temporal features; extracting features from the image data corresponding to the smoothness data to obtain image features; fusing the smoothness temporal features and the image features to obtain fused features; identifying outliers in the smoothness data based on the fused features to obtain outliers in the smoothness data; and correcting the outliers in the smoothness data to obtain corrected smoothness data of the target road surface. This disclosure automatically detects and corrects outliers in the smoothness data, thereby improving the accuracy of the measured smoothness data. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this disclosure or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1a A schematic diagram illustrating the effect of vibration markings on flatness data, provided as an exemplary embodiment of this disclosure.

[0040] Figure 1b A schematic diagram of a vibration marking provided for an exemplary embodiment of this disclosure;

[0041] Figure 2 A flowchart illustrating a method for improving the accuracy of road surface smoothness identification, provided as an exemplary embodiment of this disclosure;

[0042] Figure 3 A schematic diagram illustrating the acquisition of image data and flatness data as provided in an exemplary embodiment of this disclosure;

[0043] Figure 4 A schematic diagram illustrating image data and flatness data recognition provided for an exemplary embodiment of this disclosure;

[0044] Figure 5 A schematic diagram of a network structure for a timing signal encoder provided as an exemplary embodiment of this disclosure;

[0045] Figure 6 A schematic diagram of a network structure for an image encoder provided as an exemplary embodiment of this disclosure;

[0046] Figure 7 A schematic diagram of a network structure for a cross-attention module provided as an exemplary embodiment of this disclosure;

[0047] Figure 8 A schematic diagram of a network structure for an image decoder provided as an exemplary embodiment of this disclosure;

[0048] Figure 9 A schematic diagram of a device for improving the accuracy of road surface smoothness recognition, provided as an exemplary embodiment of this disclosure;

[0049] Figure 10 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present disclosure. Detailed Implementation

[0050] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0051] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this application's technical solution, based on the prompt message.

[0052] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0053] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0054] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0055] To make the objectives, technical solutions, and advantages of this disclosure clearer, the principles and spirit of this disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement this disclosure, and are not intended to limit the scope of this disclosure in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0056] In this article, it is important to understand that any number of elements in the accompanying figures is for illustrative purposes and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0057] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar words used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly. The article "a" or "an" preceding an element does not exclude the existence of multiple such elements.

[0058] The principles and spirit of this disclosure will be explained in detail below with reference to several representative embodiments.

[0059] As described in the background section, road surface smoothness is a core indicator for measuring driving comfort, safety, and road quality during highway construction and maintenance. Road surface smoothness is typically quantified and assessed using the International Roughness Index (IRI) or data from laser roughness meters.

[0060] The inventors of this disclosure have discovered that vibration markings (also known as oscillating markings, oscillating lines, etc.) are a common and unique interference factor encountered during actual testing and subsequent data processing. Due to their raised or recessed design (such as horizontal stripes or dotted protrusions), vibration markings cause flatness testing equipment (such as laser profilers or inertial reference systems) to oscillate during operation, leading to abnormal data fluctuations.

[0061] For example, transverse bar markings may cause periodic fluctuations every 0.5m, which is significantly different from the data characteristics of actual road surface irregularities (such as ruts, undulations, etc.). This can lead to a sudden increase in the corresponding IRI value in the vibrating marking area, for example, a sudden increase to 3.5m / km (IRI ≤ 2.0m / km for well-smooth road sections), resulting in a distortion of the overall smoothness assessment. Figure 1a As shown, vibration markings exist in T1, T3, and T4, therefore their flatness values ​​are all greater than 3.5, while vibration markings do not exist in T2 and T5, therefore their flatness values ​​are all less than 3.5.

[0062] However, vibration markings are functional structures, as referenced Figure 1b Vibration markings include group, single, and point types, and the flatness of their areas should be handled separately; otherwise, the acceptance data will not meet the standards.

[0063] To address this issue, related technologies involve significant manual intervention to remove these abnormal data. Clearly, this manual method is time-consuming, labor-intensive, and prone to subjective errors.

[0064] To address the aforementioned problems, this disclosure provides a solution to improve the accuracy of road surface smoothness identification, specifically including:

[0065] The process involves acquiring image data and smoothness data of the target road surface, and determining the correspondence between the image data and the smoothness data; extracting features from the smoothness data to obtain temporal smoothness features, extracting features from the corresponding image data to obtain image features, fusing the temporal smoothness features and the image features to obtain fused features, identifying outliers in the smoothness data based on the fused features, and correcting the outliers in the smoothness data to obtain corrected smoothness data of the target road surface.

[0066] This disclosure automatically detects and corrects outliers in flatness data, thereby improving the accuracy of the measured flatness data.

[0067] After introducing the basic principles of this disclosure, various non-limiting embodiments of this disclosure will be described in detail below.

[0068] refer to Figure 2 This is a flowchart illustrating a method for improving the accuracy of road surface smoothness identification provided by an exemplary embodiment of this disclosure.

[0069] Methods to improve the accuracy of road surface smoothness identification include the following steps:

[0070] Step S210: Obtain image data and smoothness data of the target road surface, and determine the correspondence between the image data and the smoothness data.

[0071] In some exemplary embodiments, acquiring image data and smoothness data of the target road surface, and determining the correspondence between the image data and the smoothness data, includes:

[0072] For each preset collection distance traveled on the target road surface, a camera device would capture an image of the front, and a smoothness data would be collected using a laser. The collection distance is the distance between the side of the camera device closest to the laser and the laser itself.

[0073] Based on the side of the camera device closest to the laser within its acquisition range and the acquisition distance, the image data is determined from the front image;

[0074] The current flatness data is compared with the previously described image data.

[0075] As an example, see reference Figure 3 Considering that the image captured by the vehicle's front-facing camera has a spatial deviation from the flatness detection laser, the vibration markings appearing in the field of view of the captured image will have a spatial offset from the flatness data collected at the current moment. Therefore, the information in the image cannot be used to verify whether the information in the flatness data at the corresponding moment is an abnormal output.

[0076] Therefore, the spatial distance between the acquisition sensors (camera and laser) is strictly calibrated. The distance between the lower edge of the image captured by the camera (i.e., point B) and the smoothness laser (i.e., point A) is specifically set to 10 meters. Then, a distance encoder is used to control the acquisition frequency of both. The vehicle takes one image of the road ahead every 10 meters, collecting one IRI data point. At the same time, a 10-meter range (i.e., between points B and C) is marked in the image. That is, the road surface within 10 meters of the acquisition equipment in the current image does not overlap with the image at the next moment, corresponding to the data collected by the smoothness laser at the next moment. This is equivalent to shifting the synchronously acquired IRI data sequence one unit to the left to achieve alignment between the two, meaning that the road surface within 10 meters of the image at the current moment corresponds to the IRI data at the current moment.

[0077] Step S220: Extract features from the flatness data to obtain flatness time-series features; extract features from the image data corresponding to the flatness data to obtain image features; fuse the flatness time-series features and the image features to obtain fused features; identify outliers in the flatness data based on the fused features to obtain outliers in the flatness data.

[0078] In some exemplary embodiments, the steps of extracting features from the flatness data to obtain flatness temporal features, extracting features from the image data corresponding to the flatness data to obtain image features, fusing the flatness temporal features and the image features to obtain fused features, and identifying outliers in the flatness data based on the fused features to obtain outliers in the flatness data include:

[0079] The target smoothness data at the target time and the smoothness data at a preset number of times before and after the target time are used to form a smoothness data sequence. Features are extracted from the smoothness data sequence to obtain smoothness time-series features.

[0080] Determine the target image data corresponding to the target flatness data, and perform feature extraction on the target image data to obtain image features;

[0081] Based on the cross-attention mechanism, the temporal features of flatness and the image features are fused to obtain fused features;

[0082] The target flatness data is classified based on the fusion features to determine whether the target flatness data is an outlier in the flatness data.

[0083] In some exemplary embodiments, the method further includes:

[0084] By combining the fused features to decode the image features, a vibration mark recognition result is obtained, which includes the vibration mark type, vibration mark location, and vibration mark size.

[0085] The loss function used to decode the image features by combining the fused features includes a regression loss function and a Focalloss loss function.

[0086] The regression loss function is configured to determine the position and size of the vibration mark;

[0087] The Focalloss loss function is configured to determine the vibration mark type.

[0088] As an example, see reference Figure 4This paper demonstrates a model network structure for image data and flatness data recognition, which includes the following modules: a temporal signal encoder, an image encoder, a cross-attention module, and an image decoder.

[0089] The input to the timing signal encoder is: the IRI sequence T. i -N~T i +N, the output of the timing signal encoder is: flatness timing characteristics.

[0090] The input to the image encoder is: time T i The acquired image from the front, after passing through a fully connected layer, becomes the image features.

[0091] The input to the cross-attention module is: temporal features of flatness and image features; the output of the cross-attention module is: fused features, which are passed through a classification layer (fully connected network) and output normal / outlier values. The loss function is cross-entropy.

[0092] The input to the image decoder is the image features and fused features before passing through the fully connected layer; the output of the image decoder is the vibration mark type, where the loss function is FocalLoss + regression loss.

[0093] In practical implementation, considering the inherent correlation between the apparent characteristics of vibration markings and the corresponding smoothness data, and that it's not simply about increasing the absolute value of smoothness, the smoothness data collected at the current moment may be affected by road surface information from previous and subsequent moments, such as the overall road surface smoothness baseline data. Therefore, this disclosure constructs a sequence of smoothness data from N moments before and after the current moment as the first input to the model. A temporal encoder extracts the embedded features of the smoothness data to obtain the temporal smoothness features. Next, the image of the road ahead captured at the current moment is input to the model. After an encoding and decoding process, the position, size, and type of the corresponding vibration markings on the road surface are detected. During this process, the temporal smoothness features and image features are fused using multimodal methods to obtain a fused feature. This fused feature is used to determine whether the current smoothness data is an outlier and to guide the image decoding process, improving the model's performance in detecting vibration markings from images.

[0094] In practice, the loss function for training the model is designed as follows:

[0095] Anomaly detection of IRI values ​​uses a binary classification method with cross-entropy loss (0 / 1). The closer the output value is to 1, the higher the probability that the current IRI value is an anomaly; the closer the output value is to 0, the lower the probability that the current IRI value is an anomaly. A threshold of 0.5 is generally used as the standard for determining whether a value is an anomaly.

[0096] The loss function for detecting vibration marks in images is divided into two types: regression loss, which is used to determine the coordinates of the center point of the bounding box surrounding the vibration mark and the size of the box, i.e., height and width; and Focal loss, which is used to determine the type of vibration mark. Its main advantage is to solve the problem of data imbalance, because compared with normal road surfaces, vibration marks appear relatively less frequently in the data, which can easily cause long-tail problems in the data.

[0097] refer to Figure 5 This demonstrates the network structure design of a timing signal encoder.

[0098] This architecture combines convolutional neural networks (CNNs) and self-attention mechanisms for feature extraction and processing. It is particularly effective when processing sequential data (such as text, time series, etc.) because it captures both local features (through convolution) and global dependencies (through self-attention).

[0099] Normalization: The input data is first normalized using the following formula: Here, x is the input data, μ is the mean, and σ is the standard deviation. Normalization helps accelerate the model training process.

[0100] Dimensionality increase of fully connected layers: 1→D1: The dimension of the input data is increased from 1 to D1 by using fully connected layers, increasing the dimension of features to capture more complex features.

[0101] One-dimensional convolution to extract features F: D1→D1: Use a one-dimensional convolutional layer to extract features. The convolution operation can capture local features while keeping the feature dimension unchanged (D1).

[0102] Multi-head self-attention: The multi-head self-attention mechanism allows the model to focus on the relationships between different parts of the input sequence, enhancing the model's ability to understand sequence data. Specifically, it includes:

[0103] The input sequence features F are used as the input to the model.

[0104] To enable the model to understand the positional relationships of elements in the sequence, positional encoding is added.

[0105] The self-attention mechanism repeats N times. In each iteration, attention weights are calculated through three linear projections (Key, Query, Value). The input features are linearly projected to generate Key, Query, and Value vectors.

[0106] Performing layer normalization before the feedforward neural network helps accelerate training and improve model stability.

[0107] Features are further processed using a feedforward neural network.

[0108] Applying layer normalization after the feedforward neural network helps accelerate training and improve model stability.

[0109] Global average pooling: Features are subjected to global average pooling to reduce feature dimensionality while retaining the most important feature information.

[0110] Output feature vector: 1×D1: The final output is a 1×D1 feature vector, which is used for subsequent classification or other tasks.

[0111] refer to Figure 6 This demonstrates the network structure design of an image encoder.

[0112] The image of the road surface is input;

[0113] Embedding layer with overlapping image patches: The input image is segmented into multiple overlapping image patches, which are then embedded into the feature space.

[0114] The efficient visual Transformer modules, L1, L2, and L3, are used to extract features at different levels. Each module may contain multiple self-attention layers and feedforward neural network layers for in-depth image feature extraction.

[0115] Among them, the high-efficiency visual Transformer module includes:

[0116] The input features can be the embeddings of the original image patches or features that have undergone preliminary processing.

[0117] Feature group 1, feature group 2 and feature group 3 represent features at different scales or levels, and will be fed into the self-attention mechanism for processing.

[0118] Self-attention mechanisms capture relationships between features by computing the query (Q), key (K), and value (V) of the input features. This process is repeated for each feature group.

[0119] The features processed by self-attention are concatenated together to form a new feature representation.

[0120] The concatenated features are passed through a linear projection layer, which may be used to adjust feature dimensions or perform feature transformation.

[0121] The features after linear projection are further processed by a feedforward neural network, which may include non-linear activation functions and weight updates.

[0122] The entire process is repeated M times, for example, feature extraction is performed at different scales or different levels.

[0123] The exemplary embodiments described above illustrate a multi-scale, multi-stage feature extraction and processing system that uses a visual Transformer module to extract and fuse features at different levels. This structure helps the model capture richer details and contextual information when processing images, thereby improving model performance.

[0124] refer to Figure 7 This demonstrates the network architecture design of the cross-attention module.

[0125] This method combines temporal features of flatness with image features and processes them through a self-attention mechanism.

[0126] Flatness temporal feature processing:

[0127] The input is a feature representation of time-series data, namely the time-series features of flatness.

[0128] The temporal features of flatness are processed through three different linear projection layers to generate Key(K), Query(Q), and Value(V) vectors, respectively. These vectors serve as inputs to the self-attention mechanism, used to compute attention weights and generate the output.

[0129] Perform a Mul (dot product) operation on the Query and Key vectors to calculate the attention score.

[0130] The attention score can be summed for normalization or other purposes.

[0131] Image feature processing:

[0132] The input is the feature representation of the image data, i.e., image features.

[0133] Image features are also processed through three different linear projection layers to generate Key(K), Query(Q), and Value(V) vectors for the self-attention mechanism of image features.

[0134] Perform a Mul (dot product) operation on the Query and Key vectors to calculate the attention score.

[0135] Perform a Sum operation on the attention score.

[0136] The temporal features of flatness and image features are fused together using methods such as addition and stitching to generate the final fused features.

[0137] refer to Figure 8 This demonstrates the network architecture design of the image decoder.

[0138] The input data is first processed by an efficient visual Transformer module to extract the first layer of features (L1 features). The Transformer is a model based on a self-attention mechanism that can capture long-range dependencies in the data.

[0139] The L1 features are then fed into a second, efficient visual Transformer module to extract the second layer of features (L2 features). This layer typically captures more abstract features.

[0140] The L2 features are further fed into a third, efficient visual Transformer module to extract the third layer of features (L3 features). These features are more abstract and may contain higher-level semantic information.

[0141] L3 features are upsampled to increase their resolution. Upsampling is typically used to enlarge the size of the feature map so that it can be fused with lower-level features.

[0142] The upsampled L3 features are fused with the original L2 features to generate the decoded L2 features. This step helps to preserve the detailed information of the low-level features.

[0143] The decoded L2 features are then upsampled again to further increase their resolution.

[0144] The upsampled L2 features are fused with the original L1 features to generate the decoded L1 features. This step further integrates feature information from different levels.

[0145] This approach extracts features through a bottom-up path and fuses them through a top-down path. This structure helps the model capture features at different scales, thereby improving its ability to recognize targets of different sizes.

[0146] Step S230: Correct the outliers in the smoothness data to obtain the corrected smoothness data of the target road surface.

[0147] In some exemplary embodiments, correcting outliers in the smoothness data to obtain corrected smoothness data for the target road surface includes at least one of the following:

[0148] Replace the outlier value with zero;

[0149] or,

[0150] Replace the outliers with the local average value of the flatness data.

[0151] In some exemplary embodiments, the determination of the local average value includes:

[0152] Centered on the outlier value, a subset of flatness data is obtained from the flatness data within a preset range, and the average value of the flatness data subset is calculated to obtain the local average value.

[0153] In practice, once the current IRI data is determined to be an outlier, noise reduction can be achieved by directly setting it to zero or by taking a local average value. The local average value is calculated by taking a certain range of flatness data forward and backward from the current IRI value and averaging them as the replacement value for the current IRI data.

[0154] As can be seen from the above, the method for improving the accuracy of road surface smoothness recognition provided in this disclosure includes: acquiring image data and smoothness data of a target road surface, and determining the correspondence between the image data and the smoothness data; extracting features from the smoothness data to obtain smoothness temporal features; extracting features from the image data corresponding to the smoothness data to obtain image features; fusing the smoothness temporal features and the image features to obtain fused features; identifying outliers in the smoothness data based on the fused features to obtain outliers in the smoothness data; and correcting the outliers in the smoothness data to obtain corrected smoothness data of the target road surface.

[0155] This disclosure automates the detection and correction of outliers in flatness data, improving the accuracy of the measured flatness data. Furthermore, compared to related technologies that rely on extensive manual intervention to remove these outliers, this method is time-saving, labor-saving, and eliminates the risk of subjective misjudgment.

[0156] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0157] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0158] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides a device for improving the accuracy of road surface smoothness identification.

[0159] refer to Figure 9 This is a schematic diagram of a device for improving the accuracy of road surface smoothness identification provided by an exemplary embodiment of the present disclosure.

[0160] A device for improving the accuracy of road surface smoothness recognition includes the following modules:

[0161] The image data and smoothness data association module 910 is configured to acquire image data and smoothness data of the target road surface, and determine the correspondence between the image data and the smoothness data;

[0162] The image data and flatness data recognition module 920 is configured to extract features from the flatness data to obtain flatness time-series features, extract features from the image data corresponding to the flatness data to obtain image features, fuse the flatness time-series features and the image features to obtain fused features, and identify outliers in the flatness data based on the fused features to obtain outliers in the flatness data.

[0163] The smoothness data correction module 930 is configured to correct outliers in the smoothness data to obtain corrected smoothness data of the target road surface.

[0164] In some exemplary embodiments, the image data and flatness data association module 910 is specifically configured as follows:

[0165] For each preset collection distance traveled on the target road surface, a camera device would capture an image of the front, and a smoothness data would be collected using a laser. The collection distance is the distance between the side of the camera device closest to the laser and the laser itself.

[0166] Based on the side of the camera device closest to the laser within its acquisition range and the acquisition distance, the image data is determined from the front image;

[0167] The current flatness data is compared with the previously described image data.

[0168] In some exemplary embodiments, the image data and flatness data recognition module 920 is specifically configured as follows:

[0169] The target smoothness data at the target time and the smoothness data at a preset number of times before and after the target time are used to form a smoothness data sequence. Features are extracted from the smoothness data sequence to obtain smoothness time-series features.

[0170] Determine the target image data corresponding to the target flatness data, and perform feature extraction on the target image data to obtain image features;

[0171] Based on the cross-attention mechanism, the temporal features of flatness and the image features are fused to obtain fused features;

[0172] The target flatness data is classified based on the fusion features to determine whether the target flatness data is an outlier in the flatness data.

[0173] In some exemplary embodiments, the image data and flatness data recognition module 920 is specifically configured as follows:

[0174] By combining the fused features to decode the image features, a vibration mark recognition result is obtained, which includes the vibration mark type, vibration mark location, and vibration mark size.

[0175] The loss function used to decode the image features by combining the fused features includes a regression loss function and a Focalloss loss function.

[0176] The regression loss function is configured to determine the position and size of the vibration mark;

[0177] The Focalloss loss function is configured to determine the vibration mark type.

[0178] In some exemplary embodiments, the flatness data correction module 930 is specifically configured as follows:

[0179] Replace the outlier value with zero;

[0180] or,

[0181] Replace the outliers with the local average value of the flatness data.

[0182] In some exemplary embodiments, the flatness data correction module 930 is specifically configured as follows:

[0183] Centered on the outlier value, a subset of flatness data is obtained from the flatness data within a preset range, and the average value of the flatness data subset is calculated to obtain the local average value.

[0184] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0185] The apparatus described above is used to implement the corresponding method for improving the accuracy of road surface smoothness identification in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0186] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for improving the accuracy of road surface smoothness identification as described in any of the above embodiments.

[0187] Figure 10 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0188] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0189] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0190] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0191] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0192] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0193] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0194] The electronic devices described in the above embodiments are used to implement the corresponding methods for improving the accuracy of road surface smoothness identification in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0195] The memory 1020 stores machine-readable instructions executable by the processor 1010. When the electronic device is running, the processor 1010 communicates with the memory 1020 via the bus 1030, causing the processor 1010 to execute the following instructions during operation:

[0196] Acquire image data and smoothness data of the target road surface, and determine the correspondence between the image data and the smoothness data;

[0197] The planarity data is subjected to feature extraction to obtain planarity temporal features. The planarity data corresponding to the planarity data is subjected to feature extraction to obtain image features. The planarity temporal features and the image features are fused to obtain fused features. Based on the fused features, the planarity data is subjected to outlier identification to obtain outliers in the planarity data.

[0198] The outliers in the smoothness data are corrected to obtain the corrected smoothness data of the target road surface.

[0199] In one possible implementation, the instructions executed by the processor 1010, namely, acquiring image data and smoothness data of the target road surface, and determining the correspondence between the image data and the smoothness data, include:

[0200] For each preset collection distance traveled on the target road surface, a camera device would capture an image of the front, and a smoothness data would be collected using a laser. The collection distance is the distance between the side of the camera device closest to the laser and the laser itself.

[0201] Based on the side of the camera device closest to the laser within its acquisition range and the acquisition distance, the image data is determined from the front image;

[0202] The current flatness data is compared with the previously described image data.

[0203] In one possible implementation, the instructions executed by the processor 1010 include: extracting features from the flatness data to obtain flatness temporal features; extracting features from the image data corresponding to the flatness data to obtain image features; fusing the flatness temporal features and the image features to obtain fused features; and identifying outliers in the flatness data based on the fused features to obtain outliers in the flatness data, including:

[0204] The target smoothness data at the target time and the smoothness data at a preset number of times before and after the target time are used to form a smoothness data sequence. Features are extracted from the smoothness data sequence to obtain smoothness time-series features.

[0205] Determine the target image data corresponding to the target flatness data, and perform feature extraction on the target image data to obtain image features;

[0206] Based on the cross-attention mechanism, the temporal features of flatness and the image features are fused to obtain fused features;

[0207] The target flatness data is classified based on the fusion features to determine whether the target flatness data is an outlier in the flatness data.

[0208] In one possible implementation, the instructions executed by the processor 1010 further include:

[0209] By combining the fused features to decode the image features, a vibration mark recognition result is obtained, which includes the vibration mark type, vibration mark location, and vibration mark size.

[0210] The loss function used to decode the image features by combining the fused features includes a regression loss function and a Focalloss loss function.

[0211] The regression loss function is configured to determine the position and size of the vibration mark;

[0212] The Focalloss loss function is configured to determine the vibration mark type.

[0213] In one possible implementation, the instructions executed by the processor 1010 to correct outliers in the smoothness data to obtain corrected smoothness data for the target road surface include at least one of the following:

[0214] Replace the outlier value with zero;

[0215] or,

[0216] Replace the outliers with the local average value of the flatness data.

[0217] In one possible implementation, the method for determining the local average value in the instructions executed by processor 1010 includes:

[0218] Centered on the outlier value, a subset of flatness data is obtained from the flatness data within a preset range, and the average value of the flatness data subset is calculated to obtain the local average value.

[0219] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the method for improving the accuracy of road surface smoothness identification as described in any of the above embodiments.

[0220] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0221] The aforementioned non-transitory computer-readable storage media can be any available medium or data storage device that a computer can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0222] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the method for improving the accuracy of road surface smoothness identification as described in any of the embodiments in the exemplary method section above, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0223] Based on the same inventive concept, corresponding to the method for improving road surface smoothness recognition accuracy described in any of the above embodiments, this disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processor to perform the method for improving road surface smoothness recognition accuracy. Corresponding to the execution entity for each step in each embodiment of the method for improving road surface smoothness recognition accuracy, the processor executing the corresponding step can belong to the corresponding execution entity.

[0224] The computer program product of the above embodiments is used to cause the computer and / or the processor to perform the method for improving the accuracy of road surface smoothness identification as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0225] Those skilled in the art will recognize that embodiments of this disclosure can be implemented as a system, method, or computer program product. Therefore, this disclosure can be implemented as entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this disclosure can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0226] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (not exhaustive) of a computer-readable storage medium may include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0227] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0228] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0229] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0230] It should be understood that each block of a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine that, when executed by a computer or other programmable data processing device, creates means for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0231] These computer program instructions may also be stored in a computer-readable medium that enables a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce a product comprising an instruction apparatus that implements the functions / operations specified in the boxes of a flowchart and / or block diagram.

[0232] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable apparatus can provide a process for implementing the functions / operations specified in the boxes of a flowchart and / or block diagram.

[0233] Furthermore, although the operations of the methods of this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowcharts may be executed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0234] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0235] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0236] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0237] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0238] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0239] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

[0240] While the spirit and principles of this disclosure have been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for convenience of expression. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the appended claims is to be interpreted in the broadest sense, thereby encompassing all such modifications and equivalent structures and functions.

Claims

1. A method for improving the accuracy of road surface smoothness identification, characterized in that, include: Acquire image data and smoothness data of the target road surface, and determine the correspondence between the image data and the smoothness data; The planarity data is subjected to feature extraction to obtain planarity temporal features. The planarity data corresponding to the planarity data is subjected to feature extraction to obtain image features. The planarity temporal features and the image features are fused to obtain fused features. Based on the fused features, the planarity data is subjected to outlier identification to obtain outliers in the planarity data. The outliers in the smoothness data are corrected to obtain the corrected smoothness data of the target road surface; The method further includes: By combining the fused features to decode the image features, a vibration mark recognition result is obtained, which includes the vibration mark type, vibration mark location, and vibration mark size. The loss function used to decode the image features by combining the fused features includes a regression loss function and a Focal loss function. The regression loss function is configured to determine the position and size of the vibration mark; The Focal loss function is configured to determine the vibration mark type.

2. The method according to claim 1, characterized in that, The step of acquiring image data and smoothness data of the target road surface, and determining the correspondence between the image data and the smoothness data, includes: For each preset collection distance traveled on the target road surface, a frontal image was captured by a camera device, and a flatness data was collected by a laser. The collection distance is the distance between the side of the camera device closest to the laser and the laser device within the collection range of the camera device. Based on the side of the camera device closest to the laser within its acquisition range and the acquisition distance, the image data is determined from the front image; The current flatness data is compared with the previously described image data.

3. The method according to claim 1, characterized in that, The process involves extracting features from the smoothness data to obtain smoothness time-series features, extracting features from the corresponding image data to obtain image features, fusing the smoothness time-series features and the image features to obtain fused features, and identifying outliers in the smoothness data based on the fused features. This includes: The target smoothness data at the target time and the smoothness data at a preset number of times before and after the target time are used to form a smoothness data sequence. Features are extracted from the smoothness data sequence to obtain smoothness time-series features. Determine the target image data corresponding to the target flatness data, and perform feature extraction on the target image data to obtain image features; Based on the cross-attention mechanism, the temporal features of flatness and the image features are fused to obtain fused features; The target flatness data is classified based on the fusion features to determine whether the target flatness data is an outlier in the flatness data.

4. The method according to claim 1, characterized in that, The step of correcting outliers in the smoothness data to obtain corrected smoothness data for the target road surface includes at least one of the following: Replace the outlier value with zero; or, Replace the outliers with the local average value of the flatness data.

5. The method according to claim 4, characterized in that, The method for determining the local average value includes: Centered on the outlier value, a subset of flatness data is obtained from the flatness data within a preset range, and the average value of the flatness data subset is calculated to obtain the local average value.

6. A device for improving the accuracy of road surface smoothness identification, characterized in that, include: The image data and smoothness data association module is configured to acquire image data and smoothness data of the target road surface, and determine the correspondence between the image data and the smoothness data; The image data and flatness data recognition module is configured to extract features from the flatness data to obtain flatness time-series features, extract features from the image data corresponding to the flatness data to obtain image features, fuse the flatness time-series features and the image features to obtain fused features, and identify outliers in the flatness data based on the fused features to obtain outliers in the flatness data. The smoothness data correction module is configured to correct outliers in the smoothness data to obtain corrected smoothness data of the target road surface. The device is also configured to: By combining the fused features to decode the image features, a vibration mark recognition result is obtained, which includes the vibration mark type, vibration mark location, and vibration mark size. The loss function used to decode the image features by combining the fused features includes a regression loss function and a Focal loss function. The regression loss function is configured to determine the position and size of the vibration mark; The Focal loss function is configured to determine the vibration mark type.

7. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing the computer to perform the method of any one of claims 1 to 5.

9. A computer program product, characterized in that, It includes computer program instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 5.

Images