Infrared multi-wavelength pavement state classification detection method based on multi-point fusion

Abstract

An infrared multi-wavelength pavement state classification detection method based on multi-point fusion comprises the following steps: step 1, emitting laser with different wavelengths to different points of a pavement by using an infrared laser module, and collecting radiation intensity data of each point; 2, preprocessing the original radiation intensity data; step 3, carrying out feature extraction on the preprocessed data, wherein the feature extraction specifically comprises extraction of difference value features and specific value features; 4, building an initial model architecture training model until a training target is reached, and obtaining a road surface state detection model; and step 5, for a to-be-detected road surface, inputting the road surface state detection model obtained by training in the step 4, and outputting a current road surface state. According to the invention, the infrared multi-wavelength technology is adopted to collect data, and training and pre-judgment are carried out for rough classification of pavement states, so that the overall judgment speed is improved; and compared with a single wavelength, richer pavement radiation information can be obtained, and the distinguishing capability and classification accuracy of similar pavement states are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of electric power, and particularly relates to an infrared multi-wavelength pavement state classification detection method based on multi-point fusion. BACKGROUND

[0002] The pavement state is a key factor affecting the traffic travel safety, and accurate and real-time detection of the dry, water accumulation, wet, icy and snow accumulation states of the pavement is of great significance for the traffic management department to take timely response measures and ensure the driving safety.

[0003] At present, the existing pavement state detection methods mainly include the following categories: 1. A detection method based on visual images: the pavement images are acquired through a camera, and the image recognition technology is used for state classification. However, the method is greatly affected by the light conditions, and the detection accuracy significantly decreases in the night, rain, fog and other adverse weather conditions.

[0004] 2. A contact detection method based on sensors: the sensors are buried in the pavement or contact the pavement surface to directly measure the temperature, humidity and other parameters of the pavement. The method needs to damage the pavement structure, has high installation and maintenance costs, and is easily affected by the pavement wear and has a short service life.

[0005] 3. A detection method based on a single infrared wavelength: the state is judged by using the radiation characteristics of the pavement at a specific infrared wavelength. However, due to the limited single wavelength information, it is difficult to effectively distinguish similar pavement states (such as water accumulation and wet states), and the classification accuracy is low.

[0006] 4. An existing non-contact single-point detection method: only a single point of the pavement is detected, and when the point is blocked by fallen leaves, the pavement is damaged, or there is local pollution, the detection result will be distorted, the false detection rate is high, and the overall real state of the pavement cannot be reflected.

[0007] Therefore, there is an urgent need for a non-contact detection method which is less affected by the environment, has strong anti-interference ability, and can accurately distinguish the five pavement states. SUMMARY

[0008] The application aims to solve the problems of the existing pavement state detection methods, such as being easily affected by the environment, low detection accuracy, high installation and maintenance costs, and false detection caused by local interference in single-point detection, and provides an infrared multi-wavelength pavement state classification detection method based on multi-point fusion, which realizes accurate and reliable detection of the dry, water accumulation, wet, icy and snow accumulation states of the pavement.

[0009] The infrared multi-wavelength pavement state classification detection method based on multi-point fusion comprises the following steps: Step 1. Use an infrared laser module to emit laser beams of different wavelengths to different points on the road surface, and collect radiation intensity data of each point; Step 2. Preprocess the original radiation intensity data; Step 3. Feature extraction on the preprocessed data, including extracting difference features and ratio features, where the difference features are the difference between radiation intensity data of the same point at different wavelengths, and the ratio features are the ratio of different difference features; Step 4. Build an initial model architecture, use the difference features and ratio features obtained in step 3, and the road type and road state as training set data to train the model until the training target is reached, and obtain a road state detection model; Step 5. For the road to be detected, collect data according to steps 1 to 3 to obtain a detection data set, input the road state detection model trained in step 4, and determine the road state.

[0010] Preferably, the step 2 specifically includes: Step 2. Data preprocessing: Step 2-1. Invalid data elimination: according to the radiation calibration range of the infrared laser device, eliminate abnormal data that exceeds the effective range of each point; combined with prior knowledge of road state, eliminate obviously contradictory data; Step 2-2. Data standardization: The three-band radiation intensity data after preprocessing is normalized, and the data of each band is mapped to the [0, 1] interval, and the formula is as follows: The normalized value XAV=(X-X min ) / (X max -X min ) Where X is a single data, X min and X max are the minimum and maximum values of all preprocessed data of the band respectively.

[0011] Preferably, the initial model architecture includes an input layer, two hidden layers and an output layer connected in turn, where the two hidden layers are a first RELU function layer and a second RELU function layer respectively, and the output layer is a SOFTMAX function layer.

[0012] Preferably, the exit function is set in the first RELU function layer and the second RELU function layer, and the disable probability is set, and the Euclidean norm is used for regularization.

[0013] Preferably, the initial model architecture includes a single set or multiple sets of neural network elements in a cascading form, and each neural network element includes a one-dimensional convolution layer, an activation function layer, and a normalization layer connected in turn, and the calculation method of the normalization layer is as follows: (1) (2) Data input one-dimensional convolution layer and get feature map after activation function operation in activation layer, H c The cth channel of the feature map output by the activation function layer is represented by G, and the dimension size of each channel in the feature map is represented by H c [n] represents the value of the nth dimension of the channel, and G represents the dimension size of each channel. The mean μ of each channel is obtained by formula (1) and formula (2) c and variance c The normalized feature map is obtained as follows: , where ε is a constant, and the normalized feature map V c [n] is sent to the subsequent network model for training.

[0014] Preferably, in step 4, the initial model architecture includes a preliminary screening model and a deep recognition model; the ratio feature, the road surface type and the road surface state are input into the preliminary screening model for training until the training target is reached; The ratio feature, the difference feature, the road surface state and the road surface type are input into the deep recognition model for training until the training target is reached.

[0015] Preferably, the deep recognition model is trained by using the cosine similarity to define the loss function, specifically: The data categories are defined by the road surface type, the road surface state and the wavelength of the emitted light, and the data distribution center S k of each category is calculated: where M represents the total number of data under the same category, and represents the ith data under the category, which is the ratio feature or the difference feature; S k represents the data distribution center of the kth category, and D k represents the cosine similarity value between E i and S k , then The loss function L is defined as follows:

[0016] where is a positive correlation loss function, and the convergence direction is that the points of the same category are close in space, and the points of different categories are far away; k represents the category, and N is the total number of categories.​​​ Gradient back-propagation of the loss function L is conducted to update the model parameters, and iterative iteration is repeated until convergence.

[0017] Preferably, step 5 is specifically: Step 5-1. According to steps 1 to 3, collect road data and process and feature extraction to obtain a detection data set; Step 5-2. The ratio feature and road type in the detection data set are input into the preliminary screening model trained in step 4 to identify, and according to the classification confidence of the output composite reflection state and single reflection state, it is judged whether the road state is a composite reflection state or a single reflection state; The composite reflection state includes two states of water accumulation and icing, and the single reflection state is other state except the composite reflection state; Step 5-3. Adjust the output layer of the deep identification model according to the judgment result of step 5.2, and input the detection data set into the deep identification model for identification; when step 5.2 judges that it is a composite reflection state, adjust the output layer of the deep identification model to output only the judgment results of water accumulation and icing two states, and when it is judged to be a single reflection state, adjust the output layer of the deep identification model to output only the judgment results of dry, wet and snow three states; Finally, the road state is judged by the deep identification model.

[0018] Preferably, step 5 further comprises abnormal point recognition, specifically: set a confidence abnormal threshold, when the classification result confidence of a certain point data in the input data is lower than the confidence abnormal threshold, it is determined that the point is an abnormal point, and the classification result is eliminated.

[0019] Preferably, the method for judging the road state in step 5 is to judge by using an effective result fusion judgment method, specifically: The classification results output by the model are fused by using a weighted voting method, that is: the score of each road state is the sum of the product of the number of times of the state appearing in the effective point and the confidence of the corresponding point, and the state with the highest score is taken as the final road state detection result. State score S = Σ i (V i *C i ), wherein V i and C i represent the identification value of the state appearing at the ith point and the confidence of the point, respectively. Wherein, the calculation method of V i is that if a state appears at point i, the identification value is 1, otherwise 0.

[0020] The infrared multi-wavelength road surface state classification detection method based on multi-point fusion has the advantages that the infrared multi-wavelength technology is used to collect data, rough classification of road surface states is trained and pre-judged, the overall judgment speed is improved, more abundant road surface radiation information can be obtained compared with single wavelength, the distinguishing ability of similar road surface states is improved, the classification accuracy can reach more than 90%, the present application extracts a plurality of waveband characteristics and uses a deep neural network model for classification, fully utilizes the characteristics of infrared multi-wavelength data, and further improves the classification accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 It is a specific embodiment flowchart of the infrared multi-wavelength road surface state classification detection method. Figure 2 It is a specific embodiment schematic diagram of the initial model architecture. Figure 3 It is a schematic diagram of the composite reflection state and the single reflection state. Figure 4 It is a specific embodiment schematic diagram of steps 4 and 5 in the method. Figure 5 It is a preferred embodiment schematic diagram of the initial model architecture in the method. Figure 6 It is a training effect schematic diagram of the initial model architecture in the method. DETAILED DESCRIPTION

[0022] The specific embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0023] The present application utilizes the differences in infrared reflection waves on different road surfaces to judge the road surface state. Different road surface states, such as dry, wet, icy, and snowy, have different compositions and physical structures, which will result in different absorption, reflection, and emission characteristics of different wavelengths. For example, water and ice have different absorption characteristics of near-infrared radiation, and by comparing the differences and ratios of radiation intensity at specific wavelengths, it can be determined whether there is water on the road surface and the state of the water.

[0024] According to the Beer-Lambert law, the intensity of light attenuates in proportion to the concentration of the medium, the optical path length, and the absorption coefficient of the medium. In road surface state detection, road surface materials can be regarded as a medium, different road surface states correspond to different absorption coefficients, and by measuring the changes in radiation intensity at different wavelengths, the road surface state can be inferred.

[0025] Based on the above principle, the infrared multi-wavelength road surface state classification detection method based on multi-point fusion has the advantages that the infrared multi-wavelength technology is used to collect data, rough classification of road surface states is trained and pre-judged, the overall judgment speed is improved, more abundant road surface radiation information can be obtained compared with single wavelength, the distinguishing ability of similar road surface states is improved, the classification accuracy can reach more than 90%, the present application extracts a plurality of waveband characteristics and uses a deep neural network model for classification, fully utilizes the characteristics of infrared multi-wavelength data, and further improves the classification accuracy. Figure 1As shown, comprising the following steps: Step 1. Multi-point data acquisition: Step 1-1. Device configuration: use a detection device integrated with an infrared laser module and a gimbal, the infrared laser module is used to collect the radiation intensity data of the road surface at three infrared wavebands of 980nm, 1310nm and 1550nm; the gimbal is used to drive the infrared laser module to rotate and transform, realizing the detection of multiple points of the road surface.

[0026] Step 1-2. Detection area planning: divide the detection range of the infrared laser module into multiple detection points, which are evenly distributed in the preset road surface detection area, which is generally a regular geometric shape such as a 1m x 1m square area, forming a comprehensive coverage of the detection section.

[0027] Step 1-3. Data acquisition process: control the gimbal to rotate regularly, the rotation period can be set according to actual needs, such as 10 seconds / time, drive the infrared laser module to align with k detection points in turn, and collect three waveband radiation intensity data of each point; Step 2. Data preprocessing: Step 2-1. Eliminate invalid data: according to the radiation calibration range of the infrared laser equipment, eliminate abnormal data that exceeds the effective range of each point; combined with prior knowledge of road surface state, eliminate obviously contradictory data, for example, a point has been marked as frozen, but the radiation intensity is abnormally high, the data of this point is obviously unreasonable, which may be caused by a temporary vehicle at the scanning position, and it is eliminated.

[0028] Step 2-2. Data standardization: The three waveband radiation intensity data after preprocessing is normalized, and the data of each waveband is mapped to the interval [0, 1], the formula is as follows: The normalized value XAV=(X-X min ) / (X max -X min ) Where X is a single data, X min and X max are the minimum and maximum values of all preprocessed data of this waveband.

[0029] Step 3. Feature extraction: Based on the three waveband radiation intensity data after standardization, the following features are extracted for each point: Define the normalized value of each waveband radiation intensity: the normalized values of the radiation intensity of 980nm, 1310nm and 1550nm infrared wavebands are IR980_norm, IR1310_norm and IR1550_norm respectively; For infrared laser module, the detected radiation intensity value IR=ER+I λ *(1-T λ )*m λ / D 2 ; Where ER represents the detection inherent zero drift error of each infrared laser module, I is the emitted light signal intensity, T is the infrared reflection coefficient, m represents the diffuse reflection coefficient, and D is the distance from the infrared detection module to the road. The parameters T and m are related to the road type and road condition, and the subscript λ represents the infrared light wavelength.

[0030] Step 3.1 Extract the difference feature: The first difference feature D1=IR980_norm-IR1310_norm, The second difference feature D2=IR980_norm-IR1550_norm, The third difference feature D3=IR1310_norm-IR1550_norm.

[0031] For example, D1=[I λ1 *(1-T λ1 )*m λ1 -I λ2 *(1-T λ2 )*m λ2 ] / D 2 ; λ1 and λ2 are 980 and 1310 nanometers respectively. I λ1 , I λ2 represent the emitted light signal intensity at wavelengths λ1 and λ2 respectively, T λ1 , T λ2 represent the infrared reflection coefficient at wavelengths λ1 and λ2 respectively, m λ1 , m λ2 represent the diffuse reflection coefficient at wavelengths λ1 and λ2 respectively, it can be seen that the detection inherent zero drift error is eliminated after subtracting each difference feature; when setting the same wavelength of the emitted light intensity as I0, that is, I0=I λ1 =I λ2 , For the same detection point, the distance D is equal, and the following can be obtained D1=[(1-T λ1 )*m λ1 -(1-T λ2 )*m λ2 ]*I0 / D 2 ; Step 3.2. Extract the ratio feature: The first ratio feature R1=D1 / D2, The second ratio characteristic is R2=D1 / D3. The third ratio characteristic is R3 = D2 / D3; Since the emitted light intensity and distance are the same, taking R1 as an example, R1=D1 / D2=[(1-T λ1 )*m λ1 -(1-T λ2 )*m λ2 ] / [(1-T λ1 )*m λ1 -(1-T λ3 )*m λ3 As can be seen, when the emitted light intensity is the same at different wavelengths, the first ratio feature R1 is independent of both the emitted light intensity I0 and the distance D. Similarly, the ratio features at different wavelengths at the same detection point are independent of both the emitted light intensity and the distance.

[0032] Step 4. Train the model Step 4-1. Build the model framework Build the initial model architecture, which can adopt a neural network structure. One specific implementation method is as follows: Figure 2 As shown, the neural network includes an input layer, two hidden layers, and an output layer. The hidden layers are ReLU function layers, and the output layer is a SOFTMAX function layer, outputting the classification of the road surface state. To prevent overfitting, a DROPOUT mechanism can be implemented in the neural network. The DROPOUT function is set in the first and second ReLU function layers with a disable probability PD=0.5. Euclidean norm (L2 norm) regularization is then applied to further improve generalization ability.

[0033] Regarding the present invention's use of difference and ratio features as training data, a preferred embodiment is as follows: Figure 5 As shown, the initial model architecture can use one or more cascaded neural network elements, each of which includes a cascaded one-dimensional convolutional layer, an activation function layer, and a normalization layer.

[0034] The normalization layer is calculated using the following formula: (1) (2) Data is input into a one-dimensional convolutional layer and then subjected to activation function operations in the activation function layer to obtain the feature map, H. c H represents the c-th channel of the feature map. c [n] represents H c The value of the nth dimension of the channel, G represents the dimension of each channel, and the mean μ of each channel is obtained by formulas (1) and (2). c and variance ; The normalized feature spectrum can be obtained as follows: (3), wherein the stability constant ε is a small constant value, which can be set according to the required operation accuracy to avoid instability of the normalized value, for example, can be set to 1E-4 to 1E-10; the normalized feature spectrum V is obtained by using formula (3) c [n] into the subsequent network model.

[0035] In the present application, since the difference feature and the ratio feature of the infrared radiation intensity are used as the input feature data, the normalized feature spectrum in the form of mean value and variance is processed in the training process, and the subsequent training is performed, so that the commonality of different feature spectra can be determined more easily, and the category of the data is quickly clustered and determined, and the training of the model is completed with fewer iteration times.

[0036] As shown in Figure 6 , for the deep recognition model, two cascaded neural network elements shown in Figure 5 are connected in series, and the normalized layer defined by formula (1) and formula (2) is used for training, and after 80 iterations, the model tends to converge, and when the iteration is 90 times, the recall rate (RECALL), accuracy (accuracy), and precision (precision) of the model are 0.9428, 0.9435, and 0.9447, respectively.

[0037] Step 4-2. Model training The ratio feature, the road surface type, and the road surface state are input into the preliminary screening model for training until the training target is reached.

[0038] The ratio feature, the difference feature, the road surface state, and the road surface type are input into the deep recognition model for training until the training target is reached.

[0039] The road surface type is defined as asphalt pavement, cement pavement, bridge pavement, tunnel pavement, etc., and the road surface state classification can include dry, water accumulation, wet, icing, or snow accumulation, etc. 5 kinds of states.

[0040] The ratio feature and the road surface type in the training set are input into the preliminary screening model, and the road surface state is output for training until the training target is reached, since the number of ratio feature parameters is small and only related to the road surface state, the training speed can be improved.

[0041] For the road surface state, although it can be subdivided into dry, water accumulation, wet, icy or snow accumulation and other states, for the road surface refraction state, the water accumulation and icy states have commonality, in these two states, there is a layer of transparent material on the road surface, that is, a thin ice layer and a water layer, so that the infrared ray is simultaneously affected by significant reflection and refraction when reflected on the road surface. In other states, the road surface is in a single reflection state, and the above properties enable the road surface state to be roughly divided into a composite reflection state and a single reflection state, as shown in Figure 3

[0042] For the composite reflection state and the single reflection state, there is a significant difference between the light reflection coefficient and the diffuse reflection coefficient of the two states, for example, the light reflection coefficient at this time is not caused by a single medium, but by the reflection of the road surface and the refraction of the transparent material back to the infrared laser device, and the composite reflection state has the interface of two media, and the interface formed by the ice surface or the water surface and the air has more regularity in the directionality of reflection than the random form surface of the single medium, which can make the diffuse reflection coefficient m have a relatively obvious difference from the single medium road surface.

[0043] At the same time, according to the Fresnel law, when the incident is vertical, the infrared reflection coefficient TX=(i1-i2) / (i1+i2) of the infrared ray reflected from the air to the transparent medium such as ice or water, and the refractive indices of the air and the transparent medium are ii and i2 respectively, and further simplified as TX=1-△,△=2*i2 / (i1+i2), it can be seen that TX is obviously less than 1.

[0044] For the composite reflection state, the infrared reflection coefficient T=TX*TD, where TD is the infrared reflection coefficient when the infrared ray is directly reflected on the opaque medium, it can be seen that the infrared reflection coefficient obtained by the infrared ray penetrating through the transparent medium and then being reflected on the opaque medium such as the road surface is significantly reduced relative to the infrared reflection coefficient obtained by the infrared ray being directly reflected on the opaque medium.

[0045] Using the above characteristics contained in the two parameters of the infrared reflection coefficient and the diffuse reflection coefficient, after training the initial screening model, the ratio feature and the difference feature input by the detected road surface are identified by the initial screening model, and it can be easily determined whether the road surface state is a composite reflection state or a single reflection state.

[0046] When the initial screening model is trained, it can be all points or randomly selected part of the point data, so that it can be determined whether the current road surface state is a composite reflection state or a single reflection state.

[0047] ​The intensity normalized value, the ratio feature, the difference feature, the road surface state and the road surface type of the radiation of different wavelengths are input into the depth recognition model for training until the training target is reached. In order to appropriately reduce the data amount, the intensity normalized value can also not be input, and only the ratio feature, the difference feature, the road surface state and the road surface type are input into the depth recognition model for training.

[0048] All features of all point positions are input into the depth recognition model for training, and the current road surface state can be subdivided. Since the data amount is large and comprehensive, the subdivided road surface state is more accurate.

[0049] The training target of the model training can be a certain number of training times; or the road surface state loss function obtained by training reaches a set target value; or the road surface state classification confidence obtained by training reaches a set target value; for example, the confidence target value is set to 95%, and the confidence of the classification result of the road surface state distributed in the 0-100% interval obtained by training the preliminary screening model and the depth recognition model is greater than 95% if the road surface state classification confidence is greater than 95%, then the training is ended.

[0050] In order to further improve the training speed, the depth recognition model can be trained in a cosine similarity defined loss function manner, specifically: The data categories are defined by the road surface type, the road surface state and the wavelength of the emitted light, and the data distribution centers of each category are calculated: , Where M represents the total number of data in the same category, represents the i-th data in the category, which is the ratio feature or the difference feature; S k represents the data distribution center of the k-th category, and D k represents the cosine similarity value of E i and S k , then ; The loss function L is defined as follows: (4) Where is a positive correlation loss function, and the convergence direction is that the points of the same category are close in space, and the points of different categories are far away; k represents the category, and N is the total number of categories.

[0051] For example, there are 4 types of road surfaces, i.e. asphalt pavement, cement pavement, bridge pavement and tunnel pavement, 5 types of road surface states, i.e. dry, water accumulation, wet, icy and snow accumulation, and 3 types of difference characteristics of emitted light wavelengths, so the data for training the model has N=4*5*3=60 categories, each category has 100 points, and a data distribution center S can be calculated for all points in each category, and the cosine similarity value of the data distribution center and the corresponding loss function are further calculated for model training, and the trained model is suitable for application in the present application for identifying the road surface state of different types of road surfaces according to the difference characteristics and the ratio characteristics.

[0052] The loss function L defined by formula (4) is gradient backpropagated to update the model parameters, and repeated iteration is performed until convergence, where convergence refers to the loss function L no longer changing or changing by less than a certain set threshold.

[0053] Through the design of the above loss function, the ratio characteristics and difference characteristics data of infrared reflection of the same category can be coded into a relatively close range in a high-dimensional space, while the data of different categories increases the Euclidean distance in the high-dimensional space, the model convergence speed is accelerated, and the model training and recognition speed is improved.

[0054] As shown in Figure 6 , the training result of the deep recognition model using Figure 5 two cascaded neural network elements in series is shown, the normalization layer is defined by formula (1) and formula (2), and formula (4) is used as the loss function, after 80 iterations, the model tends to converge, and at the 90th iteration, the loss function is 0.2356, and the recall rate (RECALL), accuracy (accuracy) and precision (precision) of the model are 0.9428, 0.9435 and 0.9447 respectively.

[0055] Step 5. Road surface classification detection Step 5-1. According to steps 1 to 3, collect road surface data and perform processing and feature extraction to obtain a detection data set; Step 5-2. Input the ratio characteristics and road surface type in the detection data set into the preliminary screening model trained in step 4 to identify the road surface state, and determine whether the road surface state is a composite reflection state or a single reflection state according to the classification confidence of the output composite reflection state and single reflection state. Step 5-3. Adjust the output layer of the depth recognition model using the judgment results from Step 5.2. Input the entire detection dataset into the depth recognition model for recognition. When Step 5.2 determines that it is a composite reflection state, adjust the output layer of the depth recognition model to only output the judgment results for the two states of water accumulation and freezing. When it determines that it is a single reflection state, adjust the output layer of the depth recognition model to only output the judgment results for the three states of dryness, wetness, and snow accumulation.

[0056] Finally, the deep recognition model outputs the current road surface condition recognition result.

[0057] A flowchart illustrating the training and recognition process in steps 4 and 5 is shown below. Figure 4 As shown.

[0058] In a preferred embodiment, step 5 can perform anomaly identification. Specifically, anomaly identification involves: Set a confidence anomaly threshold. When the confidence of the classification result of a certain point in the input data is lower than the confidence anomaly threshold, the point is determined to be an anomaly point. The reason for the anomaly may be that there are road obstructions, road damage, etc. The classification result of the point is removed if the confidence is lower than the confidence anomaly threshold. The method for judging the road surface condition is the effective result fusion judgment method, which can be used in the initial screening model judgment in step 5-2 and the deep recognition model judgment in step 5-3.

[0059] The specific method for determining the fusion of valid results is as follows: The classification results are fused using a weighted voting method, whereby the score for each road surface condition is the sum of the product of the number of times that condition appears in the valid locations and the corresponding location confidence, and the condition with the highest score is taken as the final road surface condition detection result.

[0060] The formula is as follows: State score S=Σ i (V) i *C i ), where V i and C i These represent the identifier value and the confidence level of the state at the i-th point, respectively. Among them, V i The calculation method is such that if a certain state occurs at point i, the flag value is 1; otherwise, it is 0. Since this invention is based on collecting road surface information at different points using infrared sensors, the road surface state result with the largest area should be taken as the standard. For example, in icy and snowy weather, most roads will have snow accumulation, but some points may not have snow accumulation due to vehicle rolling, road surface shape, etc. The weighted voting method can summarize the road surface state result with the largest proportion of area.

[0061] Compared with the prior art, the infrared multi-wavelength road surface state classification detection method based on multi-point fusion has the following beneficial effects: 1. The infrared multi-wavelength technology is used to collect data, and compared with single wavelength, more abundant road surface radiation information can be obtained, the distinguishing ability of similar road surface states is improved, and the classification accuracy can reach more than 90%.

[0062] 2. The training and pre-judgment are performed for the rough classification of road surface states, the overall judgment speed is improved, the two models independently judge and output classification results, and the misrecognition rate is reduced.

[0063] 3. The non-contact data acquisition mode is used, the road surface structure does not need to be damaged, the installation and maintenance are convenient, and the cost is reduced. Through data preprocessing and standardization, the influence of equipment individual difference and environmental interference on data is effectively eliminated, and the stability and reliability of the detection method are improved.

[0064] 4. The multi-point detection and fusion strategy is introduced, the road surface is detected at multiple points through a holder, and a confidence weighted fusion algorithm is combined, the misdetection problem caused by local interference such as leaf blocking and road damage during single-point detection is effectively avoided, and the robustness of the algorithm and the reliability of the detection result are significantly improved.

[0065] 5. A variety of band features are extracted, and a CNN model is used for classification, the characteristics of infrared multi-wavelength data are fully utilized, and the classification accuracy is further improved.

[0066] The foregoing is each preferred embodiment of the present application, and each preferred embodiment can be arbitrarily combined and used if not obviously contradictory or with a certain preferred embodiment as a prerequisite. The embodiments and specific parameters in the embodiments are only for clearly describing the inventor's verification process and are not used to limit the patent protection scope of the present application. The patent protection scope of the present application is still subject to its claims, and any equivalent structural changes using the content of the specification and drawings of the present application should also be included in the protection scope of the present application.

Claims

1. A method for classifying and detecting road surface conditions based on multi-point fusion using infrared multi-wavelength technology, characterized in that, Includes the following steps: Step 1. Use an infrared laser module to emit lasers of different wavelengths to different points on the road surface and collect radiation intensity data at each point; Step 2. Preprocess the raw radiation intensity data; Step 3. Perform feature extraction on the preprocessed data, specifically including extracting difference features and ratio features. The difference features are the differences between radiation intensity data of different wavelengths at the same location, and the ratio features are the ratios of different difference features. Step 4. Build the initial model architecture. Use the difference features and ratio features obtained in Step 3, as well as the road surface type and road surface condition as training set data to train the model until the training objective is achieved and the road surface condition detection model is obtained. Step 5. For the road surface to be detected, collect data according to steps 1 to 3 to obtain the detection dataset, input it into the road surface condition detection model trained in step 4, and determine the road surface condition.

2. The infrared multi-wavelength road surface condition classification and detection method as described in claim 1, characterized in that, Step 2 specifically includes: Step 2. Data Preprocessing: Step 2-1. Invalid data removal: Based on the radiation calibration range of the infrared laser equipment, remove abnormal data that exceeds the valid range at each point; combine prior knowledge of road conditions to remove obviously contradictory data. Step 2-2. Data Standardization: The preprocessed radiation intensity data for the three bands are normalized to map the data of each band to the [0,1] interval, as shown in the following formula: Normalized value XAV=(XX) min ) / (X max -X min ) Where X is a single data point, X min and X max These are the minimum and maximum values ​​of all preprocessed data for this band, respectively.

3. The infrared multi-wavelength road surface condition classification and detection method as described in claim 1, characterized in that, The initial model architecture includes an input layer, two hidden layers and an output layer connected in sequence, wherein the two hidden layers are a first ReLU function layer and a second ReLU function layer, and the output layer is a SOFTMAX function layer.

4. The infrared multi-wavelength road surface condition classification and detection method as described in claim 3, characterized in that, An exit function and a disable probability are set in the first and second ReLU function layers, and regularization is performed using the Euclidean norm.

5. The infrared multi-wavelength road surface condition classification and detection method as described in claim 1, characterized in that, The initial model architecture includes one or more sets of cascaded neural network elements. Each neural network element includes a one-dimensional convolutional layer, an activation function layer, and a normalization layer connected in sequence. The normalization layer is calculated using the following formula: （1） （2） Data is input into a one-dimensional convolutional layer and then subjected to activation function operations in the activation layer to obtain the feature map, H. c H represents the c-th channel of the feature map output by the activation function layer, G represents the dimension of each channel in the feature map, and H represents the dimension of each channel in the feature map. c [n] represents H c The value of the nth dimension of the channel, G represents the dimension of each channel, and the mean μ of each channel is obtained by formulas (1) and (2). c and variance ; ， Where ε is the stability constant, and V is the normalized characteristic spectrum. c [n] is fed into the subsequent network model for training.

6. The infrared multi-wavelength road surface condition classification and detection method as described in claim 1, characterized in that, In step 4, the initial model architecture includes a preliminary screening model and a deep recognition model; during training, the ratio features, road surface type and road surface state are input into the preliminary screening model for training until the training target is achieved. The ratio features, difference features, road surface condition, and road surface type are input into the deep recognition model for training until the training objective is achieved.

7. The infrared multi-wavelength road surface condition classification and detection method as described in claim 6, characterized in that, The deep recognition model is trained using a loss function defined by cosine similarity, specifically as follows: Data categories are defined by road surface type, road surface condition, and emitted light wavelength, and the data distribution center S of each category is calculated. k : Where M represents the total number of data points in the same category, This indicates that the i-th data point in this category is a ratio feature or a difference feature; S k Let D represent the data distribution center of the k-th category. k E represents i With S k The cosine similarity value, then ; The loss function L is defined as follows: ； in The positive correlation loss function converges in the direction that points of the same class move closer together in space, while points of different classes move further apart; k represents the class, and N is the total number of classes; The loss function L is backpropagated using gradients to update the model parameters, and this process is repeated iteratively until convergence.

8. The infrared multi-wavelength road surface condition classification and detection method as described in claim 6, characterized in that, Step 5 specifically involves: Step 5-1. Following steps 1 to 3, collect road surface data, process it, and extract features to obtain the detection dataset; Step 5-2. Input the ratio features and road surface type from the detection dataset into the initial screening model obtained after training in Step 4 for identification. Based on the classification confidence level of the output composite reflection state and single reflection state, determine whether the road surface state is a composite reflection state or a single reflection state. The composite reflection state includes two states: water accumulation and ice formation; the single reflection state refers to any other state besides the composite reflection state. Step 5-3. Adjust the output layer of the depth recognition model using the judgment result of step 5.

2. Input all the detection dataset into the depth recognition model for recognition. When step 5.2 determines that it is a composite reflection state, adjust the output layer of the depth recognition model to only output the judgment results of water accumulation and freezing. When it determines that it is a single reflection state, adjust the output layer of the depth recognition model to only output the judgment results of dry, wet and snow accumulation. Finally, the deep recognition model determines the road surface condition.

9. The infrared multi-wavelength road surface condition classification and detection method as described in claim 1, characterized in that, Step 5 also includes anomaly identification, specifically: setting an anomaly confidence threshold; when the confidence of the classification result of a certain point in the input data is lower than the anomaly confidence threshold, the point is determined to be an anomaly and the classification result is removed.

10. The infrared multi-wavelength road surface condition classification and detection method as described in claim 1 or 8, characterized in that, The method for determining the road surface condition in step 5 is to use a valid result fusion judgment method, specifically: The classification results output by the model are fused using a weighted voting method. That is, the score of each road surface state is the sum of the product of the number of times the state appears in the valid points and the corresponding point location confidence. The state with the highest score is taken as the final road surface state detection result. State score S=Σ i (V) i *C i ), where V i and C i These represent the identifier value and the confidence level of the state at the i-th point, respectively. Among them, V i The calculation method is as follows: if a certain state occurs at point i, the value of the identifier is 1; otherwise, it is 0.

Images