An industrial vision-based asphalt paving quality detection method

Abstract

The application relates to the technical field of road construction quality detection, in particular to an asphalt paving quality detection method based on industrial vision; the method comprises the following steps: synchronously collecting visible light images, infrared thermal images, three-axis acceleration, scattering echo intensity and vehicle speed, and realizing window synchronization through time stamps; determining smoke dust shielding degrees according to the scattering echo intensity, and combining acceleration and vehicle speed to determine vibration stability, smoke dust shielding and paving stop states; processing the visible light images or the infrared thermal images in different states respectively, extracting paving texture, surface defects, edge material shortage, temperature mean value, temperature variance, temperature gradient and local thermal anomaly features, combining motion stability coefficients to generate comprehensive quality scores and output qualified results or abnormal alarms; the application improves the reliability of vibration stability and stop state recognition, and avoids misjudgment caused by instantaneous fluctuation.

Description

Technical Field

[0001] This invention relates to the field of road construction quality inspection technology, specifically to a method for inspecting asphalt paving quality based on industrial vision. Background Technology

[0002] In asphalt paving quality inspection, it is usually necessary to make a comprehensive judgment on the texture, defects, temperature distribution of the paved surface, and the operating status of the equipment.

[0003] Currently, most detection methods rely on single-channel information from visible light images or infrared thermal images. Although some solutions introduce vibration or vehicle speed data to assist in judgment, they often still adopt independent acquisition and analysis methods.

[0004] However, when conducting detection using the above methods, inconsistencies in the sampling timing of various data can easily lead to discrepancies between the image and the motion state, resulting in unstable judgment results. At the same time, smoke, water vapor, and dust obstruction at the paving site can reduce the quality of visible light images, while support vibration and vehicle shutdown switching can further interfere with image analysis, causing frequent switching between visible light and infrared channels and invalid data entering the back-end processing link, which in turn leads to problems such as distorted quality scores and false alarms. Summary of the Invention

[0005] The purpose of this invention is to provide a method for inspecting asphalt paving quality based on industrial vision, avoiding unstable judgment results caused by inconsistent sampling timing and interference from dust, vibration, etc., and achieving a comprehensive score for paving quality by coordinating multi-source data. To achieve the above objective, this invention specifically adopts the following technical solution:

[0006] S1. Visible light images, infrared thermal images, three-axis acceleration, scattered echo intensity, and vehicle speed are acquired through a visible light industrial camera, an infrared thermal imaging camera, a six-axis inertial measurement unit mounted on the camera bracket, a side-axis laser scattering sensor, and a vehicle controller LAN bus, respectively; timestamps are added by a unified sampling clock, and each data is synchronized within the same time window according to the timestamp;

[0007] S2. Determine the dust obstruction degree based on the scattered echo intensity, determine the vibration stability state based on the triaxial acceleration, determine the average speed and paving machine stop state based on the vehicle speed, and obtain the preset dust obstruction degree threshold. The dust obstruction degree threshold is the obstruction degree threshold calibrated based on the unobstructed reference echo intensity and the dark field echo baseline value.

[0008] S3. First determine the paving machine stop status; in the non-paving machine stop status, when the dust obscuration exceeds the dust obscuration threshold or the vibration is unstable, it is determined to be a dust obscuration status, and other situations are determined to be normal collection status.

[0009] S4. Process visible light images under normal acquisition conditions, process infrared thermal images under smoke and dust obstruction conditions, and perform shutdown maintenance under paving shutdown conditions.

[0010] S5. Generate a visible light quality score based on the paving texture, surface defects, and edge missing material features extracted from the visible light image. Generate an infrared quality score based on the mean temperature, temperature variance, temperature gradient, and local thermal anomaly features extracted from the infrared thermal image. Determine the motion stability coefficient based on the triaxial acceleration. Generate a comprehensive quality score according to the preset weights. Compare the comprehensive quality score with the preset quality threshold for the corresponding working condition. Output a qualified result when the comprehensive quality score is greater than or equal to the preset quality threshold; otherwise, output an abnormal alarm result.

[0011] Preferably, the triaxial acceleration output by the six-axis inertial measurement unit is transformed to the device coordinate system and then... Processed on a unit-by-unit basis.

[0012] Preferably, the intensity of the scattered echoes within the same time window is averaged, and the smoke and dust obstruction degree is determined according to the following formula:

[0013]

[0014] in, To reduce the obstruction of smoke and dust, The average scattered echo intensity within the time window. The background echo basis value under dark field conditions. The reference echo intensity is under unobstructed calibration conditions, and ; and The equipment is calibrated after installation and updated according to the preset calibration cycle.

[0015] Preferably, the triaxial acceleration is filtered, and the filtered acceleration vector at the current sampling time is... and according to and Determine the vibration stability state;

[0016] When the conditions are met continuously within the preset stable time window and When this condition is met, it is determined to be a stable vibration state; among which, The acceleration vector at the current sampling moment. The acceleration vector at the previous sampling time. To preset the acceleration magnitude threshold, This is a preset threshold for the change in acceleration.

[0017] As a preferred method, the average speed is calculated based on the vehicle controller local area network bus speed within a preset time window:

[0018]

[0019] in, The average velocity of the motion, This represents the number of valid sampling points within the time window. For the first Vehicle speed at each sampling time;

[0020] When the average speed of motion within a continuous time window is not greater than the stopping speed threshold And the duration is not less than the downtime duration. At that time, it was determined that the paving machine was in a stopped state.

[0021] Preferably, a first smoke threshold is set, and a smoke obstruction threshold is used as a second smoke threshold, wherein the first smoke threshold is greater than the second smoke threshold;

[0022] When the smoke and dust obstruction degree is greater than the second smoke and dust threshold and not greater than the first smoke and dust threshold, the smoke and dust obstruction state is entered and infrared thermal images are collected.

[0023] When the smoke and dust obstruction exceeds the first smoke and dust threshold, increase the sampling frequency or integration time of the infrared thermal imaging camera, and adjust the infrared image gain before performing infrared thermal image processing.

[0024] The first and second smoke thresholds are calibrated based on the unobstructed reference echo intensity, the dark field echo baseline value, and historical smoke samples, and are updated according to a preset calibration cycle.

[0025] As a preferred option, under normal acquisition conditions, the control unit set on the paving equipment sends a hardware trigger pulse to the visible light industrial camera through the field programmable gate array controller, so that the visible light industrial camera uses global shutter exposure.

[0026] The field-programmable gate array (FPGA) controller transmits the raw images output by the visible light industrial camera to the computing unit connected to the FPGA controller via a direct memory access channel.

[0027] The computing unit sequentially performs black level correction, noise reduction, distortion correction, paving region segmentation, and extraction of paving texture, surface defects, and edge missing material features from the original image, and generates a visible light quality score based on the extracted features.

[0028] As a preferred embodiment, under the condition of smoke and dust obscuring the image, the field programmable gate array controller blocks the direct memory access transmission of the image from the visible light industrial camera to the computing unit, and the infrared thermal imaging camera acquires the infrared thermal image.

[0029] The infrared thermal image is corrected for emissivity, ambient temperature, distance, and non-uniformity. The regional reference temperature is determined based on the average temperature of the paving area or historical temperature data. Pixels whose difference from the regional reference temperature exceeds a preset temperature difference threshold are identified as temperature abnormal pixels. Thermal anomalies are determined based on the proportion of temperature abnormal pixels in the paving area.

[0030] Preferably, in the paving stop state, the control unit shuts down the visible light acquisition main link, triggers the infrared thermal imaging camera to perform non-uniformity correction when entering the paving stop state, clears the cache queue in the calculation unit, and reduces the infrared thermal image processing frequency from the first processing frequency to the second processing frequency, wherein the second processing frequency is less than the first processing frequency and not lower than the preset minimum processing frequency.

[0031] Simultaneously, the lens heating device, which is connected to the control unit and located at the camera lens, is activated.

[0032] As a preferred option, the visible light quality score Infrared quality score and motion stability coefficient Normalized to the preset upper and lower limits And generate a comprehensive quality score according to the following formula:

[0033]

[0034] in, For the overall quality score, , and For the preset weighting coefficients, and ;

[0035] When visible light images are not acquired or their quality is below a preset effective threshold under smoke and dust obstruction, the following conditions are met: If a score is invalid, it should be removed and the weights of the remaining valid scores should be re-normalized proportionally.

[0036] The overall quality score is output according to a fixed time window and compared with the preset quality threshold for the corresponding working condition to output a qualified result or an abnormal alarm result.

[0037] Compared with the prior art, the present invention has the following technical effects:

[0038] 1. This invention avoids timing misalignment caused by independent acquisition of each channel by uniformly collecting visible light images, infrared thermal images, triaxial acceleration, scattered echo intensity, and vehicle speed with a unified timestamp and processing them synchronously within the same time window; at the same time, it separates and determines vibration, dust obstruction, and shutdown states, so that normal acquisition state, dust obstruction state, and paving shutdown state adopt different processing flows, effectively reducing the occurrence of invalid images entering the back-end analysis link;

[0039] 2. This invention quantifies the intensity of the scattered echo as the degree of smoke and dust obstruction, and combines it with dual-threshold hierarchical control to realize channel switching. When the degree of smoke and dust obstruction is greater than the second smoke and dust threshold but not greater than the first smoke and dust threshold, the infrared thermal image takes over. When the degree of smoke and dust obstruction is greater than the first smoke and dust threshold, the infrared acquisition capability is improved, thereby reducing the interference of smoke, dust, water vapor and dust on the detection results.

[0040] 3. This invention improves the reliability of vibration stability and shutdown status identification by filtering, unifying coordinates, and using continuous window judgment on triaxial acceleration, thus avoiding misjudgments caused by instantaneous fluctuations. Attached Figure Description

[0041] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application:

[0042] Figure 1 This is a flowchart illustrating an industrial vision-based asphalt paving quality inspection method provided in an embodiment of the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings.

[0044] Please see Figure 1 A method for inspecting asphalt paving quality based on industrial vision, comprising:

[0045] S1. Visible light images, infrared thermal images, three-axis acceleration, scattered echo intensity, and vehicle speed are acquired through a visible light industrial camera, an infrared thermal imaging camera, a six-axis inertial measurement unit mounted on the camera bracket, a side-axis laser scattering sensor, and a vehicle controller LAN bus, respectively; timestamps are added by a unified sampling clock, and each data is synchronized within the same time window according to the timestamp;

[0046] S2. Determine the dust obstruction degree based on the scattered echo intensity, determine the vibration stability state based on the triaxial acceleration, determine the average speed and paving machine stop state based on the vehicle speed, and obtain the preset dust obstruction degree threshold. The dust obstruction degree threshold is the obstruction degree threshold calibrated based on the unobstructed reference echo intensity and the dark field echo baseline value.

[0047] S3. First determine the paving machine stop status; in the non-paving machine stop status, when the dust obscuration exceeds the dust obscuration threshold or the vibration is unstable, it is determined to be a dust obscuration status, and other situations are determined to be normal collection status.

[0048] S4. Process visible light images under normal acquisition conditions, process infrared thermal images under smoke and dust obstruction conditions, and perform shutdown maintenance under paving shutdown conditions.

[0049] S5. Generate a visible light quality score based on the paving texture, surface defects, and edge missing material features extracted from the visible light image. Generate an infrared quality score based on the mean temperature, temperature variance, temperature gradient, and local thermal anomaly features extracted from the infrared thermal image. Determine the motion stability coefficient based on the triaxial acceleration. Generate a comprehensive quality score according to the preset weights. Compare the comprehensive quality score with the preset quality threshold for the corresponding working condition. Output a qualified result when the comprehensive quality score is greater than or equal to the preset quality threshold; otherwise, output an abnormal alarm result.

[0050] The triaxial acceleration output from the six-axis inertial measurement unit is transformed into the device coordinate system and then... Processed by unit;

[0051] The intensity of the scattered echoes within the same time window is averaged, and the smoke and dust obstruction degree is determined as follows:

[0052]

[0053] in, To reduce the obstruction of smoke and dust, The average scattered echo intensity within the time window. The background echo basis value under dark field conditions. The reference echo intensity is under unobstructed calibration conditions, and ; and The equipment is calibrated after installation and updated according to the preset calibration cycle.

[0054] This embodiment synchronously acquires visible light, infrared, vibration, scattered echo, and vehicle speed data, using a unified sampling clock with added timestamps to ensure that all data types fall within the same time window for corresponding processing, avoiding timing misalignment; the three-axis acceleration is transformed to the device coordinate system, so as to... The unit participates in subsequent judgments to correlate support vibration with image clarity and exposure stability;

[0055] The average intensity of the scattered echo within the preset time window is taken, and the smoke and dust occlusion degree is calculated by combining the unobstructed reference echo intensity and the dark field echo baseline value. When the scattered echo is close to the dark field echo baseline value, it indicates that the occlusion of the visible light channel by smoke, water vapor or dust is enhanced. When the scattered echo is close to the unobstructed reference echo intensity, it indicates that the acquisition environment is suitable for visible light image analysis.

[0056] The degree of smoke and dust obstruction is limited to between 0 and 1, and and After the equipment is installed, it is calibrated and updated according to the preset calibration cycle to compensate for the offset caused by changes in installation angle, mirror contamination and environmental changes;

[0057] This embodiment achieves cross-channel coordinated changes by using a unified time window, quantitative evaluation of smoke and dust obstruction, and vibration judgment after coordinate unification, so as to reduce invalid images entering the back-end processing link.

[0058] Visible light quality score The three parameters are normalized and weighted: texture integrity, surface defect ratio, and edge material shortage ratio. Texture integrity is represented by the ratio of clear texture area to paved area area; surface defect ratio is represented by the ratio of abnormal pixel area (cracks, pitting, segregation, etc.) to paved area area; and edge material shortage ratio is represented by the ratio of edge material shortage pixel length to boundary length.

[0059] Among them, the area of ​​clear texture is obtained by extracting high-frequency texture pixels by setting the image gray-level gradient threshold; the area of ​​abnormal pixels is obtained by comparing the pixel gray-level values ​​in the paving area with the preset normal gray-level range and counting the number of pixels that exceed the range; the length of edge missing material pixels is obtained by extracting the paving boundary using the Canny edge detection algorithm and counting the number of pixels that deviate from the optimal fitted straight line distance greater than the preset edge tolerance value.

[0060] Infrared quality score The temperature variance is obtained by normalizing and weighting four factors: mean temperature deviation, temperature variance, temperature gradient, and the proportion of local thermal anomalies. The mean temperature deviation reflects the degree of deviation between the paved area and the reference temperature, while the proportion of local thermal anomalies reflects the coverage of abnormal pixels within the paved area. The formula for calculating the temperature variance is as follows:

[0061]

[0062] The formula for calculating the temperature gradient is:

[0063]

[0064] in, For temperature variance, For temperature gradient, For the first in the paving area Temperature value of each pixel The average temperature of the region. For pixels in Temperature partial derivative in the direction, For pixels in Temperature partial derivative in the direction, This represents the total number of effective pixels; the weighted fusion logic for infrared quality scores is the same as that for visible light scores.

[0065] The aforementioned individual features are first mapped to a range of 0 to 1 before participating in subsequent weight fusion. The specific mapping function uses a linear normalization formula:

[0066]

[0067] in, This represents the actual calculated value of a single feature. and These are the lower and upper limits of the feature extracted from historical qualified and abnormal samples, respectively, and the visible light quality score. The weighted fusion formula is as follows:

[0068]

[0069] in, Preset weights and satisfy ; This is the normalized calculated value of the i-th visible light quality feature;

[0070] In terms of status determination, the paving machine is first determined to be in a stopped state. When the vehicle speed is continuously lower than the stopping speed threshold and the stopping condition is met, the machine is put into a stop maintenance process. If it is not in a stopped state, the dust obscuration degree is determined to be in a dust obscuration state when the dust obscuration degree exceeds the dust obscuration degree threshold or the vibration is unstable. Other situations are determined to be in a normal collection state. Stop, dust obscuration and normal collection are processed separately.

[0071] Process visible light images under normal acquisition conditions, process infrared thermal images under smoke and dust obstruction conditions, and perform shutdown maintenance under shutdown conditions;

[0072] Visible light images focus on extracting paving texture, surface defects, and edge material shortage features; infrared thermal images focus on extracting temperature mean, temperature variance, temperature gradient, and local thermal anomaly features; then, combined with the motion stability coefficient obtained from triaxial acceleration, a comprehensive quality score is formed according to preset weights, and compared with the preset quality threshold for the corresponding working condition, outputting a qualified result or an abnormal alarm result.

[0073] This embodiment is based on multi-source synchronous acquisition, smoke and dust obstruction quantification and state diversion processing to achieve stable judgment of asphalt paving quality and avoid interference from smoke and dust obstruction, vibration fluctuations and machine shutdown switching on quality detection;

[0074] In an application embodiment, the method can be deployed on the edge computing unit of a paver; after the field data enters the unified processing flow through the camera, inertial measurement unit, laser scattering sensor and vehicle bus, the system selects the corresponding image channel according to the current status and uses the comprehensive quality score to output qualified or abnormal alarms for on-site quality control personnel to handle in a timely manner.

[0075] The motion stability coefficient is determined by the filtered triaxial acceleration based on the degree of vibration deviation at the current moment, and is used as a normalization coefficient corresponding to the degree of vibration instability.

[0076]

[0077] When the stability condition is continuously met within the preset stability time window The value should be close to 1; otherwise, it decreases to 0 as the exceedance increases. (Visible light quality score) Infrared quality score Each sample first determines its upper and lower limits based on its historical qualified and abnormal samples, then normalizes them to 0 to 1 before participating in subsequent weighted fusion.

[0078] In one embodiment, the triaxial acceleration is filtered, and the current sampling time acceleration vector after filtering is... and according to and Determine the vibration steady state; when the conditions are continuously met within the preset steady-state time window and When this condition is met, it is determined to be a stable vibration state; among which, The acceleration vector at the current sampling moment. The acceleration vector at the previous sampling time. To preset the acceleration magnitude threshold, The preset threshold for acceleration change;

[0079] Calculate the average speed of movement based on the vehicle controller area network bus speed within a preset time window:

[0080]

[0081] in, The average velocity of the motion, This represents the number of valid sampling points within the time window. For the first The vehicle speed at each sampling time; the average speed within a continuous time window is not greater than the stopping speed threshold. And the duration is not less than the downtime duration. At that time, it was determined that the paving machine was in a stopped state;

[0082] Set a first smoke threshold and use the smoke obstruction threshold as a second smoke threshold, and the first smoke threshold is greater than the second smoke threshold;

[0083] When the smoke and dust obstruction degree is greater than the second smoke and dust threshold and not greater than the first smoke and dust threshold, the smoke and dust obstruction state is entered and infrared thermal images are collected.

[0084] When the smoke and dust obstruction exceeds the first smoke and dust threshold, increase the sampling frequency or integration time of the infrared thermal imaging camera, and adjust the infrared image gain before performing infrared thermal image processing.

[0085] The first and second smoke thresholds are calibrated based on the unobstructed reference echo intensity, the dark field echo baseline value, and historical smoke samples, and are updated according to a preset calibration cycle.

[0086] , , and After the equipment is installed, it is calibrated using no-load stability samples and manual shutdown samples. After changing the camera bracket installation posture or vehicle chassis parameters, it is recalibrated according to the preset calibration cycle to ensure that the vibration stability and shutdown judgment boundary are consistent.

[0087] After the vibration signal is filtered, the acceleration vector at the current sampling time is used to... The stability is determined by the magnitude and the difference between adjacent sampling times; the magnitude represents the overall vibration intensity of the support, and the difference represents the vibration change rate; when both continuously meet the threshold condition within the preset stable time window, the camera support is in a stable segment to reduce motion blur.

[0088] By using continuous windows and dual-condition superposition to determine the continuous and stable trend of captured vibrations, a quantitative evaluation mechanism for data acquisition is established.

[0089] The determination of the vehicle's stopped state also adopts the sliding window averaging method; the average speed is calculated based on the vehicle controller LAN bus speed within a preset time window. ,in This indicates the number of valid sampling points, used to balance the effects of sampling sparsity and sampling jitter.

[0090] When the average speed of motion within a continuous time window is not greater than the stopping speed threshold And the duration is not less than the downtime duration. When the average speed of continuous movement is below the threshold, it indicates that the paver has entered the maintenance shutdown phase.

[0091] In terms of smoke and dust detection, this embodiment sets a first smoke and dust threshold and a second smoke and dust threshold, wherein the first smoke and dust threshold is greater than the second smoke and dust threshold, which is used to form a graded switching; when the smoke and dust obstruction degree is between the two thresholds, it enters the smoke and dust obstruction state and collects infrared thermal images.

[0092] When the smoke and dust obscuration increases further and exceeds the first smoke and dust threshold, the sampling frequency or integration time of the infrared thermal imaging camera is increased, and the infrared image gain is adjusted before infrared thermal image processing is performed; infrared acquisition is performed when the smoke and dust obscuration is greater than the second smoke and dust threshold but not greater than the first smoke and dust threshold, and the infrared acquisition capability is enhanced when the smoke and dust obscuration is greater than the first smoke and dust threshold to prevent the effective information of the thermal image from being weakened.

[0093] By using dual thresholds and hierarchical processing, the frequency of channel switching and image quality instability are reduced. The acquisition status is distinguished based on vibration stability judgment, shutdown duration judgment and smoke and dust classification threshold, preventing unsuitable images from entering the back-end processing flow.

[0094] In the application embodiment, the field control unit can send the filtered vibration data, vehicle speed data, and smoke and dust obstruction degree to the judgment module in real time; when a shutdown state is detected, the system performs shutdown maintenance processing; when a smoke and dust obstruction state is detected, the system switches to the infrared acquisition and processing process, thereby ensuring that the corresponding quality results can be output under different working conditions;

[0095] Furthermore, the vibration stable state, shutdown state, and smoke and dust obstruction state are updated by the same state machine in a polling cycle according to the sampling window. When any condition exceeds the limit, it immediately reverts to the corresponding unstable state. Only when the smoke and dust obstruction degree drops from greater than the first smoke and dust threshold to no greater than the second smoke and dust threshold, and the condition is met continuously within the preset confirmation time window, is it allowed to switch from severe obstruction back to normal acquisition, so as to avoid state jitter near the threshold.

[0096] In one embodiment, under normal acquisition conditions, the control unit set on the paving equipment sends a hardware trigger pulse to the visible light industrial camera through a field programmable gate array controller, causing the visible light industrial camera to use global shutter exposure;

[0097] The field-programmable gate array (FPGA) controller transmits the raw images output by the visible light industrial camera to the computing unit connected to the FPGA controller via a direct memory access channel.

[0098] The computing unit sequentially performs black level correction, noise reduction, distortion correction, paving region segmentation, and extraction of paving texture, surface defects, and edge missing material features from the original image, and generates a visible light quality score based on the extracted features.

[0099] In the case of smoke and dust obscuring the image, the field programmable gate array controller blocks the direct memory access transmission of the image from the visible light industrial camera to the computing unit, and the infrared thermal imaging camera acquires the infrared thermal image.

[0100] The infrared thermal image is corrected for emissivity, ambient temperature, distance, and non-uniformity. The regional reference temperature is determined based on the average temperature of the paving area or historical temperature data. Pixels whose difference from the regional reference temperature exceeds a preset temperature difference threshold are identified as temperature abnormal pixels. Thermal anomalies are determined based on the proportion of temperature abnormal pixels in the paving area.

[0101] When the number of effective pixels in the current paving area does not meet the baseline calculation conditions, the average temperature of the N most recent qualified infrared images is taken as historical temperature data, and the available value of the two is taken as the regional baseline temperature, where N is the preset window number.

[0102] When the paving machine is stopped, the control unit shuts down the main link for visible light acquisition. When entering the paving machine stop state, the infrared thermal imaging camera is triggered to perform non-uniformity correction, the cache queue in the computing unit is cleared, and the infrared thermal image processing frequency is reduced from the first processing frequency to the second processing frequency, wherein the second processing frequency is less than the first processing frequency and not lower than the preset minimum processing frequency.

[0103] Simultaneously activate the lens heating device connected to the control unit and located at the camera lens;

[0104] Visible light quality score Infrared quality score and motion stability coefficient Normalized to the preset upper and lower limits And generate a comprehensive quality score according to the following formula:

[0105]

[0106] in, For the overall quality score, , and For the preset weighting coefficients, and ;

[0107] When visible light images are not acquired or their quality is below a preset effective threshold under smoke and dust obstruction, the following conditions are met: If a score is invalid, it is removed and its corresponding weight is reset to zero. Then, the weights of the remaining valid scores are re-normalized proportionally before calculating the overall quality score.

[0108] The overall quality score is output according to a fixed time window and compared with the preset quality threshold for the corresponding working condition to output a qualified result or an abnormal alarm result.

[0109] This embodiment controls the acquisition link, processing link, and maintenance link independently. Under normal acquisition conditions, the field programmable gate array controller sends a hardware trigger pulse to the visible light industrial camera, causing the camera to use global shutter exposure to avoid the deformation of the travel direction caused by the rolling mode.

[0110] After the original image is transmitted to the computing unit via the direct memory access channel, black level correction, denoising, distortion correction, tiling region segmentation, and texture and defect feature extraction are performed in sequence to finally form a visible light quality score.

[0111] After restoring the basic imaging quality through image processing, the paving texture and edge missing features are extracted; the image transmission and subsequent analysis are separated through hardware triggering and direct memory access channel transmission to reduce redundant processing;

[0112] When the smoke and dust obscure the image, the field programmable gate array controller blocks the direct memory access transmission of the image from the visible light industrial camera to the computing unit, causing the visible light industrial camera to enter low-power standby mode, and the infrared thermal imaging camera takes over the acquisition.

[0113] When processing infrared thermal images, emissivity, ambient temperature, distance, and non-uniformity are first corrected. Then, the regional reference temperature is determined based on the average temperature of the paved area or historical temperature data. Pixels whose difference from the regional reference temperature exceeds a preset temperature difference threshold are defined as temperature abnormal pixels, and thermal anomalies are determined based on their proportion in the paved area.

[0114] Meanwhile, the infrared quality score is formed by standardizing and weighting four sub-items: regional temperature deviation, temperature dispersion, temperature gradient continuity, and thermal anomaly proportion, to ensure that thermal anomaly identification does not rely solely on a single pixel. When the visible light channel is blocked, the thermal image can reflect the temperature distribution differences in the paving area, and the proportion of temperature anomaly pixels characterizes local thermal unevenness or segregation tendency.

[0115] In the shutdown state, the control unit shuts down the visible light acquisition main link and triggers the infrared thermal imaging camera to perform non-uniformity correction when entering the shutdown state. At the same time, it clears the cache queue in the computing unit and reduces the infrared thermal image processing frequency from the first processing frequency to the second processing frequency, and the second processing frequency is not lower than the preset minimum processing frequency.

[0116] Simultaneously, the lens heating device located at the camera lens is activated to defog; this reduces resource consumption during downtime and prevents lens fogging from affecting subsequent data acquisition and recovery; in terms of quality fusion, this embodiment uses visible light quality scores. Infrared quality score and motion stability coefficient Normalization to Then, a comprehensive quality score is generated according to the weighting coefficients. ;

[0117] The weighting coefficient is used to balance the contributions of visible light, infrared light, and motion status to the quality results under different operating conditions; when the image is obscured by smoke or dust and the visible light image is not acquired or its quality is below a preset effective threshold, the weighting coefficient will be adjusted accordingly. The score is deemed invalid and removed. Its corresponding weight is then reset to zero, and the weights of the remaining valid scores are recalculated proportionally. To avoid invalid visible light results lowering the overall score;

[0118] The comprehensive quality score is output according to a fixed time window and compared with the preset quality threshold for the corresponding working condition, outputting a qualified result or an abnormal alarm result. This embodiment is based on hardware triggering, direct memory access diversion, infrared takeover and shutdown maintenance mechanism to realize image processing and result output under different working conditions, avoiding the impact of mixed acquisition links, cache accumulation and lens fogging on the detection results.

[0119] In an application embodiment, the system can be deployed between the paver control unit and the edge computing unit: the control unit controls the camera to collect and transmit data based on the status determination result, the computing unit performs quality analysis on the input image and outputs a comprehensive quality score, and the score result is then used as the basis for on-site quality feedback to indicate whether the current paving quality meets the requirements;

[0120] The buffer queue adopts a circular buffer structure corresponding to the length of the time window. When entering the system after a shutdown, the write pointer is frozen and unprocessed frames are marked as invalid in a first-in-first-out order and then cleared all at once. After the acquisition is resumed, the direct memory access channel is reopened. When the infrared processing frequency is switched from the first processing frequency to the second processing frequency, the control unit extends the frame interval in a step-by-step manner to avoid the processing thread and the acquisition thread from competing for bus bandwidth at the same time.

[0121] When outputting the overall quality score, the system only takes valid samples with consistent states and continuous timestamps within the current fixed time window for calculation. If there are missing frames or state reversals within the window, the window will be delayed by one sampling period before outputting.

[0122] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for inspecting the quality of asphalt paving based on industrial vision, characterized in that, include: S1. Visible light images, infrared thermal images, three-axis acceleration, scattered echo intensity, and vehicle speed are acquired through a visible light industrial camera, an infrared thermal imaging camera, a six-axis inertial measurement unit mounted on the camera bracket, a side-axis laser scattering sensor, and a vehicle controller LAN bus, respectively; timestamps are added by a unified sampling clock, and each data is synchronized within the same time window according to the timestamp; S2. Determine the dust obstruction degree based on the scattered echo intensity, determine the vibration stability state based on the triaxial acceleration, determine the average speed and paving machine stop state based on the vehicle speed, and obtain a preset dust obstruction degree threshold, wherein the dust obstruction degree threshold is an obstruction degree threshold calibrated based on the unobstructed reference echo intensity and the dark field echo baseline value. S3. First determine the paving machine stop status; in the non-paving machine stop status, when the dust obscuration exceeds the dust obscuration threshold or the vibration is unstable, it is determined to be a dust obscuration status, and other situations are determined to be normal collection status. S4. Process visible light images under normal acquisition conditions, process infrared thermal images under smoke and dust obstruction conditions, and perform shutdown maintenance under paving shutdown conditions. S5. Generate a visible light quality score based on the paving texture, surface defects, and edge missing material features extracted from the visible light image. Generate an infrared quality score based on the mean temperature, temperature variance, temperature gradient, and local thermal anomaly features extracted from the infrared thermal image. Determine the motion stability coefficient based on the triaxial acceleration. Generate a comprehensive quality score according to the preset weights. Compare the comprehensive quality score with the preset quality threshold for the corresponding working condition. Output a qualified result when the comprehensive quality score is greater than or equal to the preset quality threshold; otherwise, output an abnormal alarm result.

2. The asphalt paving quality inspection method based on industrial vision according to claim 1, characterized in that, The triaxial acceleration output by the six-axis inertial measurement unit is transformed into the device coordinate system and then... Processed on a unit-by-unit basis.

3. The asphalt paving quality inspection method based on industrial vision according to claim 1, characterized in that, The scattered echo intensity within the same time window is averaged, and the smoke and dust obstruction degree is determined according to the following formula: ； in, The smoke and dust shielding degree is mentioned above. The average scattered echo intensity within the time window, The background echo basis value under dark field conditions. The reference echo intensity is under unobstructed calibration conditions, and The and The equipment is calibrated after installation and updated according to the preset calibration cycle.

4. The asphalt paving quality inspection method based on industrial vision according to claim 1, characterized in that, The triaxial acceleration is filtered, and the current sampling time acceleration vector after filtering is... and according to and Determine the vibration stability state; When the conditions are met continuously within the preset stable time window and When this condition is met, it is determined to be a stable vibration state; among which, The acceleration vector at the current sampling moment. The acceleration vector at the previous sampling time. To preset the acceleration magnitude threshold, This is a preset threshold for the change in acceleration.

5. The asphalt paving quality inspection method based on industrial vision according to claim 1, characterized in that, Calculate the average speed of movement based on the vehicle controller area network bus speed within a preset time window: ； in, The average velocity of the motion, The number of valid sampling points within the time window; For the first Vehicle speed at each sampling time; When the average speed of motion within a continuous time window is not greater than the stopping speed threshold And the duration is not less than the downtime duration. At that time, it was determined that the paving machine was in a stopped state.

6. The asphalt paving quality inspection method based on industrial vision according to claim 1, characterized in that, A first smoke threshold is set, and the smoke obstruction threshold is used as a second smoke threshold, wherein the first smoke threshold is greater than the second smoke threshold; When the smoke and dust obstruction degree is greater than the second smoke and dust threshold and not greater than the first smoke and dust threshold, the smoke and dust obstruction state is entered and infrared thermal images are collected. When the smoke and dust obstruction exceeds the first smoke and dust threshold, increase the sampling frequency or integration time of the infrared thermal imaging camera, and adjust the infrared image gain before performing infrared thermal image processing. The first and second smoke thresholds are calibrated based on the unobstructed reference echo intensity, the dark field echo baseline value, and historical smoke samples, and are updated according to a preset calibration cycle.

7. The asphalt paving quality inspection method based on industrial vision according to claim 1, characterized in that, Under normal acquisition conditions, the control unit set on the paving equipment sends a hardware trigger pulse to the visible light industrial camera through the field programmable gate array controller, so that the visible light industrial camera uses global shutter exposure. The field-programmable gate array controller transmits the raw images output by the visible light industrial camera to the computing unit connected to the field-programmable gate array controller via a direct memory access channel. The computing unit sequentially performs black level correction, noise reduction, distortion correction, paving region segmentation, and extraction of paving texture, surface defects, and edge missing material features from the original image, and generates a visible light quality score based on the extracted features.

8. The asphalt paving quality inspection method based on industrial vision according to claim 7, characterized in that, In the case of smoke and dust obscuring the image, the field-programmable gate array controller blocks direct memory access transmission of the image from the visible light industrial camera to the computing unit, and the infrared thermal imaging camera acquires the infrared thermal image. The infrared thermal image is corrected for emissivity, ambient temperature, distance, and non-uniformity. The regional reference temperature is determined based on the average temperature of the paving area or historical temperature data. Pixels whose difference from the regional reference temperature exceeds a preset temperature difference threshold are identified as temperature abnormal pixels. Thermal anomalies are determined based on the proportion of temperature abnormal pixels in the paving area.

9. The asphalt paving quality inspection method based on industrial vision according to claim 7, characterized in that, When the paving machine is stopped, the control unit shuts down the main link for visible light acquisition. When entering the paving machine stop state, the infrared thermal imaging camera is triggered to perform non-uniformity correction, the cache queue in the computing unit is cleared, and the infrared thermal image processing frequency is reduced from the first processing frequency to the second processing frequency, wherein the second processing frequency is less than the first processing frequency and not lower than the preset minimum processing frequency. Simultaneously, the lens heating device, which is connected to the control unit and located at the camera lens, is activated.

10. The asphalt paving quality inspection method based on industrial vision according to claim 1, characterized in that, Visible light quality score Infrared quality score and motion stability coefficient Normalized to the preset upper and lower limits And generate a comprehensive quality score according to the following formula: ； in, For the overall quality score, , and For the preset weighting coefficients, and ; When visible light images are not acquired or their quality is below a preset effective threshold under smoke and dust obstruction, the following conditions are met: If a score is invalid, it should be removed and the weights of the remaining valid scores should be re-normalized proportionally. The overall quality score is output according to a fixed time window and compared with the preset quality threshold for the corresponding working condition to output a qualified result or an abnormal alarm result.

Images