Multifunctional sensing and local control intelligent safety guardrail system

Abstract

The invention belongs to the technical field of intelligent safety guardrails, and particularly relates to a multifunctional sensing and local control intelligent safety guardrail system, which comprises a data acquisition module, a wavelet transformation module and an early warning module, and is used for performing Morlet continuous wavelet transformation on a vertical real-time depth map and a vertical reference depth map output by the data acquisition module to obtain a vertical real-time depth map and a vertical reference depth map; respectively generating a plurality of phase spectrums of the vertical real-time depth map and the vertical reference depth map; obtaining a continuous phase difference distribution map for the phase difference between the two complex phase spectrums; the linear relation between the phase difference and the displacement is used for calculating the horizontal displacement, the Morlet wavelet transform is used for directly calculating the point cloud displacement through the frequency domain phase difference, the process that an ICP algorithm is needed for multiple iterations in traditional electric cloud calculation is avoided, the geometrical characteristics of the infrastructure are converted into the physical scale of wavelet analysis, time domain localization analysis is achieved, and the calculation efficiency is improved. Displacement detection is converted into a phase matching problem of a signal and a wavelet, and the precision is improved while the speed is increased.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent safety guardrails, and particularly relates to a multifunctional intelligent safety guardrail system with sensing and local control. BACKGROUND

[0002] In the field of intelligent operation and maintenance of traffic infrastructure, with the rapid development of infrastructure construction, many bridges are provided with guardrails, especially some sea-crossing bridges, highway bridges or railway bridges. The installed safety guardrails play a key role in reducing the loss caused by road safety accidents, especially the safety guardrails installed on long sea-crossing bridges, which need to be continuously monitored in working state to ensure that they have certain working capacity, and the rapid and accurate positioning of guardrail accidents is crucial to improving road safety. The current mainstream method relies on collecting scene image data by laser radar or depth camera and extracting point cloud information from the scene image data for guardrail space positioning. The core of this technical path is to align the coordinates of the real-time collected guardrail point cloud and the preset reference point cloud, and the iterative closest point (ICP) is usually used to realize pose solving. However, the point cloud matching process has inherent defects: due to environmental interference in the monitoring site, sensor noise and local deformation of the guardrail, there are often a large number of outliers and non-rigid deformation in real-time point cloud, which leads to the fact that a single iteration cannot achieve accurate registration. In order to meet the monitoring accuracy requirements, the system must perform multiple rounds of iterative optimization, including repeatedly searching for the corresponding relationship of the nearest points, calculating the rigid transformation matrix, evaluating the root mean square error (RMSE) and correcting the pose parameters, which consumes computing resources and slows down the system's calculation speed. SUMMARY

[0003] To solve the above problems in the prior art, the application provides a multifunctional intelligent safety guardrail system with sensing and local control, which solves the problem of large amount of calculation caused by ICP multiple iterations when calculating the guardrail displacement using point cloud data.

[0004] The purpose of the application can be achieved by the following technical scheme: a multifunctional intelligent safety guardrail system with sensing and local control, characterized by comprising: a data acquisition module for acquiring a real-time surface image of a guardrail and solving the appearance surface point cloud of the guardrail from the real-time surface image, and projecting the appearance surface point cloud to generate a vertical real-time depth map; acquiring a preset reference point cloud of a preset reference guardrail and projecting it to generate a vertical reference depth map, wherein the vertical real-time depth map and the vertical reference depth map are both two-dimensional matrices; The wavelet transform module comprises a wavelet transform processing unit, a phase difference calculation unit and a displacement conversion unit; the wavelet transform processing unit is configured to perform Morlet continuous wavelet transform on the vertical real-time depth map and the vertical reference depth map output by the data acquisition module, and generate complex phase spectra of the vertical real-time depth map and the vertical reference depth map, respectively; the phase difference calculation unit is configured to calculate phase differences according to the real-time phase spectra corresponding to the real-time depth map and the reference depth map, respectively, and obtain a continuous phase difference distribution map; and the displacement conversion unit is configured to calculate the horizontal displacement by using the linear relationship between the phase difference and the displacement. The early warning module is configured to trigger a guardrail movement alarm and upload the guardrail position when the horizontal displacement is evaluated to exceed a set displacement threshold.

[0005] Preferably, the wavelet transform processing unit performs Morlet continuous wavelet transform on the vertical real-time depth map and the vertical reference depth map, wherein the calculation formula of the Morlet wavelet basis function is as follows: ; t is time (spatial position), is a center frequency, is a normalization factor for ensuring consistency of oscillation characteristics at different scales, is a Gaussian envelope function, is a complex exponential oscillation function, and s is a scale parameter, wherein L is the average distance between guardrail posts calculated by the reference depth map.

[0006] Preferably, the calculation formula for calculating the phase difference by the phase difference calculation unit according to the two phase spectra is as follows: wherein is the real-time phase spectrum, is the reference phase spectrum.

[0007] Preferably, the phase difference calculation unit further comprises judging whether the phase difference has phase wrapping and performing phase unwrapping on the phase difference having phase wrapping, comprising: If: , then If: , then .

[0008] Preferably, the wavelet transform module further comprises a displacement optimization unit, which optimizes the generated displacement by using a median filtering method, and the optimization formula is as follows: ; wherein, is an original displacement difference, is an optimized displacement difference, is a window size, and x is a translation parameter.

[0009] The real-time surface image is a disparity map, and the data acquisition module obtaining the disparity map comprises: using a depth camera under two perspectives to respectively acquire a real-time first image and a real-time second image of the guardrail; using a stereo vision matching algorithm to perform extreme constraint on the real-time first image and the real-time second image, and calculating a disparity map of the real-time first image and the real-time second image.

[0010] Preferably, the data acquisition module further comprises an image segmentation unit, the image segmentation unit comprising: extracting real-time three-dimensional contour data and reference three-dimensional contour data of the guardrail respectively according to the appearance surface point cloud and the preset reference point cloud, calculating a deformation variable difference value between the real-time three-dimensional contour data and the reference three-dimensional contour data using a U-Net network, and transmitting the deformation variable difference value to the early warning module; the early warning module is configured to trigger a guardrail deformation alarm and upload a guardrail position when the evaluation deformation variable difference exceeds a set deformation variable threshold.

[0011] Preferably, the image segmentation unit comprises three-dimensional contour feature expansion processing of the three-dimensional contour data and the reference three-dimensional contour data, comprising: the SPP structure is used to expand the guardrail contour features of the three-dimensional contour data and the reference three-dimensional contour data, and a channel attention mechanism is used to extract the real-time three-dimensional contour data and the reference three-dimensional contour data.

[0012] Preferably, it further comprises a local control module, which processes point cloud data and wavelet transform results in real time based on an edge computing chip.

[0013] The present application has the following advantages: The present application directly calculates the point cloud displacement by the frequency domain phase difference through Morlet wavelet transform, avoids the process of multiple iterations of the ICP algorithm required in traditional point cloud calculation, and directly uses the linear relationship between the phase difference and the spatial displacement of the Morlet wavelet transform to skip the multiple ICP iteration searches required for point cloud calculation, thereby providing system calculation amount, converting the point cloud matching problem in the spatial domain into a phase difference detection problem in the frequency domain, directly associating the physical characteristics (post spacing) of the guardrail through the wavelet scale, realizing displacement calculation with clear physical meaning. At the same time, the geometric characteristics of the infrastructure are converted into the physical scale of wavelet analysis, realizing time domain localization analysis, converting displacement detection into a phase matching problem between signals and wavelets, thereby improving the speed and accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0014] For the convenience of those skilled in the art to understand, the present application is further described below in conjunction with the drawings.

[0015] Figure 1 The calculation process block diagram of the wavelet transform module of the present application. DETAILED DESCRIPTION

[0016] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined application purposes, the specific embodiments, structures, features and effects according to the present application are described in detail below in conjunction with the drawings and preferred embodiments.

[0017] Please refer to Figure 1 The embodiment provides a multifunctional intelligent safety guardrail system with perception and local control, comprising: A data acquisition module is configured to acquire a real-time surface image of a guardrail and calculate an appearance surface point cloud of the guardrail from the real-time surface image, and project the appearance surface point cloud to generate a vertical real-time depth map. A real-time data acquisition unit is configured to acquire three-dimensional point cloud data of a surface of a guardrail in real time through a high-precision stereo vision camera or a laser radar arranged on a top or side of the guardrail, and synchronously acquire auxiliary data such as ambient light intensity and temperature and humidity around the guardrail, for subsequent environmental compensation of point cloud calculation.

[0018] A point cloud calculation and preprocessing unit is configured to perform denoising (adopting a statistical filtering algorithm to remove outliers, and a filtering radius is less than or equal to 0.05 m), preliminary registration (aligning based on an ICP algorithm and a fixed identification point of the guardrail), and downsampling processing on the acquired original point cloud data, to retain key structure points, and to calculate an appearance surface point cloud of the guardrail, and a point cloud coordinate system is a local Cartesian coordinate system.

[0019] A vertical depth map generation unit is configured to project the preprocessed appearance surface point cloud along a vertical direction, i.e., a height direction of the guardrail, to generate a two-dimensional vertical real-time depth map, and a depth map pixel value corresponds to an actual distance, to ensure that a main stress area of the guardrail is covered.

[0020] A reference data storage unit is configured to prestore a preset reference point cloud of the guardrail after installation (acquired in a no-load and no-deformation state after installation of the guardrail), to project the same after a same preprocessing procedure to generate a vertical reference depth map, as a reference for subsequent deformation comparison, and the reference point cloud supports periodic updating, and cumulative errors such as environmental temperature and settlement are eliminated through a self-calibration mechanism.

[0021] A preset reference point cloud of a preset reference guardrail is acquired and projected to generate a vertical reference depth map, and the vertical real-time depth map and the vertical reference depth map are both two-dimensional matrices. A wavelet transform module comprises a wavelet transform processing unit, a phase difference calculation unit and a displacement amount conversion unit, and specifically: a wavelet transform processing unit for performing Morlet continuous wavelet transform on the vertical real-time depth map and the vertical reference depth map output by the data acquisition module, respectively, with the transform being performed in the horizontal direction (x-axis), S1: Obtain the center frequency , and the average distance L between the guardrail posts obtained from the reference depth map obtained after the completion of guardrail installation through post position detection, used to calculate the scale parameter s, which controls the width of the Gaussian function: , is a pre-set quantity.

[0022] S2: Morlet wavelet basis function is: ; where t is the time (spatial position), is the center frequency, usually taken as 6, is the normalization factor, used to ensure consistency in the oscillation characteristics at different scales, is the Gaussian envelope function, which makes the energy of the wavelet concentrated near t=0 in the spatial (or time) domain, and rapidly decays as increases. The larger the scale s, the wider the Gaussian function, the larger the support interval of the wavelet, the higher the frequency resolution, and the lower the spatial resolution; is a complex exponential oscillation function, which is Euler expanded as: where the real part is the cosine wave, used to capture symmetric features, and the imaginary part is the sine wave, used to capture asymmetric features, is used to represent the adjustment of the frequency of oscillation when the scale s changes, so that the oscillation period of the wavelet is proportional to the scale s.

[0023] S3: Perform Morlet continuous wavelet transform on each row of the real-time depth map and the reference depth map, respectively; Because the vertical real-time depth map of the guardrail has periodic characteristics on each row, and processing the rows independently can be accelerated in parallel, improving the running speed of the system, where each sub-step includes: A: Create and initialize two complex matrices and , respectively, for storing the wavelet transform results of the vertical real-time depth map and the vertical reference depth map, obtaining the real-time depth map wavelet coefficient matrix and the reference depth map wavelet coefficient matrix, and the matrix size of the two complex matrices is the same as the size of the two depth maps, and the initial values of the two matrices are set to 0; Iterate through each row of the vertical real-time depth map and the vertical reference depth map, and extract the row data respectively; B: Convolution calculation on real-time depth map and base row, wherein, the continuous wavelet transform (CWT) is performed on each row data of the base depth quasi-depth map to obtain wavelet transform coefficients, and the wavelet transform calculation formula of the real-time depth map is as follows: wherein, s is a scale parameter, x is a translation parameter, represents the position of the wavelet function on the x axis, and on the row of the depth map, from 0 to W-1, is the complex conjugate of the wavelet kernel function, is the row signal of the real-time depth map, is a normalization factor; the wavelet transform calculation result of each row is respectively stored in the corresponding row of the preset complex matrix; D: Output two complex matrices and ; After wavelet transform is performed on each depth map (real-time depth map and base depth map), the complex wavelet coefficient matrix is obtained. The phase spectrum (i.e. phase angle) is calculated, wherein the real-time phase angle calculation formula is as follows: ; the base phase angle calculation formula is the same; The phase difference calculation unit calculates the phase difference of the real-time phase spectrum corresponding to the real-time depth map and the base depth map respectively, to obtain a continuous phase difference distribution map, which specifically includes: For each position x, the difference between the real-time phase spectrum corresponding to the real-time depth map and the base phase spectrum is calculated : wherein, is the real-time phase spectrum, is the base phase spectrum, in the calculation process, in order to avoid 2 jump, phase unwrapping needs to be performed when the set condition is met: If: , then If: , then 2π phase correction is used to avoid winding error, and further to ensure the continuity of displacement calculation.

[0024] The displacement amount conversion unit calculates the horizontal displacement amount according to the linear relationship between the phase difference and the displacement; The output is the horizontal displacement amount of each horizontal position x, and the precise geographic coordinates of the guardrail with displacement are output by combining the displacement field and the boundary point cloud position. If the horizontal displacement is greater than 0, it is considered that the guardrail position has moved, and the position movement of the guardrail will cause the original guardrail to loosen, thereby affecting the stability of the guardrail. Combined with the position information of the boundary point cloud, it can be located which part of the guardrail has moved, and the coordinates are output.

[0025] For vertical real-time depth map and vertical reference depth map, the transformation process follows the unified mathematical framework of Morlet continuous wavelet transform: that is, for each pixel point in the image, taking Morlet wavelet basis function as the core, convolution calculation is carried out under the preset scale range and displacement step, and finally the corresponding complex phase spectrum is generated.

[0026] The wavelet transform calculation of the vertical real-time depth map and the vertical reference depth map is only different in the input of the original data (the depth information of the real-time collected guardrail and the preset reference guardrail depth information, respectively), but the wavelet basis function parameters (such as center frequency, bandwidth, etc.), scale space range, and translation step, which are the core calculation parameters, remain the same. The unified calculation formula can ensure that the two types of depth maps generate phase spectra under the same feature extraction rules, lay a foundation for accurate pixel-by-pixel phase difference calculation, and avoid system errors caused by inconsistent transformation rules.

[0027] The early warning module is used to trigger the guardrail deformation alarm and locate the uploaded guardrail deformation position when the evaluation horizontal displacement exceeds the set threshold. The early warning module includes a displacement evaluation unit and an alarm triggering unit: The displacement evaluation unit compares the horizontal displacement output by the wavelet transform module with the preset threshold. The threshold is set according to the guardrail material (such as steel, concrete), installation standard (such as highway guardrail anti-collision grade A / B), and use scene (guardrail on a cross-sea bridge). For example, the threshold of a steel guardrail is set to ±5mm, and the threshold of a concrete guardrail is set to ±8mm. The alarm triggering unit triggers a multi-level remote alarm when the evaluation finds that the horizontal displacement of any region exceeds the threshold: the alarm information is pushed to the management platform through the NB-IoT or LoRa local communication module, and the transmitted alarm information includes the displacement value, the deformation region coordinates, and the real-time depth map comparison; and based on the mapping relationship between the point cloud coordinates and the guardrail post number (the pre-stored post number-coordinate correspondence table), the deformation position is accurately located to the specific post number interval, and is marked on the electronic map, supporting the management personnel to quickly trace the source.

[0028] The system also includes a local control module based on an edge computing chip (such as ARM Cortex-A53) to process point cloud data and wavelet transform results in real time, reduce data transmission bandwidth requirements, support local threshold adjustment, data acquisition frequency configuration, and device self-checking (periodically detect the working state of the camera or radar, and trigger device fault alarm when abnormal); The traditional ICP has an exponential growth in the calculation amount with the monitoring distance in the long-distance scene such as the cross-sea bridge. The Morlet wavelet transform is used to directly calculate the point cloud displacement amount through the phase difference in the frequency domain in the application, the iterative process of the ICP algorithm is avoided, and the calculation complexity of the wavelet transform is almost linear growth. The Morlet wavelet transform directly uses the linear relationship between the phase difference and the spatial displacement, skips the iterative search of the ICP required by the point cloud calculation, provides the system calculation amount, converts the point cloud matching problem in the spatial domain into the phase difference detection problem in the frequency domain, directly correlates the physical characteristics (the column spacing) of the guardrail through the wavelet scale, and realizes the displacement calculation with clear physical meaning. Meanwhile, the geometric characteristics of the infrastructure are converted into the physical scale of the wavelet analysis, the time domain localization analysis is realized, the displacement detection is converted into the phase matching problem of the signal and the wavelet, the speed is improved, and the accuracy is also improved.

[0029] Due to noise and local interference, the calculated displacement amount may have some outliers, therefore, in one embodiment, the wavelet transform module further includes a displacement optimization unit, the displacement optimization unit optimizes the generated displacement amount by using the median filtering method, and the optimization formula is: ; wherein, is the original displacement difference, is the optimized displacement difference, k is the window size, and x is the translation parameter. The filtering window size is related to the column spacing (for example, the window width=1.5 times the column spacing corresponding to the number of pixels).

[0030] Due to the influence of light changes (such as strong light and shadow), weather interference (such as light rain and dust), or the existence of occlusion in one direction angle, the features (such as color and texture) of the RGB image under a single view angle are easy to be distorted, and then the extraction of the subsequent guardrail surface point cloud data is affected, therefore, in one embodiment, the real-time surface image is a disparity map, and the data acquisition module obtains the disparity map by using two views to use a depth camera to acquire a real-time first image and a real-time second image of the guardrail; using a stereo vision matching algorithm to perform extreme constraint on the real-time first image and the real-time second image, and calculating the disparity map of the real-time first image and the real-time second image, specifically: Step one: two depth cameras with consistent parameters are deployed at the guardrail monitoring point, the baseline distance is adjusted according to the monitoring distance, the left camera acquires a real-time first image (left image), the right camera acquires a real-time second image (right image), the synchronous acquisition frame rate is uniform, and the resolution is uniform; the IMU module is arranged in the two cameras, and the attitude parameters during shooting are recorded in real time; Step two: the acquired image is preprocessed, including distortion correction, graying and noise reduction: Distortion correction: Based on the camera intrinsic parameter matrix (focal length, principal point coordinates) and the set distortion coefficients, such as radial distortion k1-k3 and tangential distortion p1-p2, distortion correction is performed on the left and right images to eliminate pixel shift caused by lens optical distortion. The formula is: Corrected pixel coordinates = intrinsic parameter matrix × inverse operation of distortion model (original pixel coordinates). Grayscale conversion and noise reduction: To reduce the amount of data, the RGB image is converted to a grayscale image. Gaussian filtering (σ=1.0) and bilateral filtering (spatial sigma=2.0, grayscale sigma=50) are used to remove image noise while preserving key features such as the guardrail edges. Step 3: Polar constraint correction: Epipolar geometry calculation: The extrinsic parameter matrices (rotation matrix R, translation vector T) of the two cameras are obtained through stereo calibration. Based on the extrinsic parameters, the fundamental matrix and the essential matrix are calculated to determine the epipolar geometric relationship between the left and right images. The epipolar constraint ensures that the corresponding point of any pixel in the left image must be located on its epipolar line in the right image, simplifying the two-dimensional matching problem into a one-dimensional search.

[0031] Epipolar correction: The left and right images are projected onto the same epipolar plane using a planar projection method, ensuring that the epipolar lines of the corrected left and right images are completely horizontal and aligned (i.e., the row coordinates of corresponding points are consistent). The corrected left image pixels... The corresponding point in the right figure must be located at This simplifies the search range for subsequent matching (only the horizontal direction needs to be searched). The correction process is achieved by calculating correction matrices H1 and H2: corrected left image = H1 × original left image, corrected right image = H2 × original right image; Step 4: Stereo vision matching and disparity calculation Matching cost calculation: The matching cost is calculated using either the SAD (Sum of Absolute Differences) algorithm or the SGBM (Semi-Global Block Matching) algorithm. Taking SGBM as an example: For each pixel in the left image... Within the search range corresponding to the epipolar line (y-row) in the right figure, calculate the block matching cost under different disparities d to obtain the initial cost volume. .

[0032] Cost aggregation and disparity selection: Cost aggregation is performed in the horizontal and vertical directions using dynamic programming, considering the disparity consistency of neighboring pixels and suppressing noise interference. The aggregated cost volume is then processed using a Winner-Takes-All (WTA) strategy, selecting the disparity d with the minimum cost as the initial disparity value for that pixel, thus generating an initial disparity map D. .

[0033] Epipolar constraint verification: Perform epipolar constraint verification on the initial disparity map and calculate the consistency of left and right disparities (the disparity d in the left image and the disparity of corresponding points in the right image). The error matching points that do not satisfy the constraint are rejected, and are marked as invalid disparity (such as d=-1), and a final three-dimensional matrix disparity map is output .

[0034] First, two different visual images of the guardrail are obtained, and a disparity map of the first image and the second image is obtained by using the difference between the first image and the second image. Through the disparity map, the three-dimensional geometric information of the guardrail is converted into point cloud data according to the disparity map, and then the guardrail boundary is segmented from the point cloud data, so as to accurately position the guardrail boundary, and further improve the positioning accuracy of the guardrail boundary based on the vertical real-time depth map obtained from the point cloud data.

[0035] When the guardrail is subjected to impact or other damage, a certain deformation amount will occur. According to the detection of the deformation amount, it can be judged whether the guardrail is in a dangerous state. In the case of obvious deformation, the state of the guardrail can be quickly judged. Therefore, in an embodiment, the data acquisition module further includes an image segmentation unit, which includes a first image segmentation unit configured to extract real-time three-dimensional contour data and baseline three-dimensional contour data of the guardrail from the appearance surface point cloud and the preset baseline point cloud respectively, The three-dimensional contour data and the baseline three-dimensional contour data are subjected to three-dimensional contour feature expansion processing, including: The SPP structure is used to expand the guardrail contour features of the three-dimensional contour data and the baseline three-dimensional contour data, and the channel attention mechanism is used to extract the real-time three-dimensional contour data and the baseline three-dimensional contour data, including: The three-dimensional space pyramid pooling (3D SPP) structure is used to expand the multi-scale features of the preprocessed three-dimensional contour data, and capture the deformation features of different scales: SPP structure design: an SPP module is inserted at the output end of the 3D U-Net encoder, which contains four parallel pooling branches corresponding to different scale feature expansions: Branch one: 1x1x1 pooling (preserve original feature scale), capture small deformation (such as 0.5-2mm local concave); Branch two: 2x2x2 pooling (step 2), expand medium scale features (such as 2-5mm edge offset); Branch three: 4x4x4 pooling (step 4), expand larger scale features (such as 5-10mm overall tilt); Branch four: 8x8x8 pooling (step 8), expand global structure features (such as 10mm or more bending deformation).

[0036] Feature expansion and splicing: each pooling branch performs pooling on the input feature map (dimension CxDxHxW, C is the number of channels, D / H / W is the three-dimensional size), and then converts it to a fixed-dimensional feature vector (such as 256 dimensions) through a fully connected layer; the feature vectors of the four branches are spliced to obtain an expanded multi-scale feature vector (total dimension 1024 dimensions).

[0037] After the SPP feature expansion, a channel attention mechanism (such as the SE module) is introduced for contour feature extraction. Then the real-time three-dimensional contour data and the reference three-dimensional contour data are used to calculate the deformation difference between the real-time three-dimensional contour data and the reference three-dimensional contour data using the U-Net network. The network adopts an "encoder-decoder" architecture. The encoder extracts local features of the contour (such as edge angle, curvature change) through a 3D convolution layer (kernel_size=3x3x3). The decoder fuses multi-scale features through a skip connection and outputs the corresponding guardrail three-dimensional geometric model of the real-time contour relative to the reference contour. The volume of the two is calculated according to the three-dimensional geometric figure. The volume difference between the real-time contour and the reference contour is calculated according to the volume of the two. The volume difference is set as the deformation variable. The image segmentation unit packs the calculated deformation difference into structured data (such as JSON format) and transmits it to the warning module through the local bus. The warning module is used to trigger a guardrail deformation alarm and upload the guardrail position when the evaluation of the deformation variable difference exceeds the set deformation variable threshold.

[0038] The three-dimensional contour features after SPP expansion and channel attention enhancement are input into the decoder of the 3D U-Net for subsequent processing and finally used for deformation difference calculation. This process captures deformations of different magnitudes through multi-scale feature expansion and focuses on key deformation areas through channel attention, significantly improving the recognition ability of small deformations.

[0039] The above is only a preferred embodiment of the present application, not any form of limitation on the present application. Although the present application has been disclosed as above, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the above disclosed technical content to make equivalent embodiments with equivalent changes, but as long as it does not deviate from the technical solution of the present application, any modification, change and modification of the above embodiments according to the technical essence of the present application are still within the scope of the present application.

Claims

1. A multi-functional, perception and local control, smart safety barrier system, characterized by: The application relates to a barrier real-time surface image acquisition and processing system, which comprises the following parts: a data acquisition module, which is used for acquiring a barrier real-time surface image and calculating an appearance surface point cloud of the barrier according to the real-time surface image, and projecting the appearance surface point cloud to generate a vertical real-time depth map; a preset reference point cloud of a preset reference barrier is acquired, and a vertical reference depth map is generated by projecting the preset reference point cloud, wherein the vertical real-time depth map and the vertical reference depth map are both two-dimensional matrices; a wavelet transform module, which comprises a wavelet transform processing unit, a phase difference calculation unit and a displacement amount conversion unit; the wavelet transform processing unit is used for performing Morlet continuous wavelet transform on the vertical real-time depth map and the vertical reference depth map output by the data acquisition module to generate complex phase spectra of the vertical real-time depth map and the vertical reference depth map respectively; the phase difference calculation unit is used for calculating phase differences according to the real-time phase spectra corresponding to the real-time depth map and the reference depth map respectively to obtain a continuous phase difference distribution map; the displacement amount conversion unit is used for calculating a horizontal displacement amount by using a linear relationship between the phase difference and the displacement; a warning module is used for triggering a barrier movement alarm and uploading a barrier position when the horizontal displacement amount exceeds a set displacement threshold value.

2. The intelligent safety barrier system with multi-function perception and local control according to claim 1, characterized in that: The wavelet transform processing unit performs Morlet continuous wavelet transform on the vertical real-time depth map and the vertical reference depth map, wherein the calculation formula of a Morlet wavelet base function is as follows: ； t is time (spatial position), is the center frequency, is a normalization factor to ensure consistency in the oscillation characteristics at different scales, is a Gaussian envelope function, is a complex exponential oscillation function, s is a scale parameter, where L: average spacing between guardrail posts is calculated by the reference depth map.

3. A smart guard rail system with multi-function perception and local control according to claim 2, characterized in that: The calculation formula for calculating the phase difference according to the two phase spectra is as follows: ； wherein, is the real-time phase spectrum, is the reference phase spectrum.

4. The intelligent guardrail system with multi-function perception and local control according to claim 3, characterized in that: The phase difference calculation unit further comprises judging whether phase wrapping exists in the phase difference and performing phase unwrapping on the phase difference with phase wrapping, which comprises the following steps: If: then If: then .

5. The intelligent guard rail system with multi-function perception and local control according to claim 4, characterized in that: The wavelet transform module further comprises a displacement optimization unit, which uses a median filtering method to optimize the generated displacement amount, and the optimization formula is as follows: ； wherein, is the original displacement difference, is the optimized displacement difference, and x is a window size.

6. The intelligent guardrail system with multi-function perception and local control according to claim 1, characterized in that: The real-time surface image is a parallax map, and the data acquisition module obtains the parallax map by the following steps: using a depth camera under two visual angles to acquire a barrier real-time first image and a real-time second image respectively; using a stereo vision matching algorithm to perform limit constraint on the real-time first image and the real-time second image, and calculating a parallax map of the real-time first image and the real-time second image.

7. A multi-functional, perception and local control based, smart guard rail system as claimed in claim 6, wherein: The data acquisition module further comprises an image segmentation unit, which comprises a unit for extracting barrier real-time three-dimensional contour data and reference three-dimensional contour data according to the appearance surface point cloud and the preset reference point cloud respectively, using a U-Net network to calculate a deformation variable difference value between the real-time three-dimensional contour data and the reference three-dimensional contour data, and transmitting the deformation variable difference value to the warning module; the warning module is used for triggering a barrier deformation alarm and uploading a barrier position when the deformation variable difference value exceeds a set deformation variable threshold value.

8. The intelligent guardrail system with multi-function perception and local control according to claim 7, characterized in that: The image segmentation unit comprises three-dimensional contour feature magnification processing of the three-dimensional contour data and the reference three-dimensional contour data, which comprises the following steps: SPP structure is adopted to magnify the barrier contour features of the three-dimensional contour data and the reference three-dimensional contour data, and a channel attention mechanism is used to extract the real-time three-dimensional contour data and the reference three-dimensional contour data.

9. The intelligent guardrail system of claim 1, wherein: The application further comprises a local control module, which is used for real-time processing of point cloud data and wavelet transform results based on an edge computing chip.

Images