Tunnel exit white hole effect detection method and system based on multi-source data fusion

Abstract

The invention relates to the technical field of tunnel intelligent detection, in particular to a tunnel exit white hole effect detection method and system based on multi-source data fusion, and the method comprises the steps: collection and deployment: deploying cameras at a tunnel exit in a stepped manner, forming a continuous observation chain of the multi-stage cameras, and collecting an image of the tunnel exit; a brightness analysis step: performing brightness extraction on the hole in the image to obtain a brightness time sequence; a wide-area brightness acquisition step: performing wide-area brightness detection on the tunnel exit to obtain the wide-area brightness; a vehicle speed sudden drop proportion acquisition step: carrying out vehicle speed sudden drop detection on the vehicle in the tunnel exit area to obtain the vehicle speed sudden drop proportion; and a prediction analysis step: according to the brightness time sequence, the wide area brightness and the vehicle speed sudden drop proportion, adopting a multi-mode fusion neural network to carry out multi-mode fusion analysis, and predicting the white hole effect probability. According to the scheme, white hole effect detection can be carried out, the detection range is not limited, space-time relevance is achieved, the false alarm rate can be reduced, and the detection accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent tunnel detection technology, specifically to a method and system for detecting the white hole effect at tunnel exits based on multi-source data fusion. Background Technology

[0002] The white hole effect refers to the visual adaptation phenomenon experienced by drivers when exiting a tunnel due to a sudden change in light. Inside a tunnel, the light is dim (usually below 100 lux), causing the pupils to dilate to adapt to the low light environment. Upon exiting the tunnel and encountering intense external light (brightness on a sunny day can reach over 10,000 lux), the pupils do not have enough time to constrict, and a large influx of light into the retina causes a "white halo," resulting in a temporary whitening of the field of vision (similar to an overexposed photograph). Simultaneously, the rod cells in the retina (responsible for night vision) are active inside the tunnel. Under the sudden stimulation of intense light, the cone cells (responsible for photopic vision) require several seconds to tens of seconds to take over, during which time visual blurring occurs.

[0003] The white hole effect is a common optical-physiological effect with clear safety hazards. Therefore, to reduce its impact on drivers, brightness sensors can be used to detect ambient light and alert drivers to the presence of a white hole effect at tunnel exits. Alternatively, corresponding white hole effect mitigation devices can be activated to reduce or eliminate the effect. However, existing white hole effect detection technologies still have some limitations, as follows: 1. Limited detection range: A single brightness sensor only covers a 20° range, which cannot capture the global features of the white hole effect; 2. High false alarm rate: Sudden changes in ambient light (such as cloud cover) can easily trigger false alarms; 3. Single data analysis: Single data is analyzed independently, lacking spatiotemporal correlation and with low accuracy.

[0004] Therefore, there is an urgent need for a method and system for detecting the white hole effect at tunnel exits based on multi-source data fusion. This system should be able to detect the white hole effect, with an unrestricted detection range, spatiotemporal correlation, and the ability to reduce false alarm rates and improve detection accuracy. Summary of the Invention

[0005] One of the objectives of this invention is to provide a method for detecting the white hole effect at tunnel exits based on multi-source data fusion. This method can detect the white hole effect without limitations on the detection range, has spatiotemporal correlation, reduces the false alarm rate, and improves the accuracy of detection, thereby reducing tunnel safety risks.

[0006] The basic solution provided by this invention is a method for detecting the white hole effect at tunnel exits based on multi-source data fusion, comprising: Data collection and deployment steps: Deploy in a stepped manner at the tunnel exit. Multiple cameras form a continuous observation chain to collect images of the tunnel exit; Brightness analysis steps: Extract the brightness of the holes in the image to obtain a brightness time series; Wide-area brightness acquisition steps: Perform wide-area brightness detection at the tunnel exit to acquire the wide-area brightness; Steps for collecting the percentage of sudden speed drops: Detect sudden speed drops in vehicles in the tunnel exit area and obtain the percentage of sudden speed drops. Predictive analysis steps: Based on the brightness time series, wide-area brightness and the proportion of sudden drop in vehicle speed, a multimodal fusion neural network is used to perform multimodal fusion analysis to predict the probability of the white hole effect.

[0007] Furthermore, the data collection and deployment steps include: Two cameras are deployed in a stepped manner along the tunnel exit direction, and the proportion of the tunnel opening captured by adjacent cameras decreases in that order. The overlap between adjacent fields of view is greater than or equal to the preset overlap ratio, forming a continuous observation chain from the inside of the tunnel to the tunnel exit with each camera as the first level. The camera captures the original image before and the image after the tunnel exit is illuminated.

[0008] Furthermore, the brightness analysis step includes: Segment the export region, extract the grayscale values ​​of the export region, and calculate the first... The average brightness of the exit in the image captured by the high-level camera after supplemental lighting. : ; in This indicates the position of the pixel. and time grayscale value, These are the pixel values ​​of the exit region in the image after illumination. Calculate the brightness conversion coefficient based on the pixel values ​​of the exit region in the original image and the image after illumination. : ; in These are the pixel values ​​of the exit region in the original image. This is a mean function used to calculate the average grayscale value of pixels in the exit region. according to The effect of supplemental lighting is corrected in reverse: ; The average brightness after reverse correction for each level of camera is used as the brightness time series.

[0009] Furthermore, the wide-area brightness acquisition step includes: A luminance meter was used to perform wide-area luminance detection at the tunnel exit to obtain the wide-area luminance data. The brightness detector is a single device, with a maximum coverage of 0~180° detection range; Alternatively, a network of brightness detectors consisting of multiple sensors can be deployed, with the sensors arranged according to angle. The distribution is weighted and fused to maximize its coverage of the detection range from 0 to 180°. ; in The angle between the sensor and the centerline of the tunnel exit.

[0010] Furthermore, the aforementioned vehicle speed drop ratio for: in, This represents the average speed of vehicles inside the tunnel.

[0011] Furthermore, the multimodal fusion neural network includes: an input layer, a feature extraction layer, an attention fusion layer, and an output layer; The input layer is used to input the brightness time series, wide-area brightness, and vehicle speed drop ratio, where the brightness time series is the brightness time series of multiple cameras; The feature extraction layer is used to extract features from the input brightness time series, wide-area brightness, and vehicle speed drop ratio, respectively, to obtain the corresponding features. ; Attention fusion layer, used for For Query, calculate and weight : ; It is also used for weighted fusion to obtain the probability of the white hole effect: ; The output layer is used to output the probability of the white hole effect.

[0012] Furthermore, feature extraction is performed on the input brightness time series, wide-area brightness, and vehicle speed drop ratio to obtain the corresponding features. ,include: Temporal features are extracted from the brightness time series using a temporal convolutional network. ; For both wide-area brightness and the sudden drop in vehicle speed, a fully connected layer is used to map them as k-dimensional vectors. .

[0013] Furthermore, it also includes: a time-adaptive correction step: initializing the daily white hole detection time: start time. End time Obtain the latitude of the tunnel and adjust the white hole detection room accordingly. : .

[0014] Furthermore, it also includes: cross-camera brightness analysis steps: Calculate the rate of change over time based on the average brightness after inverse correction. : ; Based on the average brightness after inverse correction, spatial consistency analysis is performed to determine whether there is local interference.

[0015] The second objective of this invention is to provide a tunnel exit white hole effect detection system based on multi-source data fusion, which can detect white hole effects with no limit on the detection range, has spatiotemporal correlation, can reduce false alarm rate, improve detection accuracy, and thus reduce tunnel safety risks.

[0016] This invention provides a second basic solution: a tunnel exit white hole effect detection system based on multi-source data fusion, comprising: several cameras, a brightness detector, a vehicle speed measuring instrument, and a server. Stepped deployment at the tunnel exit Multiple cameras form a continuous observation chain of multiple camera levels; The camera is used to capture images of the tunnel exit and send them to the server. A brightness detector is used to perform wide-area brightness detection at the tunnel exit, acquire the wide-area brightness, and send it to the server. The vehicle speed measuring instrument is used to detect the vehicle speed in the tunnel exit area and send the speed detection results to the server. The server-side component is used to extract the brightness of holes in the image and obtain a brightness time series. It is also used to detect sudden drops in vehicle speed based on the vehicle speed detection results and to obtain the percentage of sudden drops in vehicle speed. It is also used to predict the probability of the white hole effect by performing multimodal fusion analysis based on brightness time series, wide-area brightness and vehicle speed drop ratio using a multimodal fusion neural network.

[0017] Beneficial effects: Firstly, this scheme deploys multiple cameras in a tiered manner at the tunnel exit, forming a continuous observation chain of multi-level cameras to collect images of the tunnel exit. The tiered deployment of multiple cameras allows for the collection of images showing different proportions of the tunnel exit within the overall image, while each camera captures the entire tunnel exit, ensuring the capture of global features of the white hole effect. This allows subsequent models to obtain brightness sequences from multiple images at the same time, resulting in a rich set of features of the tunnel exit. Furthermore, compared to a single camera, the multiple cameras deployed in this scheme can serve as a validation set for each other, verifying the correctness of the collected images. They can also act as backup cameras for other cameras, avoiding the problem of detection failure due to the malfunction of a single device. Secondly, this solution performs wide-area brightness detection at the tunnel exit, covering the entire tunnel exit, to capture the global characteristics of the white hole effect; it also detects sudden speed drops in vehicles in the tunnel exit area to obtain the proportion of sudden speed drops; thus, it can characterize the white hole effect from multiple aspects. Finally, this solution uses a multimodal fusion neural network to perform multimodal fusion analysis based on the acquired brightness time series, wide-area brightness, and vehicle speed drop ratio to predict the probability of the white hole effect. The multimodal fusion neural network integrates multiple data sources for comprehensive analysis. Even with sudden changes in ambient light, the multimodal fusion neural network does not analyze from a single brightness index. Therefore, the analysis results are less affected by changes in ambient light and are less likely to trigger false alarms. Compared to single data analysis, this solution comprehensively analyzes multiple data sources, has spatiotemporal correlation, and is more accurate.

[0018] In summary, this solution can detect the white hole effect with no limit on the detection range, possesses spatiotemporal correlation, reduces the false alarm rate, and improves the accuracy of detection. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating an embodiment of the tunnel exit white hole effect detection method based on multi-source data fusion according to the present invention. Detailed Implementation

[0020] The following detailed description illustrates the specific implementation method: The markings in the accompanying drawings include: Example 1 This embodiment is basically as shown in the appendix. Figure 1 As shown: A method for detecting the white hole effect at tunnel exits based on multi-source data fusion, including the following: Data collection and deployment steps: Deploy in a stepped manner at the tunnel exit. Multiple cameras form a continuous observation chain to collect images of the tunnel exit; The specific process is as follows: Two cameras are deployed in a stepped manner along the tunnel exit direction, and the proportion of the tunnel opening captured by the adjacent cameras decreases in a decreasing trend (e.g., 70%, 50%, 30%). The overlap of adjacent fields of view is greater than or equal to the preset overlap ratio, which is 20% in this embodiment, forming a continuous observation chain from the inside of the tunnel to the tunnel exit with each camera as the first level. The camera captures the original image before and the image after the tunnel exit is illuminated; in this embodiment, the image after illumination is the image taken by the camera with the flash on. Brightness analysis steps: Extract the brightness of the holes in the image to obtain a brightness time series; Segment the export region, extract the grayscale values ​​of the export region, and calculate the first... The average brightness of the exit in the image captured by the high-level camera after supplemental lighting: ; in This indicates the position of the pixel. and time The grayscale value (range 0~255, 0 is pure black, 255 is pure white). These are the pixel values ​​of the exit region in the image after illumination. In this embodiment, U-Net is used to segment the exit region; Calculate the brightness conversion coefficient based on the pixel values ​​of the exit region in the original image and the image after illumination. : ; ; in These are the pixel values ​​of the exit region in the original image. These are the pixel values ​​of the exit region in the image after illumination. This is a mean function, which calculates the average grayscale value of the pixels in the exit region. This represents the total number of pixels in the opening area. For the first The grayscale value of each pixel; according to The effect of supplemental lighting is corrected in reverse: ; The average brightness after reverse correction for each level of camera is used as the brightness time series; Wide-area brightness acquisition steps: Perform wide-area brightness detection at the tunnel exit to acquire the wide-area brightness; The specific process is as follows: A luminance meter was used to perform wide-area luminance detection at the tunnel exit to obtain the wide-area luminance data. In this embodiment, the brightness detector is a wide-angle brightness detector, which is a single device that covers a maximum detection range of 0~180° and outputs ambient brightness. Alternatively, a network of brightness detectors consisting of multiple sensors can be deployed, with the sensors arranged according to angle. The distribution and weighted fusion are used to maximize the detection range of 0~180°. ; in The angle between the sensor and the centerline of the tunnel exit; Steps for collecting the percentage of sudden speed drops: Detect sudden speed drops in vehicles in the tunnel exit area and obtain the percentage of sudden speed drops. Among them, the proportion of sudden drop in vehicle speed for: in, The average speed of vehicles inside the tunnel. The number of vehicles passing through is used to verify driver responses, thus reflecting whether a white hole effect has occurred.

[0021] Predictive analysis steps: Based on the brightness time series, wide-area brightness and the proportion of sudden drop in vehicle speed, a multimodal fusion neural network is used to perform multimodal fusion analysis to predict the probability of the white hole effect; Specifically, the multimodal fusion neural network includes: an input layer, a feature extraction layer, an attention fusion layer, and an output layer; The input layer is used to input the brightness time series, wide-area brightness, and vehicle speed drop ratio, where the brightness time series is a brightness time series from multiple cameras; in this embodiment, the step size is 1 second, such as: ; Among them, wide-area brightness For single-device data acquisition or network data acquisition by brightness detectors; The feature extraction layer is used to extract features from the input brightness time series, wide-area brightness, and vehicle speed drop ratio, respectively, to obtain the corresponding features. ; Specifically, for the brightness time series, a temporal convolutional network (TCN) is used to extract temporal features. In this embodiment, a three-layer temporal convolutional network is used, with the number of channels being [64, 128, 256]. For both wide-area brightness and the sudden drop in vehicle speed, a fully connected layer is used to map them as k-dimensional vectors. In this embodiment, k=128; Attention fusion layer, used for For Query, calculate and weight : ; It is also used for weighted fusion to obtain the probability of the white hole effect: ; The output layer outputs the probability of the white hole effect, i.e., the probability of the white hole effect occurring. Multimodal fusion allows for comprehensive analysis, rather than a single indicator determining the prediction result. It is less affected by sudden changes and has higher accuracy.

[0022] Furthermore, in this embodiment, the multimodal fusion neural network is trained in stages. During the training process, the image branch (i.e., the brightness time series branch, which uses only brightness time series data and the loss function is weighted cross-entropy) is pre-trained; the image branch is frozen, and the sensor and vehicle speed branches are trained; end-to-end fine-tuning is performed, and all parameters are jointly optimized, with the learning rate decayed to 1e-5; data augmentation is performed by adding illumination perturbations, simulating supplementary lighting noise, and random vehicle speed jitter.

[0023] If the probability of the white hole effect is greater than the preset probability threshold, a white hole effect warning will be issued or a white hole effect interference device will be activated. The white hole effect warning can be issued by setting up light signs in the tunnel, and the white hole effect interference device can be the lighting equipment at the tunnel exit section, with the brightness gradually increasing from the inside to the outside.

[0024] Example 2 This embodiment is basically the same as the above embodiments, except that: Also includes: Time-adaptive correction steps: Initialize daily white hole detection time: Start time End time Obtain the latitude of the tunnel and adjust the white hole detection room accordingly. : ; Start time in this embodiment The closing time is 10:00 AM. It is 4 PM, or 4:00 PM; Considering that the start time of sunlight varies depending on the latitude of the tunnel, the time when the white hole effect occurs also varies. Therefore, this scheme takes into account the latitude of the tunnel, with higher latitudes (…). (Approaching 90°), the detection time shifts further back (e.g., in high-latitude regions, it is corrected to 11:00-17:00). This scheme achieves latitude-adaptive white hole effect detection.

[0025] Example 3 This embodiment is basically the same as the above embodiment, except that it also includes: a cross-camera brightness analysis step. Calculate the rate of change over time based on the average brightness after inverse correction. This is used to detect anomalous abrupt changes in brightness at tunnel exits, thereby distinguishing between changes in natural lighting and the white hole effect. ; Based on the average brightness after inverse correction, spatial consistency analysis is performed to determine whether there is local interference. Specifically, it is determined whether the average brightness change trend of the image captured by the (i+1)th level camera is the same as the average brightness change trend of the image captured by the i-th level camera. If not, it is determined that there is local interference. This enables spatial consistency judgment of multiple levels of cameras, forming a mutual verification relationship and improving the accuracy of detection results. For example, if the brightness of the i-th level decreases, the brightness of the (i+1)th level should increase synchronously; otherwise, it is determined to be local interference.

[0026] Example 3 This embodiment provides a tunnel exit white hole effect detection system based on multi-source data fusion, including: several cameras, a brightness detector, a vehicle speed measuring instrument, and a server; Stepped deployment at the tunnel exit Multiple cameras form a continuous observation chain of multiple camera levels; The camera is used to capture images of the tunnel exit and send them to the server. A brightness detector is used to perform wide-area brightness detection at the tunnel exit, acquire the wide-area brightness, and send it to the server. The vehicle speed measuring instrument is used to detect the vehicle speed in the tunnel exit area and send the speed detection results to the server. The server-side component is used to extract the brightness of holes in the image and obtain a brightness time series. It is also used to detect sudden drops in vehicle speed based on the vehicle speed detection results and to obtain the percentage of sudden drops in vehicle speed. It is also used to predict the probability of the white hole effect by performing multimodal fusion analysis based on brightness time series, wide-area brightness, and the proportion of sudden vehicle speed drops using a multimodal fusion neural network. In this embodiment, the server can be a cloud service or a distributed server deployed in the tunnel.

[0027] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A method for detecting the white hole effect at tunnel exits based on multi-source data fusion, characterized in that, include: Data collection and deployment steps: Deploy in a stepped manner at the tunnel exit. Multiple cameras form a continuous observation chain to collect images of the tunnel exit; Brightness analysis steps: Extract the brightness of the holes in the image to obtain a brightness time series; Wide-area brightness acquisition steps: Perform wide-area brightness detection at the tunnel exit to acquire the wide-area brightness; Steps for collecting the percentage of sudden speed drops: Detect sudden speed drops in vehicles in the tunnel exit area and obtain the percentage of sudden speed drops. Predictive analysis steps: Based on the brightness time series, wide-area brightness and the proportion of sudden drop in vehicle speed, a multimodal fusion neural network is used to perform multimodal fusion analysis to predict the probability of the white hole effect.

2. The method for detecting the white hole effect at tunnel exits based on multi-source data fusion according to claim 1, characterized in that, The data collection and deployment steps include: Two cameras are deployed in a stepped manner along the tunnel exit direction, and the proportion of the tunnel opening captured by adjacent cameras decreases in that order. The overlap between adjacent fields of view is greater than or equal to the preset overlap ratio, forming a continuous observation chain from the inside of the tunnel to the tunnel exit with each camera as the first level. The camera captures the original image before and the image after the tunnel exit is illuminated.

3. The method for detecting the white hole effect at tunnel exits based on multi-source data fusion according to claim 2, characterized in that, The brightness analysis step includes: Segment the export region, extract the grayscale values ​​of the export region, and calculate the first... The average brightness of the exit in the image captured by the high-level camera after supplemental lighting. : ； in This indicates the position of the pixel. and time grayscale value, These are the pixel values ​​of the exit region in the image after illumination. Calculate the brightness conversion coefficient based on the pixel values ​​of the exit region in the original image and the image after illumination. : ； in These are the pixel values ​​of the exit region in the original image. This is a mean function used to calculate the average grayscale value of pixels in the exit region. according to To reverse the effect of supplemental lighting: ； The average brightness after reverse correction for each level of camera is used as the brightness time series.

4. The method for detecting the white hole effect at tunnel exits based on multi-source data fusion according to claim 1, characterized in that, The wide-area brightness acquisition step includes: A luminance meter was used to perform wide-area luminance detection at the tunnel exit to obtain the wide-area luminance data. The brightness detector is a single device, with a maximum coverage of 0~180° detection range; Alternatively, a network of brightness detectors consisting of multiple sensors can be deployed, with the sensors arranged according to angle. The distribution is weighted and fused to maximize its coverage of the detection range from 0 to 180°. ； in The angle between the sensor and the centerline of the tunnel exit.

5. The method for detecting the white hole effect at tunnel exits based on multi-source data fusion according to claim 1, characterized in that, The percentage of sudden drop in vehicle speed for: in, This represents the average speed of vehicles inside the tunnel.

6. The method for detecting the white hole effect at tunnel exits based on multi-source data fusion according to claim 1, characterized in that, The multimodal fusion neural network includes: an input layer, a feature extraction layer, an attention fusion layer, and an output layer; The input layer is used to input the brightness time series, wide-area brightness, and vehicle speed drop ratio, where the brightness time series is the brightness time series of multiple cameras; The feature extraction layer is used to extract features from the input brightness time series, wide-area brightness, and vehicle speed drop ratio, respectively, to obtain the corresponding features. ; Attention fusion layer, used for For Query, calculate and weight : ； It is also used for weighted fusion to obtain the probability of the white hole effect: ； The output layer is used to output the probability of the white hole effect.

7. The method for detecting the white hole effect at tunnel exits based on multi-source data fusion according to claim 6, characterized in that, The system extracts features from the input brightness time series, wide-area brightness, and vehicle speed drop ratio to obtain the corresponding features. ,include: Temporal features are extracted from the brightness time series using a temporal convolutional network. ; For both wide-area brightness and sudden speed drop ratio, a fully connected layer is used to map them as k-dimensional vectors. .

8. The method for detecting the white hole effect at tunnel exits based on multi-source data fusion according to claim 1, characterized in that, Also includes: Time-adaptive correction steps: Initialize daily white hole detection time: Start time End time Obtain the latitude of the tunnel and adjust the white hole detection room accordingly. : 。 9. The method for detecting the white hole effect at tunnel exits based on multi-source data fusion according to claim 1, characterized in that, It also includes: cross-camera brightness analysis steps: Calculate the rate of change over time based on the average brightness after inverse correction. : ； Based on the average brightness after inverse correction, spatial consistency analysis is performed to determine whether there is local interference.

10. A tunnel exit white hole effect detection system based on multi-source data fusion, characterized in that, include: Several cameras, brightness detectors, vehicle speed measuring instruments, and server-side components; Stepped deployment at the tunnel exit Multiple cameras form a continuous observation chain of multiple camera levels; The camera is used to capture images of the tunnel exit and send them to the server. A brightness detector is used to perform wide-area brightness detection at the tunnel exit, acquire the wide-area brightness, and send it to the server. The vehicle speed measuring instrument is used to detect the vehicle speed in the tunnel exit area and send the speed detection results to the server. The server-side component is used to extract the brightness of holes in the image and obtain a brightness time series. It is also used to detect sudden drops in vehicle speed based on the vehicle speed detection results and to obtain the percentage of sudden drops in vehicle speed. It is also used to predict the probability of the white hole effect by performing multimodal fusion analysis based on brightness time series, wide-area brightness and vehicle speed drop ratio using a multimodal fusion neural network.

Images