Intelligent signal light diagnosis method based on multi-modal perception

Abstract

The application provides a signal lamp intelligent diagnosis method based on multi-modal perception, wherein the method comprises the following steps: acquiring visual perception data, control state data and radar trajectory data of a target signal lamp; comparing the visual perception data with the control state data to generate a basic running state; when a state conflict occurs, extracting group kinematic characteristic quantities of associated vehicles in the radar trajectory data, performing closed-loop reverse deduction on the state conflict, and distinguishing between environmental interference false alarms and real physical failures; recording time delay characteristic sequences of the target signal lamp under different phase sequence switching directions; extracting asymmetric deviation components in the time delay characteristic sequences, using the asymmetric deviation components to strip network synchronization errors, and diagnosing the aging level of internal components of the target signal lamp. The application effectively eliminates external light interference and quantitatively evaluates the internal hardware aging state.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a method for intelligent diagnosis of traffic lights based on multimodal perception. Background Technology

[0002] Traffic lights are widely installed at various traffic nodes, such as urban road intersections, and their stable operation is crucial for maintaining road traffic order. To monitor the real-time status of traffic lights, management departments typically deploy image acquisition terminals and electrical status monitoring modules at intersections. This monitoring method primarily uses cameras to capture the apparent light emission status of the traffic light lenses, while simultaneously analyzing the drive levels output from the underlying control cabinet to determine the current operational status of the equipment. This monitoring model, combining visual and control commands, provides fundamental data support for the daily maintenance of traffic management systems.

[0003] In real-world outdoor environments, the angle and intensity of natural light sources are constantly changing. When external light shines directly onto the lens housing of a traffic light, specular reflection can easily occur. This simple physical reflection is often misinterpreted by image acquisition terminals as a potential light emission. Without the control motherboard actually issuing a power supply command, a logical conflict arises between the visual perception and the underlying electrical control data. Relying solely on these two data sets, technicians struggle to determine whether the conflict stems from interference from external ambient light reflections or from uncontrolled light emission caused by physical welding of relay contacts inside the cabinet.

[0004] Furthermore, after prolonged on / off cycles, the electromechanical components inside the traffic light control cabinet responsible for performing switching actions experience arc erosion on their mechanical contact surfaces, and the internal return springs also suffer from metal fatigue. This natural degradation of materials leads to asymmetrical time deviations in the engagement and disengagement actions of the electromechanical components. In conventional remote status acquisition architectures, data interaction between the device and the central server involves data packet queuing and buffering at communication nodes. This time fluctuation and synchronization error at the network transmission layer directly mixes into the underlying mechanical response delay data. Existing monitoring mechanisms struggle to extract the true mechanical delay time difference of electromechanical components under the interference of network latency fluctuations, making it difficult to objectively quantify and diagnose the physical aging level of the underlying hardware. Summary of the Invention

[0005] The purpose of this invention is to provide a smart diagnostic method for traffic lights based on multimodal perception, in order to solve the technical problems of existing technologies in distinguishing between ambient light interference and real physical faults when faced with conflicts between visual perception and control data status, and in objectively extracting hardware action time differences to diagnose component aging levels under network latency fluctuation interference.

[0006] This invention provides a smart diagnostic method for traffic lights based on multimodal sensing, comprising:

[0007] Acquire visual perception data, control status data, and radar trajectory data of the target signal lights;

[0008] By comparing the visual perception data with the control status data, a basic operating status is generated;

[0009] The intelligent diagnostic method for traffic lights based on multimodal sensing also includes:

[0010] When the basic operating state indicates that the visual perception data and the control state data are inconsistent and present a state conflict, the group kinematic features of the associated vehicles in the radar trajectory data are extracted, and the group kinematic features are used to perform closed-loop reverse deduction of the state conflict in order to distinguish between environmental interference false alarms and real physical faults.

[0011] Record the time delay characteristic sequence of the target signal light under different phase sequence switching directions;

[0012] The asymmetric deviation component in the time delay feature sequence is extracted, and the asymmetric deviation component is used to remove network synchronization error in order to diagnose the aging level of the internal components of the target signal light.

[0013] Optionally, acquiring the visual perception data, control status data, and radar trajectory data of the target traffic light includes:

[0014] A real-time image sequence covering the luminous area of ​​the target traffic light is acquired as the visual perception data;

[0015] The phase sequence level signal output by the control motherboard corresponding to the target signal light is received as the control status data;

[0016] The radar scans the stop line area at the intersection where the traffic light facing the target is located to obtain a set of millimeter-wave point clouds, which serves as the radar trajectory data.

[0017] Optionally, the comparison of the visual perception data with the control state data to generate a basic operating state includes:

[0018] Analyze the color saturation and pixel brightness in the real-time image sequence to determine the apparent luminescence state;

[0019] Analyze the high and low logic level changes in the phase sequence level signal to determine the physical drive state;

[0020] The matching degree between the apparent luminescence state and the physical driving state is evaluated, and the evaluation result is used as the basic operating state.

[0021] Optionally, evaluating the matching degree between the apparent luminescence state and the physical driving state includes:

[0022] When the apparent light emission state indicates light emission and the physical drive state indicates power failure, it is determined that there is a false positive light emission conflict.

[0023] When the apparent light emission state indicator is off and the physical drive state indicator is on, it is determined that there is a blocking power-off conflict.

[0024] The false positive emission conflict and the occlusion power-off conflict are collectively referred to as the state conflict.

[0025] Optionally, the scanning direction is directed towards the stop line area of ​​the intersection where the target traffic light is located to obtain a millimeter-wave point cloud set as the radar trajectory data, including:

[0026] Filter static background noise in the millimeter-wave point cloud set;

[0027] Lock onto the target vehicle queue that is in a dynamic approaching state;

[0028] The continuous spatial coordinates of the target vehicle convoy are integrated into the radar trajectory data.

[0029] Optionally, extracting the group kinematic features of associated vehicles from the radar trajectory data includes:

[0030] The set of associated vehicles in the deceleration zone is selected from the radar trajectory data;

[0031] Calculate the mean longitudinal deceleration and braking stopping ratio of each vehicle in the associated vehicle set;

[0032] The mean longitudinal deceleration and the braking stopping ratio are combined to form the group kinematic characteristic.

[0033] Optionally, the step of using the group kinematic characteristics to perform closed-loop back-engineering of the state conflict to distinguish between environmental disturbance false alarms and real physical faults includes:

[0034] When the state conflict manifests as the apparent light-emitting state indicator being lit and the physical drive state indicator being de-energized, it is determined whether the braking stop ratio exceeds a preset parking threshold, and whether the average longitudinal deceleration value conforms to the red light braking model.

[0035] If all conditions are met, the target traffic light is deduced to be currently red, and the current state conflict is marked as a false alarm caused by environmental interference induced by strong light reflection.

[0036] Optionally, the step of using the group kinematic characteristics to perform closed-loop back-engineering of the state conflict to distinguish between environmental disturbance false alarms and real physical faults further includes:

[0037] If the braking stop ratio does not exceed the preset parking threshold, or the average longitudinal deceleration does not conform to the red light braking model, the target traffic light is deduced to currently be a green light.

[0038] The determination is that the group kinematic characteristics contradict the environmental interference false alarm logic;

[0039] In response to the contradictory state, the system confirms that the target signal light has experienced a real physical fault and generates a hardware fault alarm command.

[0040] Optionally, recording the time delay characteristic sequence of the target signal light under different phase sequence switching directions includes:

[0041] The positive state transition time when the target signal light changes from off to on, and the corresponding time when the start-up phase sequence level signal goes high are obtained, and the start-up hysteresis time is calculated.

[0042] The reverse state transition time when the target signal light changes from lit to off is obtained, and the corresponding time when the shutdown phase sequence level signal goes low is obtained, and the shutdown hysteresis time is calculated.

[0043] The delay feature sequence is generated by combining the turn-on hysteresis time and the turn-off hysteresis time.

[0044] Optionally, the step of extracting the asymmetric deviation component from the time delay feature sequence and using the asymmetric deviation component to remove network synchronization errors in order to diagnose the aging level of the internal components of the target traffic light includes:

[0045] Calculate the absolute value of the difference between the turn-on hysteresis time and the turn-off hysteresis time, and use it as the asymmetric deviation component;

[0046] When both the start-up hysteresis time and the stop-down hysteresis time are greater than zero and the asymmetric deviation component is lower than a preset symmetry threshold, it is determined that the network synchronization error exists and the internal component status is confirmed to be healthy.

[0047] When the asymmetric deviation component is greater than or equal to a preset symmetry threshold, the mechanical contact viscosity level is mapped to the corresponding mechanical contact viscosity level using the asymmetric deviation component, and the mechanical contact viscosity level is output as the aging level.

[0048] The present invention has achieved the following beneficial effects:

[0049] This invention simultaneously acquires visual perception data, control status data, and radar trajectory data of the target traffic light. When a conflict arises between visual performance and control commands, it introduces the group kinematic characteristics of associated vehicles from the radar trajectory data for closed-loop reverse inference. This eliminates false positives induced by external natural light reflection, distinguishes between ambient light interference and actual underlying hardware faults, and improves the accuracy of traffic light operation status diagnosis in open environments. Simultaneously, this invention records the time delay characteristic sequence of the traffic light during forward illumination and reverse extinguishing. By calculating the asymmetric deviation component in this characteristic sequence, it offsets network synchronization errors during bidirectional data transmission in the time dimension, restoring the true mechanical delay differences in the opening and closing actions of the underlying electromechanical components. Based on this, it maps the physical viscosity level of the mechanical contacts, enabling an objective quantitative assessment of the aging status of traffic light electromechanical components in operating environments with network fluctuations. This provides reliable data support for preventative maintenance of traffic control equipment.

[0050] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0051] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0052] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0053] Figure 1 The main flowchart of the intelligent diagnostic method for traffic lights based on multimodal perception provided in this embodiment of the invention is shown below.

[0054] Figure 2 This is a flowchart of the sub-step for acquiring multimodal sensing data in an embodiment of the present invention;

[0055] Figure 3 This is a flowchart of the sub-step for comparing data and generating basic operating status in an embodiment of the present invention;

[0056] Figure 4 This is a flowchart illustrating the determination process for extracting group kinematic features and performing closed-loop reverse deduction in an embodiment of the present invention.

[0057] Figure 5 This is a flowchart illustrating the calculation of delay characteristic sequences and the diagnosis of internal component aging levels in an embodiment of the present invention.

[0058] Figure 6This is a physical architecture diagram of the hardware support system for performing the above method in an embodiment of the present invention. Detailed Implementation

[0059] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0060] Traffic lights are used at traffic nodes such as main road intersections, ramp merging points, and pedestrian crossings. Under long-term continuous operation, traffic lights are subjected to environmental effects such as temperature cycling and ultraviolet radiation, resulting in physical property shifts at the level of the light-emitting and control components. To address the technical problem of state determination shifts caused by external environmental light source reflection interference and internal hardware material aging, this invention discloses a traffic light intelligent diagnostic method based on multimodal perception.

[0061] like Figure 1 As shown, the intelligent diagnostic method for traffic lights based on multimodal perception includes the following execution steps:

[0062] Step S10: Acquire visual perception data, control status data, and radar trajectory data of the target signal light;

[0063] Step S20: Compare the visual perception data with the control state data to generate a basic operating state;

[0064] Step S30: When the basic operating state indicates that the visual perception data and the control state data are inconsistent and present a state conflict, the group kinematic feature of the associated vehicles in the radar trajectory data is extracted, and the group kinematic feature is used to perform closed-loop reverse deduction of the state conflict in order to distinguish between environmental interference false alarms and real physical faults.

[0065] Step S40: Record the time delay characteristic sequence of the target signal light under different phase sequence switching directions;

[0066] Step S50: Extract the asymmetric deviation component from the time delay feature sequence, and use the asymmetric deviation component to remove network synchronization error in order to diagnose the aging level of the internal components of the target signal light.

[0067] In this embodiment, before executing step S10, the method further includes: registering the spatial coordinates of the multimodal sensing devices at the target intersection and synchronizing them with the time reference within a preset time period to obtain a spatial transformation matrix and a synchronization timestamp. It is understood that data from physical calibrators is collected by cameras and millimeter-wave radar deployed at the intersection. Visual perception data is established in a two-dimensional pixel coordinate system, and radar trajectory data is established in a three-dimensional polar coordinate system. For the spatial location of the target traffic light, the system establishes a unified three-dimensional world coordinate system. The three-dimensional coordinates of feature points of the physical calibrator in different coordinate systems are extracted, and a three-dimensional rotation matrix and a three-dimensional translation vector are calculated based on the feature point mapping relationship. The rotation matrix and translation vector are then combined according to matrix combination rules to generate the spatial transformation matrix. Through the spatial transformation matrix, matrix multiplication is performed on the sensing data of different modalities, mapping them to the same physical spatial reference. Simultaneously, the internal clock crystal oscillators of each sensing device exhibit frequency drift. The edge computing node within the control cabinet is used as the master clock source, and a clock synchronization message is sent to each sensing device using a network time protocol. After receiving the synchronization message, the sensing device calculates the network transmission delay parameter and time deviation parameter of the data packet, and uses the calculated parameters to perform numerical compensation operation on the local system time register, so that the collected multimodal data has a numerically aligned synchronization timestamp.

[0068] Specifically, such as Figure 2 As shown, step S10, which involves acquiring the visual perception data, control status data, and radar trajectory data of the target signal light, includes the following sub-steps:

[0069] Step S11: Acquire a real-time image sequence covering the luminous area of ​​the target traffic light as the visual perception data. This is achieved by capturing images using a video acquisition terminal fixed to a support rod opposite the target traffic light. The video acquisition terminal includes a complementary metal-oxide-semiconductor (CMOS) image sensor. During the exposure cycle, the image sensor performs exposure with short-time integration parameters and long-time integration parameters, respectively, and linearly interpolates and fuses the pixel charge data obtained from the two exposures to control the pixel values ​​of the luminous area of ​​the target traffic light within a preset grayscale range, preventing saturation overflow or underexposure. After acquiring the original video stream, the original video stream is discretized and sampled at a preset frame rate of Hertz to generate a continuous image set composed of multiple frames of image data. Further, the pre-recorded prior two-dimensional position coordinates of the target traffic light are read, and a polygonal region of interest is delineated along the boundaries of the coordinate points in the image plane. The pixel array data within the polygonal region of interest is read and copied, and the extracted pixel array data is arranged according to the time dimension to form the real-time image sequence.

[0070] Step S12: Receive the phase sequence level signal output by the control motherboard corresponding to the target signal light as the control status data. It is understood that the electrical power supply of the target signal light is controlled by the control motherboard within the control cabinet. An optocoupler circuit component is connected in parallel to the weak current signal control terminal of the control motherboard driver module. When the control motherboard microprocessor writes a logic high-level instruction to the drive pin, the drive pin outputs a DC voltage of rated amplitude. The optocoupler circuit component captures the current voltage change, drives the light-emitting diode to emit light, and converts it into a back-end logic level via a phototransistor. The analog-to-digital conversion channel within the edge computing node reads the logic level at a set sampling frequency. To filter out level jump glitches generated during the instantaneous action of the metal contacts, a sliding window midpoint filtering process is performed on the continuously read logic level value sequence. The set of logic level values ​​within the sliding window is statistically analyzed. If the median value in the set is greater than a preset high-level judgment threshold, a high-level status word is generated in the data register; if the median value in the set is less than a preset low-level judgment threshold, a low-level status word is generated in the data register.

[0071] Specifically, this relates to typical industrial-grade traffic signal control cabinets. The Complementary Metal-Oxide-Semiconductor (CMOS) voltage level specification stipulates that the preset high-level threshold is physically calibrated at the hardware level to be greater than or equal to a certain value. The analog-to-digital conversion quantization value of the analog voltage; the preset low-level judgment threshold is calibrated to be less than or equal to The digital quantization value of the analog voltage. Furthermore, to effectively mask the inherent mechanical bounce of the high-power electromagnetic relay contacts during the engagement and disengagement processes at a physical level, the time span of the sliding window is preset to... to .

[0072] The high-level status word and low-level status word carrying a synchronization timestamp are sequentially written into a circular queue to form the phase sequence level signal. The phase sequence level signal records the underlying timing of the control motherboard's power supply operation to the target indicator light.

[0073] Step S13: Scan the stop line area of ​​the intersection facing the target traffic light to obtain a millimeter-wave point cloud set as the radar trajectory data. It is understood that the radio frequency front-end of the millimeter-wave radar equipment includes a voltage-controlled oscillator (VCO) and a transceiver antenna array. The VCO generates a frequency-modulated continuous wave (FM-CWHM) whose frequency modulates linearly with time. The transmitting antenna radiates the FM-CWHM to cover the physical space of the intersection where the target traffic light is located. When the FM-CWHM touches the physical surface of the road, it generates a backscattered electromagnetic echo. The receiving antenna captures the electromagnetic echo signal, sends it to an RF mixer for multiplication and mixing with the local oscillator signal from the transmitting link, and outputs an intermediate frequency (IF) analog signal after high-frequency components are filtered out by a low-pass filter. The analog-to-digital converter discretizes the IF-CWHM analog signal and writes it into the internal static random access memory (SRAM) to obtain an IF-CWH digital signal matrix. The processing unit performs a distance-dimensional discrete Fourier transform calculation on the intermediate frequency digital signal matrix in the fast time dimension to extract frequency domain peak data and calculate the radial distance parameter; it performs a Doppler velocity-dimensional discrete Fourier transform calculation in the slow time dimension to analyze Doppler frequency shift data and calculate the radial relative velocity parameter. It reads the physical spacing constant of the receiving antenna array and the phase difference data between channels, and performs arcsine function calculation to obtain the spatial angle of arrival parameter. The extracted distance, velocity, and angle parameters are converted into discrete point coordinate data in a three-dimensional coordinate system through coordinate mapping equations. The discrete point coordinate data generated within a single scan cycle are combined to form the millimeter-wave point cloud set.

[0074] Further, in step S13, scanning the stop line area of ​​the intersection facing the target traffic light to obtain a millimeter-wave point cloud set as the radar trajectory data includes: filtering static background noise in the millimeter-wave point cloud set; locking the target vehicle queue in a dynamic approach state; and integrating the continuous spatial coordinates of the target vehicle queue into the radar trajectory data.

[0075] Understandably, the processing unit traverses the millimeter-wave point cloud set in memory, extracting the radial relative velocity parameters encapsulated in each discrete point. It performs an absolute value operation on the radial relative velocity parameters and compares the obtained absolute value with a preset low-speed filtering threshold parameter. If the absolute value does not exceed the low-speed filtering threshold parameter, the corresponding discrete point is determined to originate from a static physical reflector. For the remaining scattered points, a sliding reference window is drawn in the distance-Doppler two-dimensional frequency domain data matrix, centered on the discrete point to be determined. The power values ​​of the surrounding scattered points within the sliding reference window are read, and the arithmetic mean is calculated to obtain the average background noise power. The average background noise power is multiplied by a multiplication constant calculated based on a set false alarm probability to generate a dynamic detection threshold. Specifically, the set false alarm probability is preferably... The multiplication constant is theoretically derived based on the cell average constant false alarm rate (CA-CFAR) algorithm and the assumption that background clutter follows an exponential distribution. Its mathematical formula is expressed as follows: ,in, Let be the multiplication constant. This represents the total number of valid scatter points within the sliding reference window. The set false alarm probability is used. When the reflected power of the discrete point to be judged is greater than the dynamic detection threshold and its radial relative velocity absolute value is greater than the low-speed filtering threshold parameter, the corresponding discrete point address is marked as a valid dynamic point in memory, and the addresses of data points that do not meet the conditions are released from the memory stack.

[0076] When locking onto a target vehicle queue in a dynamic approximation state, a spatial distance-based clustering algorithm is applied to valid dynamic scattered points. The vector map data of the target intersection is read, and the vertex coordinates of the physical boundary polygons of the controlled lanes are extracted. A ray-crossing algorithm is executed to remove scattered points whose coordinates fall outside the boundary polygons. The 3D Euclidean distance between any two valid dynamic scattered points within the lane area is calculated. If this Euclidean distance is less than a preset spatial connectivity radius threshold, the two scattered points are marked as connected in the adjacency matrix. The total number of connected nodes of an individual scattered point in the adjacency matrix is ​​counted. If the total number is greater than the core density threshold, an initial cluster is generated centered on that individual scattered point.

[0077] In this embodiment, the decision hyperparameter of the clustering analysis corresponds to the physical three-dimensional scale of a standard passenger vehicle and the resolution of the radar equipment in an urban traffic scenario: the preset spatial connectivity radius threshold is calibrated based on the physical chassis width of a conventional motor vehicle. to The physical distance between them; the core density determination threshold is calibrated based on the typical spatial angular resolution of the millimeter-wave radar antenna array used at the intersection depth distance. to One effective radar reflection scattering point. The low-speed filtering threshold parameter mentioned earlier is physically calibrated based on the lower limit of normal human walking speed. .

[0078] Clusters with intersecting nodes are merged to form independent cluster sets representing single physical vehicle entities. The 3D coordinate data of all scattered points within each independent cluster set are read, accumulated, and divided by the total number of scattered points to obtain the average coordinates, which are defined as the geometric centroid coordinates. The velocity vector of each independent cluster set is extracted and fitted with the stop line normal vector using the stop line coordinate parameters. An inner product operation is performed on the velocity vector and the stop line normal vector, and the spatial angle is calculated. If the inner product value is positive and the spatial angle is less than a preset judgment margin, the system allocates the vehicle entity data structure that meets the direction conditions into the target vehicle queue buffer. In this embodiment, considering the minor lane changes or steering wheel corrections of motor vehicles approaching intersections, the preset judgment margin is specifically defined in physical space as an angular constant between 10° and 15° to effectively filter out vehicles crossing laterally and accurately retain targets with a clear intention to approach straight.

[0079] When integrating the continuous spatial coordinates of the target vehicle queue into the radar trajectory data, an integer tracking identifier is assigned to each newly added vehicle entity within the target vehicle queue. A Kalman filter state vector is constructed, encompassing the three-dimensional position coordinate matrix and three-dimensional velocity component matrix of the vehicle's geometric centroid. A state transition matrix is ​​constructed based on the uniformly accelerated linear kinematics equations. The posterior state vector estimate from the previous time scan cycle is extracted and multiplied with the state transition matrix to derive the prior state vector estimate for the current scan cycle. The prediction error covariance matrix is ​​updated using the state transition matrix and the process noise covariance matrix. The vehicle geometric centroid coordinates obtained from clustering within the current scan cycle are set as the observation vector. The difference matrix between the observation vector and the product of the observation matrix and the prior state vector is calculated. The Kalman gain matrix is ​​derived by inverting the prediction error covariance matrix and the observation noise covariance matrix. The compensation vector is obtained by multiplying the Kalman gain matrix and the difference matrix, and then superimposed onto the prior state vector estimate to obtain the posterior state vector for the current cycle. Extract the coordinates of the posterior state vectors carrying the same tracking identifier code from each iteration, and concatenate them into continuous line segment data according to the time sequence to form the radar trajectory data.

[0080] In this embodiment, as Figure 3 As shown, step S20, which involves comparing the visual perception data with the control state data to generate a basic operating state, includes:

[0081] Step S21: Analyze the color saturation and pixel brightness in the real-time image sequence to determine the apparent luminescence state. This involves reading the pixel array data within the polygonal region of interest. Performing color space conversion algebraic operations. Reading the red, green, and blue component values ​​of each pixel and normalizing them to a preset scale. Extracting the maximum and minimum values ​​from the three components. Using a specific mapping equation, converting the normalized components into hue, saturation, and brightness parameters. Setting the reference hue parameter determination range for the target traffic light type.

[0082] Specifically, the particular mapping equation is a conventional RGB to HSV color space non-linear conversion model. To accurately match the optical emission wavelength specifications of traffic light light-emitting diodes (LEDs), the hue (H) angle is used. For example: when the target being detected is a red light, its baseline hue parameter judgment interval is mathematically strictly defined as follows: The green light determination interval is defined as follows: Meanwhile, in order to effectively eliminate low-purity visual artifacts caused by road surface water reflection and astigmatism in rainy weather from an optical perspective, the color judgment benchmark value (saturation) mentioned later is... The lower limit of the component is set to the normalized range. The lightness determination benchmark value (lightness) The lower limit of the component is set to .

[0083] Traverse all pixels within the region of interest. If the hue parameter value of a pixel falls within the reference hue parameter determination range, mark that pixel as a candidate pixel. Sum the saturation parameters of all candidate pixels. If the total number of candidate pixels is greater than zero, divide by the total number of candidate pixels to obtain the average as the color saturation. Sum the brightness parameters of all candidate pixels and divide by the total number of candidate pixels to obtain the average as the pixel brightness. Preset color and brightness determination reference values. Compare the calculated color saturation and pixel brightness with the reference values. If the color saturation is greater than the color determination reference value, and the pixel brightness is greater than the brightness determination reference value, set the apparent luminance status flag to a high logic bit in the status register to indicate illumination. If either parameter is less than or equal to the corresponding reference value, write the apparent luminance status flag to a low logic level to indicate extinguishing.

[0084] Step S22: Analyze the high and low logic level changes in the phase sequence level signal to determine the physical drive state. This can be understood as retrieving the phase sequence level signal data segment corresponding to the same time span as the real-time image sequence based on a unified timestamp reference. Traverse the logic status words within this data segment. If the main logic status word in the data segment is high, it indicates that the solid-state relay at the control cabinet has performed a closed conduction physical action; write the physical drive state flag bit to a high logic level in the status register to indicate power-on. If the logic status word in the data segment is low, it indicates that the power supply drive circuit is in a cut-off stop state; write the physical drive state flag bit to a low logic level to indicate power-off.

[0085] Step S23: Evaluate the matching degree between the apparent light emission state and the physical driving state, and use the evaluation result as the basic operating state. It is understood that the extracted apparent light emission state flag and physical driving state flag are input into a hardware XOR logic gate. Further, the evaluation operation includes: determining a false positive light emission conflict when the apparent light emission state indicates light emission and the physical driving state indicates power off; determining a blocking power-off conflict when the apparent light emission state indicates off and the physical driving state indicates power on; the false positive light emission conflict and the blocking power-off conflict are collectively referred to as the state conflict. Specifically, when the XOR logic gate outputs a true value and the input data at both ends is a combination indicating light emission and power off, the physical manifestation is that an external ambient light source directly illuminates the surface of the signal light lens and produces specular reflection. The image sensor and processing algorithm capture bright spot pixels that match the characteristics, while the control pin is actually in a zero-voltage, unpowered state. The system generates a false positive light emission conflict identification code accordingly. When the XOR logic gate outputs a true value and the input data at both ends combines to indicate an off state and an on state, the physical manifestation is that the control motherboard normally outputs a high level to close the relay and energize it. However, due to large obstacles on the road obstructing the optical propagation path or physical open circuit damage to the LED panel inside the signal light, the image sensor fails to acquire the optical signal. Based on this, the system generates an obstruction / power failure conflict identification code. The existence of such a conflict identification code indicates a substantial deviation in the data flow between the sensing module and the control module, collectively referred to as a state conflict.

[0086] In this embodiment, combined with Figure 4 The logical branch shown, step S30, involves extracting the group kinematic features of associated vehicles from the radar trajectory data, including: selecting a set of associated vehicles in the deceleration range from the radar trajectory data; calculating the mean longitudinal deceleration and braking-to-stop ratio of each vehicle in the set of associated vehicles; and combining the mean longitudinal deceleration and the braking-to-stop ratio to form the group kinematic features.

[0087] Understandably, the pre-configured road network node coordinate data table is read, and a rectangular deceleration zone is divided within the lane area based on the longitudinal distance threshold from the physical entity of the stop line. Specifically, combining the conventional speed limit standards for urban arterial roads and the physical common sense of vehicle safe braking distance, the longitudinal distance threshold is quantitatively configured as a physical length range of 50m to 80m extending backward from the stop line. Radar trajectory data in memory is read through a traversal operation. The trajectory coordinate sequence is extracted. When the first coordinate point of a vehicle's trajectory crosses the initial cross-section of the deceleration zone, the instantaneous velocity scalar value of the vehicle in the subsequent time sequence is extracted at a fixed sampling frequency. A first-order difference calculation operation is performed on the instantaneous velocity value sequence, using the velocity difference between adjacent time frames divided by the sampling interval time constant to obtain the corresponding instantaneous acceleration sequence. The algebraic sign characteristics of this instantaneous acceleration sequence are scanned. If the instantaneous acceleration value is negative within a preset continuous evaluation window, it objectively indicates that the vehicle physical entity is performing a braking deceleration action. To avoid misjudgments caused by minor high-frequency fluctuations in the radar speed measurement system, the preset continuous evaluation window is explicitly limited in time to a span of 3 to 5 consecutive radar time scan cycles (equivalent to a physical time span of 0.3s to 0.5s). All vehicle trajectory objects that meet the position constraints of entering the deceleration zone and have negative acceleration characteristics are merged to construct an index table for the associated vehicle set.

[0088] Further, when calculating the mean longitudinal deceleration, the instantaneous acceleration negative value data column of individual vehicle entities within the associated vehicle set is retrieved sequentially according to the index table. A definite integral algebraic operation using the trapezoidal rule is performed on this negative value data column to obtain the integral value of the speed difference as the total scalar of speed change. The total scalar of speed change is divided by the absolute time span of the braking process to obtain the average deceleration of the individual vehicle entity. The average deceleration data of all vehicle entities within the set is extracted iteratively, and an algebraic accumulation operation is performed. The accumulated sum is divided by the total number of vehicle entity records included in the set to calculate the mean longitudinal deceleration. For the braking stop ratio data, a small speed judgment constant and a parking holding time benchmark are set. Specifically, the small speed judgment constant is physically calibrated based on the radar sensor noise floor Doppler frequency shift speed measurement tolerance, and its value range is set as follows: to The parking maintenance time reference is set based on the minimum red light protection phase time constant of the target intersection, and its value range is set as follows: to The aforementioned boundaries are used to filter out environmental noise disturbances and occasional short-term pauses in vehicle following at the physical level, thereby confirming the actual parking waiting state controlled by traffic lights. The instantaneous speed records of each vehicle object within the set are read. If the instantaneous speed parameter of a vehicle object drops below a small speed judgment constant, and its dwell time within the corresponding coordinate area exceeds the parking maintenance time benchmark, it is marked as having completed braking and stopping in the vehicle attribute field. At the end of the evaluation period, the total number of vehicle objects marked as having completed braking and stopping in the statistical attribute field is counted. This total number is divided by the total number of all vehicles recorded within the associated vehicle set, and the braking and stopping ratio is calculated by performing division. Feature data block memory is allocated in main memory, and the average longitudinal deceleration floating-point data and the braking and stopping ratio floating-point data are sequentially written into the corresponding memory addresses to complete the data encapsulation and combination of the group kinematic feature quantities.

[0089] Furthermore, the step of using the group kinematic characteristics to perform closed-loop reverse deduction of the state conflict to distinguish between environmental interference false alarms and real physical faults includes: when the state conflict manifests as the apparent light emission state indicator being lit and the physical drive state indicator being de-energized, determining whether the braking stop ratio exceeds a preset parking threshold, and evaluating whether the average longitudinal deceleration conforms to the red light braking model; if both conform, then the target traffic light is deduced to currently exhibit a red light attribute, and the current state conflict is marked as an environmental interference false alarm induced by strong light reflection.

[0090] Understandably, the processor loads a preset mathematical model for assessing the physical condition of vehicles braking at red lights. This model, based on preset parameters of road surface friction coefficient and gravitational acceleration constant, sets the braking boundary equations. It reads the calculated braking stopping ratio constant and compares it with a preset parking threshold constant stored in the system. If the braking stopping ratio constant is greater than the preset parking threshold constant, it confirms a large-scale queuing and stopping behavior on the road surface. The calculated mean longitudinal deceleration parameter is then substituted into the mathematical model for data verification. The model extracts the overall initial average speed of the vehicle group entering the deceleration zone and the remaining physical distance to the stop line, outputting the calculated upper and lower limits of the theoretical safe braking deceleration.

[0091] Specifically, the steps for deriving the upper and lower limits of the theoretical deceleration range for safe braking strictly follow the laws of classical physics: let the extracted overall initial average velocity be... The remaining physical distance parameter to the stop line is Based on the fundamental uniformly decelerated linear kinematics theorem, the theoretical reference deceleration for maintaining a smooth stop at the stop line is: Based on the preset road surface friction coefficient parameter (e.g., dry asphalt pavement) ) and the gravitational acceleration constant (Pick ), calculate the maximum deceleration boundary constrained by the physical adhesion limit of the road surface. To accommodate differences in braking habits among drivers, a lower limit factor for braking tolerance is introduced. (Values) to (between) and upper limit factor (Values) to (between). Accordingly, the theoretical lower limit of deceleration is precisely set as The theoretical upper limit of deceleration is set as follows: This constitutes a precise closed interval. Furthermore, the preset parking threshold constant used to diagnose group physical parking behavior is established based on a large-sample macro-level statistical analysis of historical red light cycles at intersections, and its specific value range is clearly defined as follows: to .

[0092] If the mean longitudinal deceleration parameter is within the theoretical deceleration range, the system logic output conforms to the judgment result of the red light braking model. When both data judgment conditions are true, based on the macroscopic physical response data of traffic flow damping control, the optical signal radiated by the target traffic light to the outside is actually calculated at the logic layer to present a red light attribute prohibiting passage. Superimposed with the fact that the underlying phase sequence level signal obtained above is clearly in a low-level, unpowered state, the entire data deduction process achieves a logical closed loop. It is inferred that the luminous image pixels captured by the video acquisition terminal originate from the specular reflection physical phenomenon of external natural light on the polycarbonate lens shell of the traffic light. The system updates the conflict record in the log and marks the current state conflict field as the environmental interference false alarm.

[0093] Furthermore, the method of using the group kinematic characteristics to perform closed-loop reverse deduction of the state conflict to distinguish between environmental interference false alarms and real physical faults also includes: if the braking stopping ratio does not exceed a preset parking threshold, or the average longitudinal deceleration does not conform to the red light braking model, reverse deducing that the target traffic light is currently showing a green light attribute; determining that the group kinematic characteristics contradict the environmental interference false alarm logic; and in response to the contradictory state, confirming that the target traffic light has experienced the real physical fault and generating a hardware fault alarm command.

[0094] Understandably, in the comparison calculation branch, if the extracted braking stopping ratio constant is not greater than the preset parking threshold constant, and the mean longitudinal deceleration parameter fails to fall within the theoretical deceleration upper and lower limit range, it indicates that the road traffic flow exhibits kinematic data characteristics of uniform or accelerating passage through the monitored intersection. Based on the aforementioned unobstructed passage physical motion characteristics, the calculation system reverse-engineers the parameters currently presented to the road surface by the target traffic light as either a green light attribute allowing passage without a prohibition warning or an off-light state. The off-light or green light attribute result obtained from the above reverse-engineering calculation directly contradicts the apparent indication of light emission judgment result output by the image processing system at the data level. After excluding the possibility of optical interference caused by external ambient light reflection in the condition constraints, combined with the data premise that the control motherboard has clearly issued a zero-voltage power-off command, the state contradiction points to a substantial physical hardware cause: there is substantial hardware damage to the internal electrical execution circuit or light-emitting array component of the target traffic light. Specific physical manifestations include, for example, the normally open contacts of AC contactors or solid-state relays inside the control cabinet undergoing physical welding due to high temperatures, causing uncontrolled conduction and illumination, while the drive monitoring circuit normally detects the power-on control word issued by the processor. The processor responds to the contradictory state data generated by the current judgment, confirming that the device node has experienced the aforementioned physical fault. It calls the structured message generation routine, encapsulating the fault channel identification code that triggered the alarm, GPS coordinate parameters, hexadecimal contents of the status conflict register, and radar trajectory spatial coordinate sequence data into the protocol frame, generating a hardware fault alarm command and delivering it to the Ethernet interface for network push.

[0095] In this embodiment, combined with Figure 5 The judgment logic shown, step S40, involves recording the time delay feature sequence of the target signal light under different phase sequence switching directions, including: obtaining the positive state transition time when the target signal light changes from off to on, and the corresponding time when the on-phase sequence level signal goes high, and calculating the on-hinder time; obtaining the reverse state transition time when the target signal light changes from on to off, and the corresponding time when the off-phase sequence level signal goes low, and calculating the off-hinder time; combining the on-hinder time and the off-hinder time to generate the time delay feature sequence.

[0096] It is understandable that a dedicated hardware timer / counter module is configured inside the main control processor. For the control cycle where the control channel transitions from off to on, when the processor writes a logic high level to the input / output pin to perform a closing operation, the rising edge of the pin level triggers a hardware external interrupt event. The system interrupt service function directly suspends background tasks, reads the high-precision clock count value of the hardware timer / counter at this time, and stores it in a specified memory address as the time when the corresponding start-up phase sequence level signal goes high. In the visual monitoring data stream processing pipeline, the image processing algorithm iteratively extracts the average pixel brightness statistics within the polygon's region of interest. Inter-frame brightness gradient interpolation is performed for two consecutive frames of images. Considering the objective reality that the background noise intensity of the target intersection varies by orders of magnitude due to the all-weather environment (such as day-night transitions, extreme backlighting), the preset lighting action brightness reference value and the subsequent dark field noise reference threshold avoid using fixed static constants. Instead, an environment-adaptive algebraic solution mechanism is introduced: the system dynamically extracts a preset time window (e.g., continuous) before the phase sequence action occurs. The average brightness within a frame (image) is used as the current ambient noise floor constant. Then, retrieve the historical average peak brightness of the target traffic light when it is in a stable lighting state recently. Therefore, the brightness reference value for the lighting action is algebraically derived based on the midpoint rule of luminous transition. The aforementioned dark field noise floor reference threshold is defined as follows: (in The tolerance constant used to prevent false triggering by thermal noise in the image sensor typically takes the following value: (Grayscale). When the average pixel brightness value of the back frame image is greater than the preset brightness reference value for the lighting action, while the average brightness value of the front frame image is less than the reference value, a linear univariate first-order equation is constructed based on the time interval constant between the two frames and the brightness difference. The subframe-level microsecond-precise timestamp, theoretically equal to the brightness reference value for the lighting action, is then solved.

[0097] The specific algebraic expression and derivation steps of the linear equation in one variable are as follows: Let the system capture timestamp of the forward frame image be... The corresponding average pixel brightness value is The timestamp of the backward frame image is The corresponding average pixel brightness value is The brightness reference value for the lighting action is set to a certain value. Based on the objective law that the brightness of a light-emitting component changes approximately linearly in the extremely short instant of illumination, an interpolation ratio equation for time and brightness is constructed. After algebraic rearrangement, the formula for directly solving the subframe-level microsecond-precise timestamp is as follows: This algebraic interpolation step effectively improves the temporal resolution of the hardware equivalent sampling. Similarly, the inverse state transition time is also derived and calculated using this algebraic equation in conjunction with the corresponding dark field noise baseline threshold.

[0098] The calculated microsecond-precise timestamp is saved as the positive state transition time. The processor performs a subtraction algebra operation, subtracting the corresponding high-level signal value from the positive state transition time value, to calculate the opening hysteresis time, representing the overall closing action time. The opening hysteresis time is composed of multiple time constants, including the buffering and queuing time of network communication packets in the switch queue, the response time of microcontroller firmware execution instructions, and the mechanical time of the relay moving and stationary armature components overcoming the static friction of the spring to perform the physical engagement and closing action.

[0099] Similarly, for the control cycle from on to off, when the processor writes a logic low level to the input / output pin to perform a power-off operation, the falling edge of the pin level triggers a system capture action, storing the acquired clock count value in a designated memory address as the time when the corresponding shutdown phase sequence level signal goes low. The image processing algorithm continuously calculates the fallback curve data of the pixel average brightness statistical parameter. When this parameter drops to a preset dark field noise floor reference threshold, the same linear interpolation equation is applied to calculate the subframe-level timestamp corresponding to the physical transient of power-off, which is saved as the time of the reverse state transition. A subtraction algebra operation is performed to subtract the corresponding time when the shutdown phase sequence level signal goes low from the value of the reverse state transition, calculating the shutdown hysteresis time representing the overall power-off release action time. Within a defined long-term observation window, multiple sets of on-time hysteresis time parameters generated by forward drive and shutdown hysteresis time parameters generated by reverse drive are continuously extracted. The extracted time parameters are paired up and written into a linear list data structure queue in memory in chronological order of occurrence, and then spliced ​​together to form the delay feature sequence including time sequence attributes.

[0100] In this embodiment, step S50, extracting the asymmetric deviation component from the time delay feature sequence and using the asymmetric deviation component to remove network synchronization errors in order to diagnose the aging level of the internal components of the target traffic light, includes: calculating the absolute value of the difference between the turn-on hysteresis time and the turn-off hysteresis time as the asymmetric deviation component; when both the turn-on hysteresis time and the turn-off hysteresis time are greater than zero and the asymmetric deviation component is lower than a preset symmetry threshold, determining that the network synchronization error exists and confirming that the internal components are in good condition; when the asymmetric deviation component is greater than or equal to the preset symmetry threshold, using the asymmetric deviation component to map the corresponding mechanical contact viscosity level, and outputting the mechanical contact viscosity level as the aging level.

[0101] It is understandable that the queuing and buffering delays caused by data packets in Ethernet communication networks at router forwarding nodes exhibit a statistically symmetrical distribution characteristic conforming to the law of large numbers in the bidirectional data interaction process. However, after experiencing frequent on / off cycles over a long lifespan, the physical hardware such as contactors and relays in the signal light cabinet undergoes degradation. This degradation is due to changes in the static friction coefficient caused by arc erosion on the metal contact surface and a decrease in the tensile constant caused by metal fatigue in the internal return spring. The aforementioned material degradation causes a deviation between the time increment of the disconnection / release mechanical phase and the time increment of the engagement / closure phase, exhibiting asymmetric physical characteristics in the time dimension. The computation module traverses and reads data entries in the delay characteristic sequence queue in the memory space. For each pair of read data, a subtraction algebra operation is performed to subtract the opening hysteresis time parameter value from the closing hysteresis time parameter value. The absolute value mathematical function is then applied to the subtraction result, and the obtained unsigned absolute value constant is defined as the asymmetric deviation component. In the algebraic difference operation model, since the transmission time fluctuations of network data packets in bidirectional communication links exhibit symmetrical superposition data characteristics, the equivalent symmetrical network synchronization error constant superimposed on the delay parameters at both ends after subtraction is canceled out by algebraic principles. The calculated and retained asymmetrical deviation component parameters characterize the actual mechanical response time difference between the physical damping of the internal hardware of the traffic light electrical control circuit when performing a closing action and the physical damping of performing a opening and releasing action.

[0102] The system calls a numerical detection routine to verify whether the sign attribute of both the on and off hysteresis parameters is a positive real number greater than zero. If the sign verification passes, the calculated asymmetric deviation component value is compared with a preset symmetry threshold constant pre-programmed into memory. If the asymmetric deviation component value is lower than the preset symmetry threshold constant, the data characteristics indicate that the time fluctuation component of the system's action delay is controlled by network data packet queuing jitter, and no significant parameter offset is observed in the symmetry of the opening and closing physical actions of the internal relay execution component. Based on this, the system generates a judgment result word, identifies the data state with the network synchronization error in the register, and confirms that the current physical electromechanical operating state of the output internal component remains healthy. If the asymmetric deviation component value is greater than or equal to the preset symmetry threshold constant, it indicates that a unidirectional mechanical resistance increment physical phenomenon has occurred within the hardware component. The main processor loads a two-dimensional degradation mapping matrix lookup table file from the flash memory area. This two-dimensional degradation mapping matrix lookup table defines the specific mapping relationship between the discrete numerical range of the asymmetric deviation component and the mechanical status code.

[0103] This mapping table is constructed based on a defined physical time threshold ladder. First, the preset symmetrical threshold is physically calibrated based on the maximum standard deviation of the bidirectional round-trip delay of Network Time Protocol (NTP) messages under network idle conditions, and is set as a reference constant. (The typical value of its embodiment is) The specific mapping determination rules and discrete numerical ranges are as follows:

[0104] When the asymmetric deviation component satisfy When the logic judgment falls into the first interval, the mechanical state code extracted by mapping is the first level (such as 0x01), and the corresponding output aging level is mild viscous, which accurately characterizes the initial metal fatigue of the reset spring of the bottom control relay, resulting in a slight increase in release damping.

[0105] When the asymmetric deviation component satisfy When the logic judgment falls into the second interval, the mechanical state code extracted by mapping is the second level (such as 0x02), and the corresponding output of the aging level is moderate viscosity, which indicates that obvious arc ablation and oxidation viscosity have appeared on the surface of the mechanical contact.

[0106] When the asymmetric deviation component satisfy When the logic judgment falls into the third interval, the extracted mechanical state code is at the third level (e.g., 0x03), and the corresponding aging level is output as severe viscosity, indicating that the contact faces a serious risk of irreversible physical welding failure. Relying on the above-mentioned discrete numerical judgment and electromechanical degradation characteristic correlation table, the system accurately transforms the time deviation characteristics into objective hardware diagnostic results.

[0107] The calculated current asymmetric deviation component parameters are used as input variables. A step-by-step conditional comparison and addressing is performed in the matrix lookup table to extract the status identifier code stored in the corresponding physical address, which is defined as the mechanical contact viscosity level. The main processor calls the format conversion protocol stack to fill the hexadecimal identifier code representing the mechanical contact viscosity level into a data packet structure of the specified format, converting it into the aging level data frame. This aging level data frame is periodically submitted to the disk controller for persistent writing to the health record sector of the internal storage medium.

[0108] like Figure 6As shown, the hardware support system for executing the above data verification and judgment logic and matrix calculation flow steps adopts a multi-core master-slave physical architecture. The edge computing device substrate is equipped with a main control central processing unit chip, a digital signal coprocessor chip, and a field-programmable gate array (FPGA) chip. Video interface data from external camera devices is directly routed to the FPGA chip. The FPGA chip uses a hardware description language to construct multiple parallel computing pipelines, performing color space matrix mapping multiplication operations on the incoming video pixel data array. The intermediate frequency analog signal output from the radar RF front-end, after high-frequency sampling and analog-to-digital conversion, forms a digital matrix which is then imported into the digital signal coprocessor chip. This coprocessor chip is specifically designed for complex multiplication and addition operations, performing high-frequency fast Fourier transforms to analyze the spectrum, extracting coordinate scatter data, and storing it in a shared memory block between chips. The main control central processing unit chip is equipped with a memory management unit. Under the time-slice round-robin scheduling of the microkernel real-time operating system, it pulls data from each modal sensor in the shared memory block through the on-chip high-speed communication bus, and then performs high-order control logic operations such as Kalman filter state matrix iterative estimation, nonlinear system state parameter deduction and comparison decision.

[0109] For disk storage management of system data, edge computing devices integrate industrial-grade solid-state storage components with power-down protection management circuits. When the onboard voltage monitoring circuit detects that the power supply bus voltage has dropped below the preset operating voltage reference, the power-down sustaining capacitor array releases charge energy to the controller and storage chips. The solid-state storage control chip loads a dynamic wear leveling algorithm management module, which uses the time window of backup power supply to query the physical erase / write count record table of flash memory blocks. The aging level data frame parameters and high-level dispatch instruction message status words remaining in the dynamic random access buffer are redirected and routed to non-volatile flash gate blocks with low physical erase / write count parameters in the record table to perform fixed write operations. This low-level read / write scheduling mechanism helps ensure the traceability and integrity of the system's low-level state data throughout its lifecycle.

[0110] The instruction steps for implementing the aforementioned control method embodiment are compiled into binary machine opcode using a high-level programming language. The machine opcode is stored in a non-volatile computer-readable storage chip. After the device is powered on and reset, the instruction fetching unit of the main control CPU chip loads the binary machine opcode into the on-chip cache memory via the memory bus. Driven by a clock signal, the main control CPU chip sequentially executes instruction fetching, decoding, and physical cycle loops. It controls the arithmetic logic unit, shift registers, comparators, and other low-level circuit gates to complete objective calculation processes such as multimodal data array mapping, Boolean algebraic logic state determination, kinematic constant extraction and accumulation, closed-loop mathematical equation verification, and absolute difference calculation. All mentioned data flow changes and status bit determination operations are precisely implemented and supported at the physical level through the on / off state transitions of hardware transistors.

[0111] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for intelligent diagnosis of traffic lights based on multimodal perception, comprising: Acquire visual perception data, control status data, and radar trajectory data of the target signal lights; By comparing the visual perception data with the control status data, a basic operating status is generated; The method for intelligent diagnosis of traffic lights based on multimodal perception is characterized in that it further includes: When the basic operating state indicates that the visual perception data and the control state data are inconsistent and present a state conflict, the group kinematic features of the associated vehicles in the radar trajectory data are extracted, and the group kinematic features are used to perform closed-loop reverse deduction of the state conflict in order to distinguish between environmental interference false alarms and real physical faults. Record the time delay characteristic sequence of the target signal light under different phase sequence switching directions; The asymmetric deviation component in the time delay feature sequence is extracted, and the asymmetric deviation component is used to remove network synchronization error in order to diagnose the aging level of the internal components of the target signal light.

2. The intelligent diagnostic method for traffic lights based on multimodal perception according to claim 1, characterized in that, The acquisition of visual perception data, control status data, and radar trajectory data of the target signal light includes: A real-time image sequence covering the luminous area of ​​the target traffic light is acquired as the visual perception data; The phase sequence level signal output by the control motherboard corresponding to the target signal light is received as the control status data; The radar scans the stop line area at the intersection where the traffic light facing the target is located to obtain a set of millimeter-wave point clouds, which serves as the radar trajectory data.

3. The intelligent diagnostic method for traffic lights based on multimodal perception according to claim 2, characterized in that, The comparison of the visual perception data with the control state data to generate a basic operating state includes: Analyze the color saturation and pixel brightness in the real-time image sequence to determine the apparent luminescence state; Analyze the high and low logic level changes in the phase sequence level signal to determine the physical drive state; The matching degree between the apparent luminescence state and the physical driving state is evaluated, and the evaluation result is used as the basic operating state.

4. The intelligent diagnostic method for traffic lights based on multimodal perception according to claim 3, characterized in that, The evaluation of the matching degree between the apparent luminescence state and the physical driving state includes: When the apparent light emission state indicates light emission and the physical drive state indicates power failure, it is determined that there is a false positive light emission conflict. When the apparent light emission state indicator is off and the physical drive state indicator is on, it is determined that there is a blocking power-off conflict. The false positive emission conflict and the occlusion power-off conflict are collectively referred to as the state conflict.

5. The intelligent diagnostic method for traffic lights based on multimodal perception according to claim 2, characterized in that, The scanning direction is directed towards the stop line area of ​​the intersection where the target traffic light is located to obtain a millimeter-wave point cloud set, which serves as the radar trajectory data, including: Filter static background noise in the millimeter-wave point cloud set; Lock onto the target vehicle queue that is in a dynamic approaching state; The continuous spatial coordinates of the target vehicle convoy are integrated into the radar trajectory data.

6. The intelligent diagnostic method for traffic lights based on multimodal perception according to claim 3, characterized in that, The extraction of group kinematic features of associated vehicles from the radar trajectory data includes: The set of associated vehicles in the deceleration zone is selected from the radar trajectory data; Calculate the mean longitudinal deceleration and braking stopping ratio of each vehicle in the associated vehicle set; The mean longitudinal deceleration and the braking stopping ratio are combined to form the group kinematic characteristic.

7. The intelligent diagnostic method for traffic lights based on multimodal perception according to claim 6, characterized in that, The method of using the group kinematic characteristics to perform closed-loop reverse deduction of the state conflict to distinguish between environmental disturbance false alarms and real physical faults includes: When the state conflict manifests as the apparent light-emitting state indicator being lit and the physical drive state indicator being de-energized, it is determined whether the braking stop ratio exceeds a preset parking threshold, and whether the average longitudinal deceleration value conforms to the red light braking model. If all conditions are met, the target traffic light is deduced to be currently red, and the current state conflict is marked as a false alarm caused by environmental interference induced by strong light reflection.

8. The intelligent diagnostic method for traffic lights based on multimodal perception according to claim 7, characterized in that, The method of using the group kinematic characteristics to perform closed-loop reverse deduction of the state conflict to distinguish between environmental disturbance false alarms and real physical faults also includes: If the braking stop ratio does not exceed the preset parking threshold, or the average longitudinal deceleration does not conform to the red light braking model, the target traffic light is deduced to currently be a green light. The determination is that the group kinematic characteristics contradict the environmental interference false alarm logic; In response to the contradictory state, the system confirms that the target signal light has experienced a real physical fault and generates a hardware fault alarm command.

9. The intelligent diagnostic method for traffic lights based on multimodal perception according to claim 1, characterized in that, The recording of the time delay characteristic sequence of the target signal light under different phase sequence switching directions includes: The positive state transition time when the target signal light changes from off to on, and the corresponding time when the start-up phase sequence level signal goes high are obtained, and the start-up hysteresis time is calculated. The reverse state transition time when the target signal light changes from lit to off is obtained, and the corresponding time when the shutdown phase sequence level signal goes low is obtained, and the shutdown hysteresis time is calculated. The delay feature sequence is generated by combining the turn-on hysteresis time and the turn-off hysteresis time.

10. The intelligent diagnostic method for traffic lights based on multimodal perception according to claim 9, characterized in that, The step of extracting the asymmetric deviation component from the time delay feature sequence and using the asymmetric deviation component to remove network synchronization errors in order to diagnose the aging level of the internal components of the target traffic light includes: Calculate the absolute value of the difference between the turn-on hysteresis time and the turn-off hysteresis time, and use it as the asymmetric deviation component; When both the start-up hysteresis time and the stop-down hysteresis time are greater than zero and the asymmetric deviation component is lower than a preset symmetry threshold, it is determined that the network synchronization error exists and the internal component status is confirmed to be healthy. When the asymmetric deviation component is greater than or equal to a preset symmetry threshold, the mechanical contact viscosity level is mapped to the corresponding mechanical contact viscosity level using the asymmetric deviation component, and the mechanical contact viscosity level is output as the aging level.

Images