A high-speed multi-type event detection misjudgment correction method based on LoRA

Abstract

This application discloses a method for correcting misjudgments in high-speed multi-type event detection based on LoRA, relating to the field of intelligent transportation technology. The method includes: acquiring real-time event information on the highway, identifying it using the YOLO algorithm, and outputting the original preliminary event detection results and environmental parameters; standardizing the preliminary detection results and extracting semantic and scene features of the event targets from the preliminary detection results to obtain standard preliminary detection results; performing secondary verification and misjudgment correction using a LoRA-optimized large model based on the standard preliminary detection results to obtain the event detection results; uploading the event detection results to the road segment central monitoring platform, and retaining misjudged, missed, and new scene samples for iterative optimization. This application aims to resolve the contradiction between the low reliability of event detection in complex highway environments and the inability to deploy high-precision large models at low cost and high efficiency under existing hardware conditions.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a method for correcting misjudgments in high-speed multi-type event detection based on LoRA. Background Technology

[0002] Current highway incident monitoring systems generally use the hardware-integrated YOLO algorithm for event recognition. This algorithm acquires and recognizes images at a frequency of 25 frames per second, achieving a high level of accuracy in event detection under ideal conditions. However, in actual large-scale deployments, the accuracy of the YOLO algorithm drops significantly due to interference from various external factors, failing to meet the high-precision and high-reliability requirements of highway incident monitoring. Existing technologies do not incorporate large-scale models optimized by LoRA, resulting in the following technical shortcomings: External interference leads to a decrease in detection accuracy: Environmental factors such as strong glare, backlight, nighttime vehicle lights, rain, fog, and dust in highway scenarios can interfere with the image recognition accuracy of the YOLO algorithm, resulting in missed detection of real events and false alarms of invalid events, which affects the emergency response efficiency of the road section center.

[0003] Lack of a professional secondary verification mechanism: In existing technologies, the YOLO algorithm outputs results directly to report alarms without setting up an effective secondary verification process, making it impossible to correct misjudgments and missed detections; it also fails to introduce the semantic understanding and logical reasoning capabilities of large models, making it difficult to distinguish similar scenarios and unable to improve detection accuracy from the root.

[0004] Traditional large-scale model training and deployment costs are high: existing event detection large-scale models do not use gradient optimization technology, resulting in high gradient values ​​during training, low training efficiency, high resource consumption, high deployment difficulty, and difficulty in adapting to existing highway hardware equipment, making it impossible to scale up.

[0005] To address the aforementioned shortcomings, this invention proposes a secondary verification system and method for highway event detection based on a LoRA-optimized large model. By relying on a large model with gradients optimized by the LoRA algorithm, the original event detection results are re-verified, misjudgments are corrected, and the event detection accuracy is improved. This fills the technical gap in the prior art where there is no professional secondary verification and the training and deployment costs of large models are high. Summary of the Invention

[0006] This invention provides a high-speed multi-type event detection misjudgment correction method based on LoRA, including: The system acquires real-time event information on highways, identifies events using the YOLO algorithm, and outputs preliminary event detection results and environmental parameters. The preliminary detection results are standardized, and the semantic features and scene features of the event targets in the preliminary detection results are extracted to obtain standard preliminary detection results; Based on the initial standard detection results, the LoRA-optimized large model is used for secondary verification and misjudgment correction to obtain the event detection results; The event detection results are uploaded to the road section central monitoring platform, and samples of misjudgments, missed detections, and new scenarios are retained for iterative optimization.

[0007] The aforementioned method for correcting misjudgments in high-speed multi-type event detection based on LoRA acquires real-time event information on highways, identifies events using the YOLO algorithm, and outputs preliminary event detection results and environmental parameters, including: Real-time video streams are collected from existing roadside monitoring equipment on highways and transmitted to edge computing units with built-in YOLO target detection algorithms. The YOLO algorithm in the edge computing unit performs frame-by-frame target detection and event classification on the video stream, generating preliminary detection results.

[0008] The aforementioned high-speed multi-type event detection misjudgment correction method based on LoRA uses the YOLO algorithm in the edge computing unit to perform frame-by-frame target detection and event classification on the video stream, generating preliminary detection results, including: Full-image feature extraction and multi-scale target detection are performed on each frame of the image; Events are classified and judged based on the detected target attributes and scene status; Simultaneously extract environmental parameter information associated with the event frame.

[0009] The aforementioned high-speed multi-type event detection misjudgment correction method based on LoRA standardizes the preliminary detection results and extracts the semantic features and scene features of the event targets from the preliminary detection results to obtain standard preliminary detection results, including: The preliminary test result data package is cleaned and its validity is verified to remove invalid data. Semantic features are extracted from valid data to obtain visual semantic information of the event target; Scene features are extracted from all retained valid image frames to obtain complete environmental context information of the scene where the event occurred. The aforementioned high-speed multi-type event detection misjudgment correction method based on LoRA, according to the initial standard detection results, performs secondary verification and misjudgment correction through LoRA optimization of a large model to obtain the event detection results, including: The sequence of semantic feature vectors from multiple frames is mapped to an adaptive manifold space, and the temporal semantic stability of events is calculated. A scenario-constrained manifold is constructed based on an event-level comprehensive stability score, and misjudgments are corrected by the reachability of the target semantic representation projected onto the manifold.

[0010] The aforementioned high-speed multi-type event detection misjudgment correction method based on LoRA maps a multi-frame semantic feature vector sequence to an adaptive manifold space and calculates the temporal semantic stability of events, including: Based on the initial event detection type and scene environment parameters, a dedicated manifold basis for the event is dynamically constructed. Multi-frame semantic feature vectors are mapped to a high-dimensional event manifold space using an event-specific manifold basis. In the event manifold space, instantaneous semantic stability metric and event-level comprehensive stability score are calculated to determine the authenticity of the event.

[0011] The aforementioned high-speed multi-type event detection misjudgment correction method based on LoRA constructs a scene-constrained manifold based on an event-level comprehensive stability score, and corrects misjudgments by projecting the target semantic representation onto the manifold. The method includes: Based on scene feature vectors and environmental parameters, construct the physical reachable areas of each type of event in the current scene; Calculate the physical projection distance from the semantic feature vector of the event target to the physically reachable region of each category; Misjudgment is corrected based on physical projection distance, and the corrected event type is output.

[0012] The beneficial effects achieved by this invention are as follows: The large-scale model's secondary verification significantly improves the accuracy of original event detection, meeting the high-precision monitoring needs of highways; it accurately distinguishes similar scenarios, effectively corrects misjudgments, and reduces maintenance workload; LoRA optimization reduces resource consumption, adapts to existing hardware, and reduces deployment and iteration costs; the large-scale model's response speed matches the YOLO algorithm, ensuring real-time event detection and handling; it supports incremental fine-tuning, continuously supplementing samples to improve model scenario adaptability and reduce maintenance costs. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0014] Figure 1 This is a flowchart of a high-speed multi-type event detection misjudgment correction method based on LoRA provided in Embodiment 1 of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Example 1 like Figure 1 As shown, Embodiment 1 of this application provides a high-speed multi-type event detection misjudgment correction method based on LoRA, including: S1: Obtain real-time event information on the highway, identify it using the YOLO algorithm, and output the preliminary detection results of the original events and environmental parameters; The process of acquiring real-time event information on highways, identifying it using the YOLO algorithm, and outputting preliminary event detection results and environmental parameters includes the following sub-steps: S11: Collect real-time video streams through existing roadside monitoring equipment on highways and transmit them to an edge computing unit with built-in YOLO target detection algorithm; The existing highway roadside monitoring equipment, which has been deployed on a large scale, is used as the front-end sensing unit. The roadside monitoring equipment typically includes high-definition bullet cameras, dome cameras, or pan-tilt cameras, which continuously cover key monitoring sections such as the main line, ramps, and tunnel entrances and exits in fixed positions or patrol mode.

[0017] The camera captures real-time video streams of the road at a rate of no less than 25 frames per second, and pushes the raw video streams to nearby edge computing units in real time via a fiber optic ring network or a 5G / 4G wireless transmission link.

[0018] We collect surveillance videos from different road sections, time periods, and weather conditions. After frame extraction, we annotate the bounding boxes of targets such as vehicles, pedestrians, and spilled objects in the images. At the same time, we label the event types and divide them into training, validation, and test sets according to the proportions.

[0019] Deploy the PyTorch deep learning framework and CUDA computing library on a server equipped with GPUs, load YOLO pre-trained weights, and fine-tune the training using the dataset until the loss function converges and the average accuracy of the model on the validation set reaches the preset standard.

[0020] The model performance is evaluated using a test set, the trained model is exported in an inference format supported by the edge computing unit, and the corresponding model configuration file is prepared.

[0021] The edge computing unit is deployed in the road section equipment room or roadside integrated cabinet, and has a pre-configured YOLO target detection algorithm model built in, serving as the first-level event detection engine of this invention.

[0022] S12: The YOLO algorithm in the edge computing unit performs frame-by-frame target detection and event classification on the video stream to generate preliminary detection results; The process involves using the YOLO algorithm in the edge computing unit to perform frame-by-frame target detection and event classification on the video stream, generating preliminary detection results. This includes the following sub-steps: S121: Perform full-image feature extraction and multi-scale target detection for each frame of the image; First, the YOLO algorithm is used to divide each frame of the input image into S×S grid cells. Each grid cell is responsible for predicting the bounding box of the target falling into it and the probability of its class.

[0023] By extracting multi-level features from images through the backbone structure of a convolutional neural network, target bounding box regression and classification are performed at multiple feature map scales, thereby enabling the detection of targets of different sizes.

[0024] In the highway scenario to which this invention applies, the YOLO algorithm is configured to identify conventional and unusual target types such as vehicles, pedestrians, non-motorized vehicles, debris, animals, traffic cones, and accident vehicles.

[0025] S122: Classify and determine events based on detected target attributes and scene status; After completing target detection, the YOLO algorithm processes the detection results as events according to predefined event judgment rules.

[0026] When a detected target meets the preset event triggering conditions, a corresponding event record is generated. Typical event types include, but are not limited to: illegal parking events, pedestrian intrusion events, vehicle driving in the wrong direction events, traffic congestion events, littering events, vehicle leaving the driving lane events, and abnormal speed events.

[0027] The event determination rules take into account factors such as the bounding box coordinates of the target, the target category, changes in motion trajectory across frames, and the spatial relationship between multiple targets within the scene.

[0028] S123: Synchronously extract environmental parameter information associated with the event frame.

[0029] While generating event logs, environmental parameters associated with the current event frame are simultaneously extracted using the YOLO algorithm. These environmental parameters come from two sources: firstly, parameters directly evaluated from video frames using image analysis algorithms, such as light intensity assessment values, visibility assessment values, and rain / fog level judgment values; secondly, parameters obtained through linkage with roadside meteorological monitoring equipment, such as ambient illuminance count values, rainfall, wind speed, and temperature and humidity data output by meteorological sensors.

[0030] Subsequently, a structured preliminary detection result data packet is output for each event. This data packet contains at least the following information fields: event unique identifier, event type, trigger timestamp, event duration, detection confidence, key image frame sequence that triggered the event, target bounding box coordinates, number of targets, target movement speed and direction, current road segment location information, and associated environmental parameters.

[0031] The unique event identifier is used to link the same event records in subsequent verification and correction stages; the key image frame sequence includes at least the event trigger frame, the event peak frame, and the event clearing frame, providing visual material for subsequent time-series analysis by the large model; the detection confidence is used as a deterministic measure of the detection result by the YOLO algorithm to assist the large model in determining whether low-confidence events need to be verified in detail.

[0032] After generating the initial detection result data packet, the data packet is encapsulated in a standardized JSON or Protocol Buffers format via an internal high-speed local area network or message queue mechanism, and pushed to the data preprocessing layer via HTTP / gRPC protocol or message middleware.

[0033] The transmission process employs an asynchronous, non-blocking mode to ensure that the real-time detection rhythm of the YOLO algorithm is not affected by the latency of subsequent processing links. Simultaneously, a data verification mechanism is implemented in the transmission link to verify the integrity of data packets. Abnormal data packets lacking critical fields or failing verification are marked and written to the transmission log, ensuring traceability throughout the entire data flow.

[0034] S2: Standardize the preliminary detection results and extract the semantic and scene features of the event targets from the preliminary detection results to obtain standard preliminary detection results; The process involves standardizing the preliminary detection results and extracting semantic and scene features of the event targets from the preliminary detection results to obtain standard preliminary detection results. This includes the following sub-steps: S21: Perform data cleaning and validity verification on the preliminary test result data package, and remove invalid data; After receiving the preliminary detection result data packet, the data preprocessing layer first verifies the integrity of the data packet. The verification includes: checking whether the data packet can be deserialized normally and whether the data structure is complete; checking whether the image data in the key image frame sequence can be read normally by the standard decoder, and whether there are hardware-level errors such as corrupted file headers or truncated data.

[0035] For all valid data that passes verification, pre-classification and labeling are performed based on their original detection confidence level and environmental parameters. Specifically, data with a detection confidence level higher than the high confidence threshold are labeled as "high confidence initial detection results"; data with a detection confidence level between the high and low confidence thresholds are labeled as "initial detection results pending verification"; and data with a detection confidence level lower than the low confidence threshold but still identified as an event by YOLO are labeled as "low confidence initial detection results". Simultaneously, data accompanied by complex environmental parameters such as rain, fog, backlight, and nighttime are labeled with environmental complexity. For example, a label value of "1" indicates a slightly complex environment, meaning that at least one environmental parameter deviates from the normal range but by a small margin (e.g., light fog, cloudy to overcast skies, dusk).

[0036] S22: Extract semantic features from valid data to obtain visual semantic information of the event target; For all key image frames of valid events that are fully retained, they are uniformly adjusted to the preset standard input size. During adjustment, the original aspect ratio of the image is maintained, and non-symmetrical images are padded to the standard size using edge padding. The resized image frames then undergo pixel value normalization, mapping the pixel value range to a unified interval.

[0037] All normalized key image frames are input into a pre-trained visual feature extraction model to extract high-level semantic feature vectors of the event target. The visual feature extraction model can employ a convolutional neural network backbone structure pre-trained on general visual tasks, such as the backbone network portion of ResNet, Vision Transformer, or ConvNeXt models. This model has been pre-trained on large-scale image datasets such as ImageNet and possesses the ability to extract general visual semantic features.

[0038] During feature extraction, the high-dimensional vector output by the fully connected layer or global pooling layer before the model's output layer is used as the semantic feature vector of the event target. This vector typically has dimensions of 512, 768, 1024, or 2048, depending on the structural design of the chosen backbone network. This semantic feature vector can represent high-level visual semantic information of the event target, such as its category semantics, morphological contours, texture details, and local component features.

[0039] Next, for events containing multiple key image frames, the semantic feature vector of each frame is extracted, and then temporal feature fusion is performed to obtain a fused semantic feature vector. The fusion method can be average pooling, max pooling, or self-attention weighted pooling. The temporal fusion process fully preserves the feature information of all frames, ensuring that cross-frame patterns such as abnormal trajectories and flicker detection that may be caused by misjudgment in a single frame are fully recorded.

[0040] S23: Extract scene features from all retained valid image frames to obtain complete environmental context information of the scene where the event occurred.

[0041] Global scene features are extracted from the normalized key image frames. This can be achieved by extracting the overall scene feature vector from the backbone network of a pre-trained scene classification model, or by dividing the image into blocks and extracting the statistical features of each block to construct the scene feature vector. This scene feature vector can characterize global environmental attributes such as lighting conditions, weather conditions, road structure, traffic density, and background complexity of the scene where the event occurred.

[0042] Transform environmental parameters into structured numerical feature representations.

[0043] For continuous numerical parameters such as light intensity assessment value, visibility assessment value, rainfall, and wind speed, their values ​​are directly taken and normalized to map them to a unified numerical range. For discrete or categorical parameters such as rain / fog intensity and weather type, one-hot encoding or label encoding is used to convert them into numerical vectors. The normalized continuous parameter vector is then concatenated with the encoded discrete parameter vector to form an environmental parameter feature vector with a unified dimension.

[0044] The concatenated target semantic feature vector, scene feature vector, and environmental parameter feature vector are associated and encapsulated with the structured information of the event. The structured event information includes a unique event identifier, original event type label, detection confidence level, trigger timestamp, target bounding box coordinates, road segment location information, and also retains complete classification labels (such as "low-confidence initial detection result" and "complex environment label") as well as a complete copy of the original preliminary detection result. The encapsulated data object is serialized into JSON or Protocol Buffers format to obtain the standard preliminary detection result.

[0045] S3: Based on the initial standard detection results, the LoRA-optimized large model is used for secondary verification and misjudgment correction to obtain the event detection results; Specifically, the LoRA-optimized large model serves as the core execution unit. The LoRA-optimized large model employs a pre-trained Transformer model in its basic architecture. LoRA low-rank adaptation matrices are inserted into the query projection layer, key projection layer, and value projection layer of the self-attention module in each Transformer encoder layer, as well as into the first fully connected layer of the feedforward neural network.

[0046] For the original weight matrix The LoRA low-rank adaptation matrix is ​​defined as follows: ,in For a dimension reduction matrix, For an increased-dimensional matrix, the rank much smaller (For example ).

[0047] During training, the original weights Completely frozen, only updating and The parameters; the output of the forward propagation is ,in, For example, the LoRA scaling factor, a value of During model inference, the trained... Products are combined to their corresponding original weights. In this model, the inference and computation process is completely consistent with the original model structure, without introducing any additional computational delay.

[0048] In addition to the basic LoRA adapter mentioned above, four lightweight modules specifically designed for highway event detection scenarios have been added to the LoRA optimized large model. The parameters of each module are trained and stored in the form of LoRA low-rank decomposition.

[0049] The process involves initial standard detection results, followed by secondary verification and misjudgment correction using a LoRA-optimized large model to obtain event detection results. This includes the following sub-steps: S31: Map the multi-frame semantic feature vector sequence to the adaptive manifold space and calculate the temporal semantic stability of the event; The process of mapping multi-frame semantic feature vector sequences to an adaptive manifold space and calculating the temporal semantic stability of events includes the following sub-steps: S311: Dynamically construct the exclusive manifold basis for the event based on the initial event detection type and scene environment parameters; Specifically, the raw event types determined by YOLO are read from the preliminary test results of the standard. Environmental parameter feature vector And environmental complexity markers.

[0050] Based on event type Retrieve the category semantic prototype matrix that has been learned and stored during the pre-training phase. .

[0051] The category semantic prototype matrix is ​​stored in the form of an embedded parameter table for each event category. Maintain a separate matrix. This matrix is ​​parameterized using LoRA low-rank decomposition. ,in The initialization matrix is ​​(in one implementation, the identity matrix I is used as the initialization, indicating that the manifold basis degenerates to standard Euclidean space in the absence of prior knowledge). , The LoRA decomposition matrix; For the corresponding LoRA rank (in one implementation) Only updates during training. and The parameters.

[0052] Simultaneously, an environmental modulation network is invoked. This network is a lightweight two-layer feedforward structure dedicated to generating modulation matrices from environmental parameters. The input layer receives the feature vectors of the environmental parameters. The first layer is a dimensionality reduction layer, using a weight matrix obtained through pre-training. The input is projected onto the bottleneck layer and passed through the ReLU activation function to obtain an intermediate representation; the second layer is an up-dimensional layer, using a pre-trained weight matrix. Project the bottleneck layer representation onto × The dimension is reshaped into a matrix form through a reshape operation, thus obtaining the environmental modulation matrix. The above. and All are implemented in the form of LoRA low-rank decomposition, specifically represented as .

[0053] In obtaining and Then, construct the dedicated manifold basis for this event according to the formula. The specific calculation formula is as follows: in, The basis matrix of the event-specific manifold has dimensions of × , belonging to the real number field; provides an adaptive linear transformation basis for the temporal semantic feature mapping of the current event, so that when the semantic feature vectors of multiple frames are subsequently mapped to the high-dimensional manifold space, the mapping rules are constrained by the inherent normal temporal evolution law of the event type and modulated by the real-time environmental conditions, thereby realizing differentiated processing of temporal stability measurement for different types of events and different environmental conditions. This is the category semantic prototype matrix; it encodes the basic linear transformation constraint skeleton that the semantic features of event type c should follow in the inter-frame evolution when they occur normally. Different types of real events have drastically different temporal evolution patterns. ⊙ represents element-wise multiplication (Hadamard product). and Element-wise multiplication implies that environment modulation does not involve overall scaling or linear transformation of the semantic prototype matrix, but rather applies environmental influences independently to each element position of the matrix, thus... The constraint strength at different locations can be amplified or suppressed differently according to environmental conditions, achieving a more refined adaptive adjustment capability than global modulation. For the environmental modulation matrix; To characterize the modulating effect of current environmental conditions on the evolution of temporal semantics; for example, under favorable conditions such as sunny daytime, The values ​​of each element in the middle approach 1. After element-wise multiplication, the manifold basis remains largely unchanged, dominated by the evolutionary constraints inherent in the event type; in complex environments such as rain, fog, and nighttime glare, The element values ​​corresponding to the characteristic direction will increase or decrease accordingly, making The corresponding constraint strength is modulated by environmental conditions, thereby stretching or shrinking the allowable variance range in each characteristic direction in the manifold space, avoiding misjudging normal inter-frame fluctuations caused by severe weather as timing anomalies.

[0054] The learnable residual retention coefficients are determined by optimization using the gradient descent algorithm during the model pre-training phase, with γ initialized to 0.1. For the identity matrix, the dimensions are... same; Ensure that the basis matrix is ​​always full-rank and invertible to avoid irreversible information compression caused by manifold mapping.

[0055] S312: Map multi-frame semantic feature vectors to a high-dimensional event manifold space using an event-specific manifold basis; In constructing event-specific manifold bases Subsequently, multiple frames of semantic feature vector sequences were read from the standard preliminary detection results. Where T is the number of key image frames (typically T = 3~8 frames), and the semantic feature vector of each frame is... Extracted by the visual feature extraction model pre-trained in S2.

[0056] For time semantic feature vector By mapping it to a high-dimensional event manifold space using a formula, we obtain the manifold representation vector. The specific calculation formula is as follows: in, Let be the semantic feature vector at time t. The manifold representation vector in the high-dimensional event manifold space after mapping; The basis matrix of the event-specific manifold; Let t be the original semantic feature vector at time t, i.e., the t-th frame in the multi-frame semantic feature vector sequence; The learnable nonlinear modulation coefficients are determined through gradient descent during the model pre-training phase and initialized to 0.5. The specific value is automatically learned and determined during the training phase based on the temporal pattern complexity in the actual data; ⊙ represents the Hadamard product. The sigmoid activation function has an output range of (0,1). The resulting vector of the aforementioned linear mapping, i.e., the original semantic features. event-specific manifold basis The result obtained after linear projection Dimensional vector.

[0057] The first term of the formula For a linear manifold mapping, the second term is a self-gated nonlinear enhancement term. The self-gated mechanism refers to the fact that the strength of the nonlinear enhancement is generated by the mapping result itself via a sigmoid function—when... When a certain dimension value deviates significantly from zero, When the output is close to 1, the nonlinear enhancement effect is strong; when it approaches zero, The output is close to 0.5, indicating that the mapping approaches linearity. This achieves an adaptive correlation between the degree of nonlinearity of the mapping and the position of the feature in the manifold space.

[0058] S313: Calculate the instantaneous semantic stability metric and the event-level comprehensive stability score in the event manifold space, and determine the authenticity of the event accordingly.

[0059] Obtaining multi-frame manifold representation vector sequences Then, the instantaneous semantic stability metric is calculated. ; For a moment The instantaneous semantic stability metric, quantized by the first frame manifold representation vector Deviating from its local time series trend The higher the value, the greater the suspicion that YOLO may have made a false detection due to momentary interference; It is the manifold representation vector of the t-th frame; It is an exponentially weighted moving average of the manifold representation vectors of each frame within a local time window centered at time t; The manifold representation residual vector represents the instantaneous deviation direction and magnitude between the semantic representation of the current frame and the recent temporal trend; for example, when the event is a real event, It should be smoothly located Near the vicinity, the values ​​of each dimension of the residual vector are relatively small; when the event is a false alarm caused by transient interference, It may suddenly jump out of the normal time series trend, and some dimensions of the residual vector will show large outliers; Here, is the precision matrix specific to the event, and is the covariance matrix of the normal representation distribution in the manifold space of the event. The inverse matrix, i.e. ,in, For category The feature vector matrix of the normal event time series distribution is obtained by eigenvalue decomposition from the manifold representation covariance matrix of similar real event samples during the pre-training stage. For a small, learnable network, the environment modulation matrix is ​​used. Input, Output A dimensional scaling vector, After converting to a diagonal matrix, the eigenvalues ​​are adjusted. Instantaneous semantic stability metric based on each frame The event-level comprehensive stability score is calculated using the following formula: The event-level comprehensive stability score quantifies the temporal stability of the semantic representation of an event across its entire lifecycle. The function is denoted by '-'; '-' represents the negation operation; 1 / T is the event frame normalization factor, where T is the total number of frames contained in the event, i.e., the length of the multi-frame semantic feature vector sequence; this factor is used to calculate the number of frames in each frame. The arithmetic mean of makes Unaffected by the duration of the event; For a moment The instantaneous semantic stability metric.

[0060] Will Compared with the preset verification threshold The threshold is compared by maximizing it on the validation set. The score is determined. When < If the event is determined to be a false alarm caused by transient interference in YOLO, it is filtered out, and the event data does not enter the false alarm correction process; instead, a filtered event record is directly generated. ≥ When an event passes the time-series verification, a scenario-level category correction is performed. For events categorized as having a highly complex environment, the verification threshold is... It can be adaptively reduced (e.g., by multiplying by a discount factor of 0.8) to balance the risk of missed detection in complex environments.

[0061] S32: Construct a scene-constrained manifold based on event-level comprehensive stability score, and correct misjudgments by projecting the target semantic representation onto the manifold; The process involves constructing a scene-constrained manifold based on an event-level comprehensive stability score, and correcting misjudgments by projecting the target semantic representation onto the manifold. This includes the following sub-steps: S321: Based on scene feature vectors and environmental parameters, construct the physically reachable areas of each type of event in the current scene; Extract scene feature vectors from standard preliminary test results and environmental parameter feature vector .

[0062] Will and The features are concatenated along the feature dimension and then input into the scene constraint generation network. The shared encoding layer in this network is a two-layer MLP, with the first layer's weight matrix... (Obtained through pre-training), after ReLU activation, it is input into the second layer weight matrix. Output scene latent variables This coding layer fuses scene visual information with environmental parameter information into a unified latent variable representation of the scene.

[0063] Above the shared coding layer, for each candidate event category ( Three sets of parallel output heads (center generation head, radius generation head, and direction generation head) are set up, and each set of output heads is implemented in the form of LoRA adapter.

[0064] The central generator head, through its linear projection matrix initialized during the pre-training phase, Hidden variables in the scene Perform a linear transformation; and through its initialized bias vector. The linear transformation result is translated and corrected to generate the reachable center point.

[0065] The radius generator head, through its linear projection matrix initialized during the training phase, Hidden variables in the scene A linear transformation is performed to obtain the scalar intermediate value; and its bias, initialized during the training phase, is used. After shifting and correcting the intermediate values, the output values ​​are forcibly mapped to the positive range by the softplus activation function, generating the reachable radius.

[0066] The direction generation head, through its linear projection matrix initialized during the training phase, Hidden variables in the scene Perform a linear transformation; and use its bias vector initialized during the training phase. After the transformation result is translated and corrected, the vector magnitude is forced to 1 by L2 normalization to generate the physical rationality main direction.

[0067] in, , , Parameterized in LoRA form.

[0068] Then, the reachable center points of the network output are generated using scene constraints. and reachable radius Combining the Mahalanobis distance metric matrix M output by the metric generation function, in In the dimensional semantic feature space, it will be based on Centered on the distance defined by the metric matrix M, the distance from the center does not exceed... The set of all points is determined as the category. Physically accessible area in the current scenario .

[0069] The metric generation function is a lightweight network, and its input is environmental parameters. The internal structure is a two-layer MLP; the first layer will... Project to Bottleneck layer (in one implementation) Take 64), through Activation; the second layer maps the bottleneck layer representation to The output is a lower triangular part of the Cholesky factorization factor L of M. The output is then reorganized into a lower triangular matrix L∈ Then the Mahalanobis metric matrix The parameterization using the Cholesky decomposition form ensures that M is always a positive definite matrix. All weight parameters of the metric generating function are trained using a LoRA adapter.

[0070] S322: Calculate the physical projection distance from the semantic feature vector of the event target to the physically reachable area of ​​each category; Read the fused semantic feature vector from the standard preliminary test results. According to each candidate category Physically accessible area ,calculate Physical projection distance to the area The specific calculation formula is as follows: in, For the semantic feature vector of the event target To candidate categories Physically accessible area The physical projection distance, comprehensively measuring target features and category. The degree of physical matching in the current scenario. The smaller the value, the higher the physical plausibility of the target belonging to that category; This is a function to find the maximum value. For target semantic feature vector To Category Reachable center point Mahalanobis distance; The Mahalanobis metric matrix output by the metric generator function; The fused semantic feature vector of the event target, with dimension . , is the benchmark point for distance measurement. Its Mahalanobis distance to the reachable center point of each category reflects the degree of similarity between the target features and each category at the visual semantic level. For category The reachable center point in the current scenario represents the category under the current scenario and environmental conditions. The ideal position of the semantic features of the event target in the feature space; the reachable center point of the same category is different in different scenarios. For category In the current scenario, the reachable radius will be adaptively adjusted for different categories in complex environments. For example, in rainy or foggy weather, the radius of all categories will be appropriately increased to compensate for the decrease in target feature quality caused by environmental noise. The radial overshoot distance is calculated as follows: when the Mahalanobis distance of the target feature is less than or equal to the reachable radius, it indicates that the target feature is located inside the reachable region, and the value is negative or zero; when the Mahalanobis distance of the target feature is greater than the reachable radius, it indicates that the target feature exceeds the boundary of the reachable region, and the value is positive.

[0071] The penalty coefficient is a learnable value, determined through gradient descent during model training and initialized to 0.3. The direction deviation factor has a value range of [0,1]; where For vectors ( ) and the main direction of physical rationality The angle between them; The absolute value of the cosine of the angle between the two; when the direction of change of the target feature is... When they are completely identical, =1, direction deviation factor is 0, direction deviation penalty disappears; when the target feature direction is... When orthogonal, =0, the direction deviation factor is 1, and the direction deviation penalty reaches its maximum; For category The main direction of physical plausibility in the current scenario represents the category under the current scenario conditions. The direction of feature change most likely to occur in the feature space of the event target; the main direction of physical rationality of the same category is different in different scenarios.

[0072] S323: Correct misjudgments based on physical projection distance and output the corrected event type.

[0073] After obtaining the physical projection distance of all candidate categories Then, the final event type is determined according to the minimum physical projection distance criterion; when Compared with the original YOLO classification Inconsistency indicates a misjudgment correction has occurred. Further calculation of the corrected confidence level is then performed. ,in, The minimum physical projection distance, This is the sum of the physical projection distances for all candidate categories. ,when Clearly falls within the reachable area of ​​a certain category ( When it approaches 0 and is far from other category regions, Approaching 1.

[0074] The final output includes the event type and the adjusted confidence level.

[0075] Subsequently, the processing results are integrated at the event granularity to generate a unified final event detection result data object. This data object is a structured record, containing a unique event identifier, filter tags, temporal semantic stability score, a sequence of instantaneous stability metrics for each frame, the original YOLO judgment category, the event trigger timestamp, road segment location information, the final judgment event type, a mark indicating whether misjudgment correction has been performed, the original event type before correction, the correction confidence level, details of physical projection distance, and the index of associated key image frame sequences, etc.

[0076] The final event detection result data object is asynchronously transmitted to the application output layer through the internal communication interface.

[0077] The LoRA optimization model described above employs a three-stage training strategy. The first stage involves pre-training the temporal verification branch, using YOLO to optimize the S31 temporal verification module based on correctly detected positive event samples. The loss function consists of a temporal continuity loss constraining the manifold representation of temporal smoothness and a contrast loss distinguishing different event types. The hyperparameters are set to a learning rate of 1e-4, a batch size of 16, and the AdamW optimizer, and training is performed for 10-20 epochs.

[0078] The second stage is the pre-training of the scene correction branch. The S32 scene correction module is optimized using samples with misjudged and corrected labels. The loss function includes a margin of 1.0, a triplet center loss to bring the center of the correct class closer and push away the center of the wrong class, a radius constraint loss to constrain the radius of the correct class to include the target feature and the radius of the wrong class to exclude the target feature, and a direction alignment loss to align the target feature with the main direction of physical rationality of the correct class. The hyperparameters are set to a learning rate of 5e-5, a batch size of 8, and an AdamW optimizer, and the training is conducted for 20-30 rounds.

[0079] The third stage is end-to-end joint fine-tuning, which uses the full training set containing three types of samples: false positives, correct detections, and misjudgment corrections, to jointly optimize the S3 module. The loss function consists of three parts: event authenticity as the supervision signal, and the predicted value is taken as... The binary classification cross-entropy loss; using the real event category as the supervision signal, and the predicted value as... The system employs a multi-class cross-entropy loss; regularization loss includes manifold basis orthogonality constraints and reachability margin constraints. The manifold basis orthogonality constraint penalizes the deviation of the product of the event manifold basis matrix and its transpose from the identity matrix, with a weight coefficient of 0.01. The reachability margin constraint penalizes the deviation when the distance between the centers of reachable regions of any two different classes is less than 0.5, with a weight coefficient of 0.1. Hyperparameters are set to a learning rate of 1e-5, batch size of 4, and the AdamW optimizer, with 5-10 training epochs. After training, the B·A products of all LoRA adapters are merged into the corresponding original weights to ensure no additional computational overhead during inference. After deployment, the system periodically uses samples retained from S332 for incremental LoRA fine-tuning, requiring only 50-200 new samples and 3-5 training epochs each time. Elastic weights are used to reinforce regularization and prevent catastrophic forgetting, enabling continuous iterative optimization of the model.

[0080] S4: Upload the event detection results to the road section central monitoring platform, and retain samples of misjudgments, missed detections, and new scenarios for iterative optimization.

[0081] After the large model completes secondary verification and misjudgment correction, the final event detection results are classified according to event type and urgency, and asynchronously uploaded to the road section central monitoring platform through a standardized data interface; Events deemed false alarms are only logged and do not trigger alarms or dispatch orders, while verified genuine events are directly pushed to the monitoring screen and emergency dispatch system.

[0082] Meanwhile, false alarm events, missed detection events detected by scene feature deviation detection, misjudgment correction events after category correction, and new scene events with environmental complexity significantly deviating from the training distribution, along with the corresponding standard preliminary detection results and verification reasoning process data, are classified, labeled, and stored in the sample library. The sample library is organized according to event type and scene conditions, with a capacity limit and an elimination strategy based on timeliness and coverage.

[0083] When the sample library increment reaches a preset threshold or the periodic iteration cycle is reached, the LoRA adapter is incrementally fine-tuned using the retained samples, the base weights are frozen, only the low-rank matrix parameters are updated, and elastic weights are introduced to consolidate regularization to prevent catastrophic forgetting. After verification, the updated weights are merged, solidified, and replaced for deployment, so as to realize the model's continuous adaptive optimization to highway environment changes and event type evolution.

[0084] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A high-speed multi-type event detection misjudgment correction method based on LoRA, characterized in that, include: The system acquires real-time event information on highways, identifies events using the YOLO algorithm, and outputs preliminary event detection results and environmental parameters. The preliminary detection results are standardized, and the semantic features and scene features of the event targets in the preliminary detection results are extracted to obtain standard preliminary detection results; Based on the initial standard detection results, the LoRA-optimized large model is used for secondary verification and misjudgment correction to obtain the event detection results; The event detection results are uploaded to the road section central monitoring platform, and samples of misjudgments, missed detections, and new scenarios are retained for iterative optimization.

2. The method for correcting misjudgments in high-speed multi-type event detection based on LoRA according to claim 1, characterized in that, Acquire real-time event information on highways, identify it using the YOLO algorithm, and output preliminary event detection results and environmental parameters, including: Real-time video streams are collected from existing roadside monitoring equipment on highways and transmitted to edge computing units with built-in YOLO target detection algorithms. The YOLO algorithm in the edge computing unit performs frame-by-frame target detection and event classification on the video stream, generating preliminary detection results.

3. The method for correcting misjudgments in high-speed multi-type event detection based on LoRA according to claim 2, characterized in that, The YOLO algorithm in the edge computing unit performs frame-by-frame object detection and event classification on the video stream, generating preliminary detection results, including: Full-image feature extraction and multi-scale target detection are performed on each frame of the image; Events are classified and judged based on the detected target attributes and scene status; Simultaneously extract environmental parameter information associated with the event frame.

4. The method for correcting misjudgments in high-speed multi-type event detection based on LoRA according to claim 1, characterized in that, The preliminary detection results are standardized, and semantic and scene features of the event targets are extracted from the preliminary detection results to obtain standard preliminary detection results, including: The preliminary test result data package is cleaned and its validity is verified to remove invalid data. Semantic features are extracted from valid data to obtain visual semantic information of the event target; Scene features are extracted from all retained valid image frames to obtain complete environmental context information of the scene where the event occurred.

5. The method for correcting misjudgments in high-speed multi-type event detection based on LoRA according to claim 1, characterized in that, Based on the initial standard detection results, a second verification and misjudgment correction were performed using a LoRA-optimized large model to obtain the event detection results, including: The sequence of semantic feature vectors from multiple frames is mapped to an adaptive manifold space, and the temporal semantic stability of events is calculated. A scenario-constrained manifold is constructed based on an event-level comprehensive stability score, and misjudgments are corrected by the reachability of the target semantic representation projected onto the manifold.

6. The method for correcting misjudgments in high-speed multi-type event detection based on LoRA according to claim 5, characterized in that, Mapping multi-frame semantic feature vector sequences to an adaptive manifold space and calculating the temporal semantic stability of events includes: Based on the initial event detection type and scene environment parameters, a dedicated manifold basis for the event is dynamically constructed. Multi-frame semantic feature vectors are mapped to a high-dimensional event manifold space using an event-specific manifold basis. In the event manifold space, instantaneous semantic stability metric and event-level comprehensive stability score are calculated to determine the authenticity of the event.

7. The method for correcting misjudgments in high-speed multi-type event detection based on LoRA according to claim 5, characterized in that, A scene-constrained manifold is constructed based on an event-level comprehensive stability score. Misjudgment correction is performed by projecting the target semantic representation onto the manifold, including: Based on scene feature vectors and environmental parameters, construct the physical reachable areas of each type of event in the current scene; Calculate the physical projection distance from the semantic feature vector of the event target to the physically reachable region of each category; Misjudgment is corrected based on physical projection distance, and the corrected event type is output.

Images