Road key element recognition and traffic scene judgment method based on large model driving

Abstract

The invention discloses a road key element recognition and traffic scene judgment method based on large model driving, and the method comprises the steps: 1, obtaining a traffic scene image, and carrying out the quality screening of an input image, so as to obtain an effective image after the quality screening; step 2, implementing prompt language semantic embedding based on the effective image after quality screening, compressing or expanding two different modes of embedding of text and vision to a unified dimension space, and forming consistent category semantic vectors as category semantic prompts; 3, based on category semantic prompt and vision-language alignment, road key element screening is completed; and 4, based on road key element screening, further utilizing a knowledge base based on a large model to carry out traffic scene reasoning and type determination. According to a preset large model knowledge rule, the identified key elements are mapped to a specific traffic scene category, and precise judgment of scenes such as roads, ramps, tunnels and toll station squares is realized.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, specifically to a method for identifying key road elements and determining traffic scenarios based on a large model. Background Technology

[0002] With the continuous expansion of transportation infrastructure, roads, ramps, tunnels, toll stations, and other traffic scenarios together constitute a complex highway network environment. In road monitoring systems, the ability to accurately identify the types of traffic scenarios reflected in monitored images is of great significance for road safety management, traffic flow scheduling, accident early warning, and facility maintenance. Traditional traffic monitoring mainly focuses on vehicle detection or speed measurement analysis, but lacks the ability to identify road infrastructure and traffic scenario types, resulting in a still very limited understanding of complex traffic environments.

[0003] Existing methods attempt to monitor road vehicle conditions using sensors, radar, and inductive loop detectors. Visual methods also analyze traffic conditions by recognizing vehicles, pedestrians, and traffic signs. However, these methods are highly dependent on the environment, have limited coverage, and struggle to identify road structures, ramp guide lines, tunnel entrances, and tollbooths. Especially in multi-lane highways, scenarios with significant lighting variations, severe occlusion, or complex road structures, vehicle detection alone cannot distinguish between different traffic scenarios such as roads, ramps, tunnels, or tollbooths, thus failing to meet the needs for refined road understanding. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention discloses a method for identifying key road elements and determining traffic scenarios based on a large model, comprising the following steps:

[0005] Step 1: Acquire traffic scene images and perform quality screening on the input images to obtain valid images after quality screening;

[0006] Step 2: Based on the valid images after quality screening, implement semantic embedding of prompts, compress or expand the embeddings of the two different modalities of text and vision into a unified dimensional space to form a consistent category semantic vector as a category semantic prompt;

[0007] Step 3: Based on category semantic cues and visual-language alignment, complete the screening of key road elements;

[0008] Step 4: Based on the screening of key road elements, further utilize the knowledge base based on the large model to perform traffic scenario reasoning and type determination.

[0009] Preferably, step 1 includes:

[0010] Step 1.1: Acquire traffic scene images from the front-end camera device;

[0011] Step 1.2: Basic image quality screening.

[0012] Preferably, step 1 further includes:

[0013] Step 1.3: Judging sharpness and occlusion;

[0014] Step 1.4: Scene integrity check.

[0015] Preferably, step 1 further includes:

[0016] Step 1.5: Output the valid images after quality screening.

[0017] Preferably, step 2 includes:

[0018] Step 2.1: Text prompt embedding and extraction.

[0019] Preferably, step 2 further includes:

[0020] Step 2.2: Reference image prompt embedding extraction.

[0021] Preferably, step 2 further includes:

[0022] Step 2.3: Unified category-level semantic embedding.

[0023] Preferably, step 3 includes:

[0024] Step 3.1: Read and initialize category prompt information;

[0025] Step 3.2: Construct the multi-scale visual coding structure of YOLO-World.

[0026] Preferably, step 3 further includes:

[0027] Step 3.3: Cross-modal feature fusion;

[0028] Step 3.4: Region embedding vector generation.

[0029] Preferably, step 3 further includes:

[0030] Step 3.5: Region-Semantic Similarity Calculation and Key Element Screening.

[0031] The beneficial effects of this invention are as follows:

[0032] This invention, based on pre-defined large-scale model knowledge rules, maps identified key elements to specific traffic scene categories, achieving accurate identification of scenes such as roads, ramps, tunnels, and toll plazas. By constructing a unified element recognition mechanism and scene reasoning mechanism, and ultimately using confidence-weighted and conflict-resolving strategies, the scene matching results obtained from large-scale model semantic reasoning are fused with the judgment scores output from the rule base to form a comprehensive scene score. The scene type that best matches the current image content is then selected as the final output. This invention can quickly and accurately complete the automatic identification of typical traffic scenes in traffic monitoring environments, providing reliable support for road safety supervision and operation scheduling. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating the overall process of a method for identifying key road elements and determining traffic scenarios based on a large model, as described in one embodiment of the present invention. Figure 2 A flowchart of a more specific method in another embodiment of the present invention;

[0034] Figure 3 This is a framework diagram for identifying key elements in a traffic scene based on YOLO-World, as shown in another embodiment of the present invention.

[0035] Figure 4 This is a statistical chart showing the detection rate and recognition accuracy of key elements in various scenarios in another embodiment of the present invention;

[0036] Figure 5 This is a PR curve diagram for each scenario in another embodiment of the present invention;

[0037] Figure 6 In another embodiment of the present invention, a scene classification confusion matrix diagram is shown. Detailed Implementation

[0038] To enable those skilled in the art to understand the technical solutions disclosed in this invention, the technical solutions of various embodiments will be described below in conjunction with the embodiments and related drawings. The described embodiments are only some, not all, of the embodiments of this invention. The reference to "embodiment" herein means that a specific feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will understand that the embodiments described herein can be combined with other embodiments.

[0039] The inventive concept of this invention and its technical contribution to the prior art are as follows:

[0040] This invention proposes a method for identifying key road elements and determining traffic scenes based on a large-scale model knowledge-driven approach. Using road monitoring images as the core processing object, it focuses on identifying and determining structural elements with indicative significance within traffic scenes. By leveraging the visual-language alignment capability of the open-vocabulary object detection model YOLO-World, it constructs category semantic representations through text prompts and example image prompts. Similarity matching between image region features and category prompt vectors is achieved in a unified semantic space, thereby automatically identifying key elements such as road bodies, ramp guide lines, tunnel entrances, and toll plaza structures. Based on this, according to pre-defined large-scale model knowledge rules, the identified key elements are mapped to specific traffic scene categories, achieving accurate determination of scenes such as roads, ramps, tunnels, and toll plazas. By constructing a unified element identification mechanism and scene reasoning mechanism, and ultimately using confidence weighting and conflict resolution strategies, the scene matching results obtained from the large-scale model semantic reasoning are fused with the determination scores output from the rule base to form a comprehensive scene score. The scene type that best matches the current image content is then selected as the final output. This invention can quickly and accurately identify typical traffic scenarios in traffic monitoring environments, providing reliable support for road safety supervision and operation scheduling.

[0041] Therefore, this invention achieves stable and robust scene-level intelligent recognition of road scenes, ramp scenes, tunnel scenes, and toll plaza scenes without relying on complex road topology information.

[0042] In one embodiment, see Figure 1 This invention discloses a method for identifying key road elements and determining traffic scenarios based on a large model, comprising the following steps:

[0043] Step 1: Acquire traffic scene images and perform quality screening on the input images to obtain valid images after quality screening;

[0044] Step 2: Based on the valid images after quality screening, implement semantic embedding of prompts, compress or expand the embeddings of the two different modalities of text and vision into a unified dimensional space to form a consistent category semantic vector as a category semantic prompt;

[0045] Step 3: Based on category semantic cues and visual-language alignment, complete the screening of key road elements;

[0046] Step 4: Based on the screening of key road elements, further utilize the knowledge base based on the large model to perform traffic scenario reasoning and type determination.

[0047] In another embodiment, step 1 includes:

[0048] Step 1.1: Acquire traffic scene images from the front-end camera device;

[0049] Step 1.2: Basic image quality screening.

[0050] In another embodiment, step 1 further includes:

[0051] Step 1.3: Judging sharpness and occlusion;

[0052] Step 1.4: Scene integrity check.

[0053] In another embodiment, step 1 further includes:

[0054] Step 1.5: Output the valid images after quality screening.

[0055] In another embodiment, step 2 includes:

[0056] Step 2.1: Text prompt embedding and extraction.

[0057] In another embodiment, step 2 further includes:

[0058] Step 2.2: Reference image prompt embedding extraction.

[0059] In another embodiment, step 2 further includes:

[0060] Step 2.3: Unified category-level semantic embedding.

[0061] In another embodiment, step 3 includes:

[0062] Step 3.1: Read and initialize category prompt information;

[0063] Step 3.2: Construct the multi-scale visual coding structure of YOLO-World.

[0064] In another embodiment, step 3 further includes:

[0065] Step 3.3: Cross-modal feature fusion;

[0066] Step 3.4: Region embedding vector generation.

[0067] In another embodiment, step 3 further includes:

[0068] Step 3.5: Region-Semantic Similarity Calculation and Key Element Screening.

[0069] It should be specifically noted that the contributions of this invention to the prior art specifically include:

[0070] Open identification capabilities for multiple types of transportation infrastructure

[0071] Traditional traffic scene recognition methods rely on training with fixed categories, requiring extensive manual annotation to adapt to new road structure types, resulting in extremely limited scalability. This invention utilizes an open-vocabulary visual model, constructing category semantic representations through text prompts or example image cues. This enables the recognition of new road structure elements, such as the main road structure, ramp guide lines, tunnel entrances, and toll plaza structures, without requiring retraining. Compared to traditional methods that can only identify vehicles or pedestrians, this invention can directly perceive the traffic infrastructure itself, significantly improving the breadth and adaptability of traffic scene understanding.

[0072] It has robustness and generalization ability in complex road environments.

[0073] Traffic monitoring images are often affected by factors such as changes in lighting, weather interference, occlusion, and the diversity of road structures, causing traditional models to exhibit fluctuating recognition performance in such scenarios. This invention proposes a semantic construction mechanism that allows users to choose between textual and image prompts. It provides two semantic sources for each element type: a linguistic description and a reference image. The mechanism can automatically select the appropriate prompt format based on configuration and generate a unified semantic embedding. This mechanism enhances the model's generalization ability under different road structures and complex environmental conditions, enabling it to maintain high recognition robustness even with significant scene changes.

[0074] Scenario-level reasoning capabilities that combine large model knowledge rules

[0075] Existing scene recognition methods often rely on human experience or complex road topology information, making consistency difficult. This invention establishes a knowledge-based reasoning rule base between key elements and scene categories, and utilizes the semantic reasoning capabilities of a large model to structurally map the identified key elements to specific traffic scene types, achieving a closed-loop reasoning process from element recognition to scene determination. This invention eliminates the need for manually constructing complex rule systems, achieving consistent and accurate classification results across various traffic scenes such as roads, ramps, tunnels, and toll plazas. This fully demonstrates the innovation of this invention in scene understanding and semantic reasoning: by analyzing road monitoring images using an open-vocabulary object detection model, it automatically detects key elements specific to traffic scenes such as road bodies, guide lines, tunnel entrance structures, and toll plaza buildings, and combines this with the knowledge rules of a large model to automatically classify and determine traffic scenes such as roads, ramps, tunnels, and toll plazas.

[0076] In another embodiment, see Figure 2 Step 1: Acquire traffic scene images and perform quality screening on the input images to obtain valid images after quality screening. Specifically, this includes:

[0077] Step 1.1: Acquire traffic scene images from the front-end camera device.

[0078] Keyframes are extracted from the video stream of the road monitoring equipment at preset time intervals or triggered events, and each original image frame is recorded as... ;right By scaling and pixel normalization, traffic scene images of uniform size are obtained. ,in and These represent the set standard image height and width, respectively.

[0079] Step 1.2: Basic Image Quality Screening

[0080] Traffic scene images of uniform size The quality screening process includes: sharpness assessment, exposure anomaly detection, and occlusion detection, with sharpness scores calculated for each. Exposure score and the occlusion ratio This serves as a quantitative basis for subsequent quality assessment.

[0081] Step 1.3: Judging Sharpness and Occlusion

[0082] Will With the first threshold Comparison, will With the second threshold In comparison, when or At that time, it was considered that there was severe blurring or large-area occlusion.

[0083] Step 1.4: Scene Integrity Check

[0084] The system determines whether the main road element in the current image remains largely intact and is not heavily obscured. If it meets these requirements, the image capture area is considered to cover the main road element; otherwise, it is judged as missing road information or insufficient scene integrity. Simultaneously, an exposure score is calculated based on the brightness histogram. and the exposure lower limit threshold Exposure upper limit threshold Compare to determine if the image is too dark or too bright.

[0085] Step 1.5: Output the valid images after quality screening

[0086] When all the indicators in steps 1.3 and 1.4 above meet the corresponding thresholds, i.e., the set conditions are met, the original image of the frame and its corresponding traffic scene image are processed. Label the images as high-confidence images and output traffic scene images. For valid images Otherwise, it is marked as a low-confidence image and can be discarded or have its weight reduced in subsequent inference.

[0087] Step 1 provides a unified input data foundation for subsequent prompt embedding construction and key element detection.

[0088] Step 2: Based on the valid images after quality screening, perform semantic embedding of prompts, compressing or expanding the embeddings of the two different modalities of text and vision into a unified dimensional space to form a consistent category semantic vector as a category semantic prompt.

[0089] Step 2 constructs an embedding vector for each traffic element category in the semantic space based on the accompanying text or reference image. This embedding vector serves as the semantic basis for image region matching. For example, a category configuration table can be used. Each category is indexed. This indicates that the category is configured with several symbolic fields, including the category number. Category Name Text prompt sequence Reference image and prompt type markers For each type of element, two resources are preset: text prompts and reference image prompts. One is selected as the semantic construction input based on the configuration. Specifically, step 2 includes the following sub-steps:

[0090] Step 2.1: Text Prompt Embedding and Extraction

[0091] For categories that use text prompts First, the text string is segmented, tokenized, and encoded to form a prompt sequence. And input it into the CLIP text encoder. To obtain the text semantic embedding vector :

[0092] ,

[0093] The CLIP text encoder, based on a Transformer architecture, uses positional encoding, multi-head self-attention, and a feedforward network to model the contextual semantics of the prompt. This embedding vector... This represents the semantic information of the traffic element in the language modality.

[0094] Step 2.2: Reference Image Cue Embedding and Extraction

[0095] For categories that have been configured with reference images Read the prompt image And input it into the CLIP visual encoder. The visual semantic embedding vector is obtained as follows:

[0096] ,

[0097] The visual encoder employs a Vision Transformer (ViT) network structure, which extracts visual pattern features of the cue image through image segmentation, multi-head attention, and hierarchical aggregation, enabling the understanding of the typical appearance structure of the element during the cueing stage.

[0098] Step 2.3: Unified Category-Level Semantic Embedding

[0099] To ensure that text and image prompts can be processed uniformly during the detection phase, the text semantic embedding vectors and visual semantic embedding vectors mentioned above are input into the YOLO-World projection network. To ensure that text and image cues are processed uniformly, the projection network consists of several linear mapping layers and normalization components. This is used to compress or expand the embeddings of text and visual modalities to a unified dimensional space, thereby forming consistent category semantic vectors. The final result Includes: category semantic vectors representing the use of text prompts. , and represent the category semantic vector when using image cues. This is to achieve unified category-level semantic embedding.

[0100] The final result As a category semantic cue, it is aligned with the subsequent image region embedding based on similarity to determine whether the current monitored image contains traffic element structures consistent with the cue.

[0101] Step 3: Based on category semantic cues and visual-language alignment, complete the screening of key road elements.

[0102] After completing the category semantic cues in Step 2, this step, based on the YOLO-World vision-language alignment structure, matches the visual features in the surveillance image with the semantic vectors of the cues, thereby completing the screening of key road elements such as the main road body, ramp guide lines, tunnel entrances, and toll plaza structures. This step includes the following four sub-steps:

[0103] Step 3.1: Category prompt information reading and initialization

[0104] Before performing key element detection / screening / identification, the system sequentially reads the prompts for four categories of elements—roads, ramps, tunnels, and toll stations—according to a preset category configuration table. For each category... (e.g., from the hint cache) read its category semantic vector, and denote the semantic vector of the current category as... Each category's semantic vector contains both text prompts and reference image prompts, based on the configured tags. Choose one as the source of the prompt for the current category: if a reference image path exists and If an instruction uses an image-based prompt, its visual semantic embedding vector is invoked, and then projected through the projection network to obtain the semantic vector for the corresponding category. Otherwise, the text semantic embedding vector is invoked and projected through the projection network to obtain the semantic vector of the corresponding category. .

[0105] In addition, while calling either the visual semantic embedding vector or the text semantic embedding vector, a candidate region and its list are initialized for each category, and a structure of the detection result or the final result obtained by the determination method of the present invention is initialized for each category. The candidate region and its list, as well as the structure, are used to store the subsequently generated region embedding vector, bounding box and matching score, and to provide an input-output space organized by category for subsequent steps.

[0106] Step 3.2: Construct the multi-scale visual coding structure of YOLO-World

[0107] First, the valid images obtained after quality screening in step 1 are... The data is fed into the YOLO-World Visual Backbone Network (VBN) to extract multi-scale visual feature maps. ,in Indicates the backbone network number The feature maps output by each stage, where l takes values ​​from 3 to 5; due to insufficient semantics in the first two stages, the third stage onwards is denoted as... For testing purposes; each stage corresponds to a scale.

[0108] Step 3.3: Cross-modal feature fusion

[0109] In step 3.2, multi-scale visual feature maps are obtained. Then, these are denoted as feature tensors at different scales. And combined with the semantic vector of the corresponding category obtained in step 3.1 Cross-modal fusion of semantic vectors and visual features is achieved through a re-parameterizable Vision-Language Path Aggregation Network (RepVL-PAN). For example, cross-modal feature fusion includes the following two sub-steps:

[0110] Text-guided cross-stage feature layer

[0111] Based on the semantic vector of the corresponding category Enhance the visual regions associated with that category. For example, for each scale... In the feature tensor The semantic vector corresponding to the above category is calculated. The relevant responses are used to generate enhanced features through an attention mechanism. :

[0112] ,

[0113] in The sigmoid function is used to generate attention weights; This indicates taking the maximum response in the channel or spatial dimension. Therefore, this embodiment assigns higher weights to regions related to the current category within the feature space, achieving semantically guided feature recalibration.

[0114] Image pooling attention mechanism

[0115] In obtaining enhanced features Subsequently, spatial pooling is performed on features at different scales to generate small-scale visual fragments and semantic vectors of prompts. Attention interaction is performed to obtain an enhanced set of semantic feature maps, which highlights structural information related to key elements such as the main road, ramp guide lines, tunnel entrances and toll station buildings in a multi-scale feature space.

[0116] For example, after the above two sub-steps (1) and (2), the enhanced semantic feature map set is obtained as follows: , Indicates the main feature map The multi-scale fused features are generated after RepVL-PAN fusion. The results obtained in step 3.3 provide input for the generation of region embedding vectors in the subsequent step 3.4.

[0117] Step 3.4 Region Embedding Vector Generation

[0118] The enhanced semantic feature map set after RepVL-PAN fusion in step 3.3 is as follows: The data is fed into a detection head (DH) network, where the detection head generates a series of candidate regions on feature maps at various scales, using grid cells or anchor boxes as basic units. For each candidate region, a feature vector is extracted from the corresponding location and encoded through a small feedforward network to generate a region embedding vector. The region embedding vector contains the visual semantic information of the candidate region, used to subsequently determine whether the region belongs to a certain traffic element category. Simultaneously, based on the prompt source determined in step 3.1, and combined with the semantic vector of the corresponding category... Following a choice between "text hints" or "reference image hints," the corresponding semantic representation is loaded for the current category. For example, the semantic vector of the corresponding category is... Write the hint cache to be used as the semantic alignment basis for filtering elements of the current category.

[0119] It should be noted that, during the generation of the region embedding vector, the candidate region is embedded into the region embedding vector. And compared with the corresponding semantic vector of the corresponding category. Establish one-to-one matching relationships to form data pairs, which can be used as paired inputs for region-semantic similarity calculation, enabling quantitative judgment of the correlation between candidate regions and category prompts.

[0120] Step 3.5 Region-Semantic Similarity Calculation and Key Element Screening

[0121] After completing step 3.4, generate the region embedding vectors and the corresponding category semantic vectors. After loading, the degree of matching between the candidate regions and the semantics of the prompts is further calculated to complete the screening of key elements.

[0122] For example, a similarity calculation method based on vector normalization and affine transformation is used to map the region embedding vector and the category semantic vector to the same metric space before matching. Specifically, if the region embedding vector is... The semantic vector for the corresponding category is The matching score between the two is then expressed as:

[0123] ,

[0124] in , These are the scaling and bias parameters learned during model training. Indicates the inner product. This is the L2 normalization operation for vectors. The numerical value indicates the degree of correlation between the category cue vector and the candidate region.

[0125] This similarity metric measures the semantic consistency between different candidate regions and category suggestions. Subsequently, the region with the highest score among all candidate regions corresponding to that category is selected as the final key element filtering result, with a confidence level of:

[0126] ,

[0127] Meanwhile, the bounding box, category label, and confidence score of the region are also output as a structured feature vector, which serves as the input for the next step of traffic scene reasoning and type discrimination, so as to achieve the final scene type determination based on the large model.

[0128] Step 4: Based on the screening of key road elements, further utilize the knowledge base based on a large model to perform traffic scenario reasoning and type determination.

[0129] This step, as the final decision-making stage of the overall process of this invention, is responsible for transforming the key element identification results obtained through key element screening into clear traffic scene categories. Based on structured element feature vectors, combined with a large-scale model semantic knowledge base and a rule-driven judgment mechanism, multi-source evidence fusion is used to complete comprehensive reasoning and type output for typical scenarios such as general roads, ramps, tunnels, and toll stations, forming a closed-loop decision-making process from element-level identification to scene-level understanding. This step includes the following four sub-steps:

[0130] Step 4.1: Summarize and format input

[0131] First, the key element identification results are summarized and organized, and then formatted.

[0132] For example, the key element identification results include the main road structure, ramp guide lines, tunnel entrances, and toll plaza structures, and these are formatted. Specifically, the detection confidence of these elements can be expressed in numerical form, letting... , , , Representing the detection confidence levels of the main road structure, ramp guide lines, tunnel entrances, and toll station structures respectively, construct column vectors. =[ , , , ]do This serves as the numerical feature input for subsequent reasoning. Given the context, the column vector necessarily contains both natural language prompts and numerical features. The column vector can also be used to extract necessary combined features or contextual information, forming a unified input data format for subsequent reasoning.

[0133] Step 4.2: Construct a traffic scenario knowledge base based on a large model

[0134] Based on the structured input from step 4.1, for example, a traffic scene knowledge base is pre-built within the locally deployed large model DeepSeek as the inference engine. Specifically, the DeepSeek model pre-learns and stores the mapping relationships between typical traffic scenes (such as general roads, ramps, tunnels, and toll plazas) and their key elements in the knowledge base. Specifically, the knowledge base associates traffic elements with scene types, such as associating the "tunnel entrance" element with the tunnel scene, and the "toll plaza building" element with the toll plaza scene. These descriptions are organized into a set of scene templates and written into the DeepSeek knowledge base in text form, enabling the large model to retrieve and match the closest scene template from the knowledge base when it receives key element descriptions.

[0135] To enable interpretable explicit decision-making, a rule base is configured to define threshold relationships between each scenario and the confidence levels of its key elements. For example, a confidence threshold for the tunnel entrance is configured for the tunnel scenario. Configure building confidence thresholds for toll station scenarios. Configure confidence thresholds for ramp guide lines in ramp scenarios. The rule base stores the following example rule: when The scene is biased towards tunnels, when The scene is more geared towards toll booths. Scenes such as those biased towards ramps.

[0136] Step 4.3: Scenario Reasoning Logic

[0137] Using the traffic scenario knowledge base constructed in step 4.2, the input column vector is processed. Semantic parsing is performed to calculate the semantic matching degree of the current image to each candidate scene type, resulting in a set of semantic scores. , , , These correspond to general road scenarios, ramp scenarios, tunnel scenarios, and toll station scenarios, respectively. Based on the rule base... Perform explicit logical judgments and give a rule score.

[0138] The reasoning process can be formally represented by the following scenario determination rules:

[0139] ,in , A pre-set reliability threshold is used. If a certain condition is triggered, the rule base will give the priority judgment result and rule score for the corresponding scenario. For example, a higher rule score will be assigned to the triggered scenario, and a lower rule score will be assigned to the untriggered scenario.

[0140] To fully utilize information, a fusion score is defined for each candidate scenario. calculate

[0141] ,

[0142] in The semantic matching degree given for the large model. The rule score is obtained based on the threshold rule mentioned above. The weighting coefficients are used. This fusion strategy preserves the comprehensive judgment ability of the large model on complex semantic relationships while introducing explicit constraints provided by the rule base, making scenario reasoning more stable and controllable.

[0143] Step 4.4: Scene Type Determination

[0144] Based on the candidate scenarios obtained in step 4.3 Fusion score In this step, the fusion scores of all candidate scenes are first compared according to the following rules:

[0145] ,

[0146] Select the scene type with the highest fusion score As the initial judgment result for the current image, "road" represents a general road scene, "ramp" represents a ramp scene, "tunnel" represents a tunnel scene, and "toll" represents a tollbooth scene. Subsequently, a consistency check is performed on the results using threshold conditions from the rule base: when the confidence level of a certain element (e.g., ...) reaches a certain threshold, the result is checked for consistency. , , (meeting the corresponding threshold) , When, prioritize maintaining the scene label triggered by that element; when all When all values ​​are below the corresponding threshold and the fusion scores of each scene are similar, the general road scene (road) is used as the default output to avoid overly aggressive judgments when element information is insufficient. Finally, the determined scene type... As a traffic scene category output, it realizes a complete closed loop from key element identification to scene-level judgment.

[0147] Furthermore, in another embodiment, the present invention also discloses a system for implementing the large model-driven road key element identification and traffic scene determination, comprising:

[0148] Surveillance video image acquisition and preprocessing module

[0149] This module serves as the input to the overall traffic scene recognition process. It is responsible for acquiring traffic scene images from front-end cameras within the road monitoring system and performing basic screening for image quality and scene integrity. Keyframes are extracted from the monitoring video stream at preset time intervals or triggered conditions, and each frame is uniformly denoted as... ,in and These represent the height and width of the original monitored traffic scene image, respectively. Through fixed scaling and normalization operations, the original pixels are converted into floating-point tensors suitable for YOLO-World model processing. Basic quality screening is performed on the acquired images to ensure that the input images meet the requirements for subsequent key element recognition based on the YOLO-World model. Quantitative features, including image sharpness, exposure, and occlusion ratio, are calculated and compared with preset thresholds. , , , The comparison is performed. Frames with insufficient sharpness, abnormal exposure, excessive occlusion, or low road coverage are marked as low-confidence images; otherwise, they are marked as high-confidence images and used as valid input for subsequent modules. Through this top-down preprocessing flow, this module provides a unified and quality-controlled input data foundation for subsequent cue embedding construction and key feature detection.

[0150] Prompt semantic embedding building module

[0151] This module leverages the open vocabulary detection capabilities of the YOLO-World model to obtain category semantic representations through text or image prompts, and then performs semantic matching with visual regions in traffic monitoring images to achieve automatic identification of key road elements. First, a category configuration table is established for elements such as the main road structure, ramp guide lines, tunnel entrances, and toll plaza structures. Each category is indexed. This indicates that the category is configured with several symbolic fields to describe its semantics and indicate the source of the resources, such as the category number. Category Name Text prompt sequence Reference image and prompt type markers This is used to describe the semantics and source of prompting resources for this category. Specifically, two types of prompting resources are configured for each type of scene element: one type is descriptive text prompts. Another type is example reference images of the corresponding elements. Based on the availability and tags of the resources provided. The model uses a binary selection mechanism to generate semantic vectors for this category, in order to support robust recognition of different traffic elements under conditions of large differences in appearance and complex morphology.

[0152] In the specific operation process, according to category Read the configuration sequentially, if Instructions use image prompts and reference images When available, for Perform preprocessing similar to that on the input surveillance image, and obtain a visual semantic vector representation through the CLIP visual encoder, denoted as... When using text prompts, the prompt text sequence will be displayed. After word segmentation and tokenization, the text is input into the CLIP text encoder to obtain a text semantic vector, denoted as . Next, the cue vectors from the text or image are all passed through a projection network consisting of a set of linear mappings and normalization operations. and Mapping to a unified-dimensional semantic space yields category semantic embedding vectors. This semantic embedding mechanism for prompts enables the formation of a unified semantic expression among various scene elements such as roads, ramp guide lines, tunnel entrances, and toll stations, which can be directly invoked by subsequent road key element recognition modules.

[0153] Road key element identification module based on vision-language alignment

[0154] After generating the category-based prompt embedding, this module, based on YOLO-World's vision-language alignment structure, matches the visual features in the surveillance image with the aforementioned prompt semantic vectors, thereby completing the detection of key elements in the traffic scene. Specifically, it first uses YOLO-World's multi-scale visual coding structure to extract image region features, and then processes the pre-processed image... Inputting the Visual Backbone Network (VBN) yields a set of multi-scale feature maps. ,in Indicates the backbone network number The feature representations output at each stage are then used. Subsequently, the RepVL-PAN reparameterized vision-language path aggregation network is introduced to aggregate semantic vectors for each category. Features injected at different scales In this process, enhanced features with stronger relevance to the category are obtained through methods such as attention weighting. This improves the model's ability to distinguish between areas such as the main road, ramp guide lines, tunnel entrances, and toll station buildings within the feature space. Based on the fused multi-scale features, the DetectionHead (DH) network generates embedding vectors for candidate regions at each grid location. It also calculates the corresponding bounding box parameters for each candidate region and the semantic vector of the current category. Normalized similarity scores between The score will be... By comparing the results with a preset detection threshold and combining them with post-processing steps such as non-maximum suppression, a small number of high-confidence candidate boxes are selected. Finally, a key element detection result containing location, category label and confidence is formed, providing structured input for subsequent traffic scene reasoning and type determination modules.

[0155] Traffic Scene Reasoning and Type Determination Module

[0156] In the traffic scene determination stage, this invention constructs a joint reasoning mechanism of "semantic knowledge base + explicit rule base + confidence fusion" based on the key element identification results. This module, as the decision output stage of the overall method, is responsible for converting the element-level detection results into explicit traffic scene category labels. First, the detection confidence scores of elements such as tunnel entrances, tollbooth buildings, and ramp guide lines are summarized into structured features. The confidence scores corresponding to the main road structure, ramp guide lines, tunnel entrances, and tollbooth buildings are respectively denoted as... , , , And along with the necessary spatial layout information, it is encoded into a scene feature vector. This serves as a unified input representation for subsequent semantic reasoning and rule determination.

[0157] Based on this, on the one hand, the feature vector The input is fed into the locally deployed large model DeepSeek, which matches it against pre-built scene semantic templates and a traffic scene knowledge base to obtain a set of semantic similarity and matching scores for candidate scene types, including general road scenes, ramp scenes, tunnel scenes, and toll plaza scenes. On the other hand, explicit judgment is performed using a rule base: when... When the value is significantly greater than a preset threshold, the current image is preferentially identified as a tunnel scene; when When the detection results are high and exhibit typical structural features of a tollbooth building, the current image is identified as a tollbooth scene; when When the confidence level of the above key elements is high and the geometric shape of the ramp guide line is identified, the current image is determined to be a ramp scene; when the confidence level of the above key elements does not meet the corresponding conditions, the current image is tended to be determined to be a general road scene.

[0158] The invention and its technical effects are illustrated below with more specific examples:

[0159] For example, a total of 2,174 scene images were obtained from various highway scene video devices. After quality screening based on factors such as sharpness, exposure, occlusion ratio, and road coverage integrity, the valid images were used in experiments.

[0160] Test results of this invention in a real road monitoring system show that, under existing monitoring network deployment conditions, it has successfully identified four traffic scenarios: ramp scenarios, tunnel scenarios, toll station scenarios, and ordinary road scenarios. Under specific test conditions, the detection rate of key scene elements reaches over 80%, and the accuracy rate of key scene element identification reaches over 75%.

[0161] Figure 4 A visual comparison of the detection rate (DR) and recognition accuracy (RA) of key elements in four scenarios was presented. A bar chart represents DR, and a line chart represents RA. The results show that the road scenario has the highest DR, approximately 0.85, followed by tollbooth and tunnel scenarios, while the ramp scenario has a relatively low DR of approximately 0.78. This trend aligns with the complexity of real-world scenarios: road scenarios have the simplest structure, with significant and stable key elements, while ramp scenarios contain more curves, occlusions, and background interference, making the model more susceptible to influence during detection.

[0162] The overall fluctuation of the recognition accuracy RA was small, and all four scenarios remained in the range of 0.76 to 0.79, indicating that the model's recognition and classification ability was relatively stable after the key elements were successfully detected.

[0163] The formulas for calculating DR and RA are as follows:

[0164] Key element detection rate:

[0165]

[0166] Key element identification accuracy:

[0167]

[0168] Figure 5 The presentation displays the precision-recall (PR) curves and average precision (AP) for four scenarios: roads, ramps, tunnels, and toll booths. The curves show typical PR curve characteristics for all scenarios: precision is close to 1 in the low recall phase; as recall increases, precision gradually decreases, exhibiting some fluctuation in the middle. This fluctuation is due to changes in the number of total recall (TP) and prime numbers (FP) at different confidence thresholds, and is considered normal.

[0169] The road scene had the highest AP, indicating that the model could maintain both a high detection rate and high accuracy across different thresholds in this scenario. The tollbooth and tunnel scenes had the next highest AP, while the ramp scene had the lowest AP, consistent with its lowest DR. This suggests that the ramp scene faces greater challenges in both overall prediction recall and accuracy, reflecting the impact of the scene's visual complexity on model performance.

[0170] AP is calculated based on the area under the PR curve, using the following formula:

[0171]

[0172] In practical discrete calculations, the following is typically used:

[0173]

[0174] in and The first Precision and recall at each threshold.

[0175] Figure 6 The confusion matrix shows the distribution of prediction results for the four scene types in the scene classification model. The elements on the diagonal of the figure represent the proportion of correctly classified scenes: approximately 84% for road scenes, approximately 84% for toll station scenes, approximately 81% for tunnel scenes, and approximately 79% for ramp scenes. The ramp scene has the lowest accuracy, consistent with the trend of low DR and low AP in the previous two figures, further confirming that the ramp scene is the most challenging part of the model.

[0176] Off-diagonal elements reflect the model's misclassification direction. For example, a high proportion of tunnel scenes are misclassified as tollbooth scenes, indicating a possible visual similarity between the two. The overall confusion matrix is ​​relatively balanced, and the accuracy of each type of diagonal element is above 0.79, indicating that the model has good overall performance in scene-level classification tasks.

[0177] From the perspective of the confusion matrix, DR and RA can be understood as recall and precision counted by row and column, respectively, and their corresponding relationships are as follows:

[0178] For the Scenario type:

[0179]

[0180] in For the truth, for the first The prediction is also the first. Number of samples in the class For the truth, for the first The number of samples that were classified as one class but were misclassified as other classes.

[0181]

[0182] in For prediction of the first The number of samples that are classified as belonging to other categories but actually belong to other categories.

[0183] In another embodiment, the training method based on the YOLO-World model involved in this invention includes the following steps:

[0184] A large number of traffic scene images covering different times, weather, and lighting conditions are collected. Image sources may include historical archives of front-end camera devices, public datasets, etc. It should be noted that the images need to be standardized, such as size normalization and pixel value normalization. For the training set images, enhancement operations such as random horizontal flipping, random cropping, color dithering, rotation, adding noise, and simulating blur / occlusion should also be considered to improve the robustness and generalization ability of the model. However, the enhancement operations should not compromise the recognizability of key elements in this invention or the realism of the scene.

[0185] Finally, the prepared dataset is divided into training, validation and test sets according to a certain ratio (such as 7:2:1 or 8:1:1).

[0186] The specific training is as follows:

[0187] Load the weights of the selected pre-trained YOLO-World-based model as initial weights; and use a random initialization strategy (such as Xavier or He initialization);

[0188] The model is then fine-tuned in a supervised manner using a training set of prepared historical image data; among which...

[0189] The optimizer is selected from commonly used optimizers such as AdamW and SGD with Momentum; the learning rate scheduling adopts a warm-up strategy, followed by cosine annealing or step decay learning rate scheduling strategies; the batch size is set reasonably according to the GPU memory size.

[0190] It is important to note that the training process is accompanied by performance monitoring on the validation set. Training is stopped early when performance no longer improves to prevent overfitting. During this period, the model is saved periodically, and the model with the best performance on the validation set is selected as the final model.

[0191] In another embodiment, the present invention illustrates the method for identifying key road elements and determining traffic scenarios based on a large model using pseudocode, as shown in Tables 1 and 2 below: Table 1

[0192] Table 2

[0193] In summary, this invention proposes a method for identifying key road elements and determining traffic scenarios based on an open vocabulary detection model. This method utilizes a model with open vocabulary recognition capabilities, enabling efficient identification of multiple categories of traffic infrastructure elements without requiring extensive annotation and retraining for each element category. By combining a vision-language alignment mechanism, category semantic expressions are constructed through text prompts or example images, and image region features are matched with category prompt vectors in a unified semantic space, thereby automatically identifying key elements such as road bodies, ramp guide lines, tunnel entrances, and toll plaza structures. Based on these identification results, and according to pre-defined large-scale model knowledge rules, key elements are further mapped to specific traffic scenario categories, achieving accurate determination of various scenarios such as roads, ramps, tunnels, and toll plazas.

[0194] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying key road elements and determining traffic scenarios based on a large model, characterized in that, Includes the following steps: Step 1: Acquire traffic scene images and perform quality screening on the input images to obtain valid images after quality screening; Step 2: Based on the valid images after quality screening, implement semantic embedding of prompts, compress or expand the embeddings of the two different modalities of text and vision into a unified dimensional space to form a consistent category semantic vector as a category semantic prompt; Step 3: Based on category semantic cues and visual-language alignment, complete the screening of key road elements; Step 4: Based on the screening of key road elements, further utilize the knowledge base based on the large model to perform traffic scenario reasoning and type determination.

2. The method according to claim 1, wherein, Step 1 includes: Step 1.1: Acquire traffic scene images from the front-end camera device; Step 1.2: Basic image quality screening.

3. The method according to claim 2, wherein, Step 1 also includes: Step 1.3: Judging sharpness and occlusion; Step 1.4: Scene integrity check.

4. The method according to claim 3, wherein, Step 1 also includes: Step 1.5: Output the valid images after quality screening.

5. The method according to claim 4, wherein, Step 2 includes: Step 2.1: Text prompt embedding and extraction.

6. The method according to claim 5, wherein, Step 2 also includes: Step 2.2: Reference image prompt embedding extraction.

7. The method according to claim 6, wherein, Step 2 also includes: Step 2.3: Unified category-level semantic embedding.

8. The method according to claim 7, wherein, Step 3 includes: Step 3.1: Read and initialize category prompt information; Step 3.2: Construct the multi-scale visual coding structure of YOLO-World.

9. The method according to claim 8, wherein, Step 3 also includes: Step 3.3: Cross-modal feature fusion; Step 3.4: Region embedding vector generation.

10. The method according to claim 9, wherein, Step 3 also includes: Step 3.5: Region-Semantic Similarity Calculation and Key Element Screening.

Images