A traffic scene change detection method and system

Abstract

The application discloses a traffic scene change detection method and system, the method comprising: converting a double-phase image pair corresponding to a change detection data set and a task instruction into a task guide feature; obtaining a multi-scale physical feature representation of the double-phase image pair; using the task guide feature as a semantic query, enhancing and reconstructing the multi-scale physical feature representation in the time-space dimension to obtain a double-phase perception feature map; calculating a change difference feature between the phases based on the double-phase perception feature map, and fusing and activating the change difference feature and the task guide feature to obtain a prediction mask; jointly and iteratively optimizing parameters of a unified change detection model based on a multi-task loss function comprising an instruction understanding loss and a mask prediction loss, and obtaining a change detection result corresponding to a to-be-detected traffic scene based on the optimized unified change detection model. The scheme of the application breaks the limitation of traditional models that can only perform indiscriminate change detection or are limited to a single task.

Description

Technical Field

[0001] This application generally relates to the fields of computer vision and intelligent transportation technology. More specifically, this application relates to a method and system for detecting changes in traffic scenes. Background Technology

[0002] Change detection is a core technology in computer vision and image processing. It identifies areas that have changed by analyzing sequences of images or videos taken from the same scene at different times. In dynamic and complex traffic scenarios, this technology plays a crucial role and is widely used in various fields, including intelligent operation and maintenance of traffic infrastructure (such as road defect detection and facility status monitoring), traffic monitoring and anomaly detection (such as illegal parking, congestion, and debris detection), high-precision map and digital twin updates, construction area safety monitoring, and traffic flow analysis.

[0003] To address different application needs, existing change detection primarily employs two technical approaches: traditional methods and deep learning methods. Traditional methods mainly include frame differencing, background subtraction, and image differencing. While these methods are simple and fast, they are prone to producing "ghosting" effects and are sensitive to noise. In recent years, deep learning technologies, represented by Convolutional Neural Networks (CNNs) (such as Siamese network architectures and end-to-end encoder-decoder architectures), have become mainstream. They can automatically learn robust features, effectively suppress interference from lighting and shadows, and possess a certain degree of semantic understanding and extremely high detection accuracy.

[0004] However, in real-world traffic applications, existing technologies still face the following bottlenecks and shortcomings:

[0005] First, the complex environment causes significant interference. Real traffic scenarios involve dramatic changes in lighting and weather (sunny days, cloudy days, rain, snow, night), seasonal changes (the growth and shedding of branches and leaves, snow accumulation), moving shadows (shadows cast by buildings and vehicles), dynamic backgrounds (swaying trees, flashing billboards), and perspective shifts caused by camera shake, which can easily lead to false detections and require the model to have a very high level of scene understanding.

[0006] Second, binary and semantic change detection tasks are fragmented and difficult to unify. Existing change detection methods are strictly divided into "binary change detection" and "semantic change detection." Binary change detection only identifies the location of the change; while semantic change detection not only determines the location but also identifies the specific semantic transformation properties. In complex traffic scenarios, both tasks are urgently needed (for example, determining whether a vehicle has left a parking space only requires binary detection, while detailed traffic flow analysis requires semantic detection). However, due to the incompatibility between multi-source datasets and differences in underlying model architecture, existing change detection methods cannot be unified. When dealing with different traffic tasks, it is often necessary to train and deploy different change detection models separately to adapt to different application scenarios, which greatly increases the training cost and deployment difficulty of the models.

[0007] In view of this, there is an urgent need to provide a traffic scene change detection solution to solve the above problems and achieve universality and adaptability in various complex traffic scenarios. Summary of the Invention

[0008] In order to at least address one or more of the technical problems mentioned above, this application proposes a traffic scene change detection scheme in several aspects.

[0009] In a first aspect, this application provides a method for detecting changes in a traffic scene, comprising: using a multimodal coding unit in a pre-constructed unified change detection model to convert a pair of two-temporal images and a task instruction corresponding to a change detection dataset into task-guided features; using a visual feature extraction unit in the unified change detection model to obtain multi-scale physical feature representations of the two-temporal images; using a decoder in the unified change detection model, with the task-guided features as semantic queries, to enhance and reconstruct the spatiotemporal dimensions of the multi-scale physical feature representations to obtain a two-temporal perception feature map; calculating inter-temporal change difference features based on the two-temporal perception feature map, and fusing and activating the change difference features with the task-guided features to obtain a prediction mask corresponding to the task instruction; jointly iteratively optimizing the parameters of the unified change detection model based on a multi-task loss function including instruction understanding loss and mask prediction loss, and obtaining change detection results corresponding to the traffic scene to be detected based on the optimized unified change detection model.

[0010] In some embodiments, the change detection dataset includes a binary change detection dataset and a semantic change detection dataset.

[0011] In some embodiments, during the process of converting the bi-temporal image pairs corresponding to the change detection dataset and the task instructions into task guidance features, the following steps are performed: the task instructions are converted into a text embedding sequence by a text encoder in a multimodal coding unit; the bi-temporal image pairs and the text embedding sequence are input into a multimodal large language model to generate a response sequence; the state embedding representation of the response sequence in the last hidden layer of the multimodal large language model is extracted; and the state embedding representation is mapped to the visual feature alignment space by a multilayer perceptron network to obtain task guidance features.

[0012] In some embodiments, during the generation of the response sequence, the following steps are performed: multiple semantic tags are preset in the vocabulary of the multimodal large language model, wherein the multiple semantic tags include [C], [T1] and [T2], [C] is used to indicate the changing region of the image, [T1] is used to indicate the target state in the first time phase image, and [T2] is used to indicate the target state in the second time phase image; the multimodal large language model embeds the corresponding semantic tags at preset positions in the generated response sequence according to the intent of the task instruction.

[0013] In some embodiments, during the acquisition of multi-scale physical feature representations of the dual-temporal image pairs, the dual-temporal image pairs are processed by a visual backbone network combined with a feature pyramid to generate physical feature representations corresponding to multiple different resolution scales; wherein, the expression for the multi-scale physical feature representation is: , This represents the physical features of the first temporal image at the i-th layer of the feature pyramid. This represents the physical features of the second-phase image at the i-th layer of the feature pyramid. For visual backbone network, This is the first phase image. This is the second phase image.

[0014] In some embodiments, the following steps are performed during the process of obtaining the dual-temporal-aware feature map: the physical feature representations at corresponding scales and the task-guided features are processed by multiple sub-decoders in the decoder; within each sub-decoder, the physical feature representations at corresponding scales are refined using the task-guided features as query vectors through a cross-attention mechanism, and then processed in conjunction with a feedforward neural network to obtain refined visual sequences at different scales; the refined visual sequences are reconstructed into spatial dimension feature maps corresponding to the dual-temporal images through separation and dimensionality transformation operations to obtain the dual-temporal-aware feature map.

[0015] In some embodiments, the expression for the dual-temporal sensing feature map is: ,in, This is the perceptual feature map at the i-th scale corresponding to the first temporal image. This is the perceptual feature map at the i-th scale corresponding to the second temporal image. For separation operations and dimension transformation operations, For the refined visual sequence at the i-th scale.

[0016] In some embodiments, during the process of obtaining the prediction mask corresponding to the task instruction, the following steps are performed: The dual-temporal-aware feature maps are unified to the same scale using an upsampling operation, and features are concatenated along the channel dimension to obtain a first temporal-dense feature map and a second temporal-dense feature map; a transformation feature map is obtained by performing a pixel-by-pixel subtraction operation on the first temporal-dense feature map and the second temporal-dense feature map, wherein the expression of the transformation feature map is: , For the change feature map, This is a dense feature map of the first time phase. The second phase dense feature map is used; the change feature map is then multiplied by the corresponding task guidance feature to obtain the final prediction mask corresponding to the task instruction, wherein the expression for the prediction mask of the task instruction is: , For the prediction mask of the task instructions, For the change feature map, This is a task-guided feature.

[0017] In some embodiments, the expression for the multi-task loss function is: Where L is the multi-task loss, To predict loss using a mask, Loss of understanding instructions Weighting coefficients for instruction comprehension loss; , For cross-entropy loss, These are the weighting coefficients for the cross-entropy loss. For Dice's loss, These are the weighting coefficients for the Dice loss.

[0018] In a second aspect, this application provides a traffic scene change detection system, employing the traffic scene change detection method as described in any embodiment of the first aspect. The system includes: a unified change detection model and a model optimization module; the unified change detection model includes a multimodal coding unit, a visual feature extraction unit, and a decoder; the multimodal coding unit is used to convert the dual-temporal image pairs and task instructions corresponding to the change detection dataset into task guidance features; the visual feature extraction unit is used to obtain multi-scale physical feature representations of the dual-temporal image pairs; the decoder is used to enhance and reconstruct the spatiotemporal dimensions of the multi-scale physical feature representations using the task guidance features as semantic queries, to obtain a dual-temporal perception feature map, and to calculate the temporal change difference features based on the dual-temporal perception feature map, and to fuse and activate the change difference features with the task guidance features to obtain a prediction mask corresponding to the task instruction; the model optimization module is used to jointly iteratively optimize the parameters of the unified change detection model based on a multi-task loss function including instruction understanding loss and mask prediction loss; wherein, the change detection result corresponding to the traffic scene to be detected is obtained based on the optimized unified change detection model.

[0019] Through the traffic scene change detection scheme provided above, this application embodiment constructs a unified change detection model, trains the model using a multi-source change detection dataset, and deeply integrates dual-temporal images with natural language task instructions. This breaks the limitations of traditional models that can only perform indiscriminate change detection or are restricted to a single task. First, it endows the model with strong versatility and intelligence, enabling it to flexibly and simultaneously handle binary change detection and semantic change detection tasks according to different instructions within a unified architecture. Second, it utilizes task-guided features as semantic query enhancement multi-scale visual-physical features, achieving cross-modal deep interaction and accurate reconstruction of spatiotemporal dimensions, effectively overcoming background interference in complex traffic scenes, and enabling the model to accurately focus on the change targets of the instructions. Third, the fusion activation of temporal difference features and task features greatly improves the matching degree between the predicted mask and the user intent, as well as the accuracy of boundary localization. Finally, through joint iterative optimization using multi-task losses including instruction understanding and mask prediction, it coordinates the model's language cognition ability and visual segmentation ability, significantly enhancing the model's generalization, robustness, and comprehensive detection performance when facing multi-source data and various complex and changing traffic scenes.

[0020] Furthermore, in some embodiments, a core mechanism for task-guided features is constructed. First, by fully leveraging the powerful cross-modal cognitive and reasoning capabilities of large language models, natural language instructions are deeply integrated with dual-temporal visual contexts, and precisely projected into the visual feature alignment space through a multilayer perceptron network, thereby effectively bridging the semantic gap between abstract text instructions and concrete image features. Second, pre-defined, proprietary semantic tags are creatively introduced and dynamically embedded, enabling the model to explicitly, directionally, and accurately distinguish the specific target state and its spatiotemporal evolution process in a specific temporal phase when executing complex instructions. This significantly improves the fine-grained resolution capability, multimodal alignment accuracy, and the accuracy and controllability of responses to specific intentions in traffic scene change detection tasks.

[0021] Furthermore, in some embodiments, dual-temporal perceptual feature maps are reconstructed based on multi-scale decoders. First, by constructing a multi-scale processing architecture containing multiple sub-decoders, the model can comprehensively capture rich visual information from global semantics to local details, effectively handling targets with varying scales in complex traffic scenarios. Second, a cross-attention mechanism is cleverly employed within each sub-decoder, using task-guided features as query vectors to perform deep feature refinement of multi-scale physical features. This allows the network to strictly follow the user's linguistic instructions, dynamically focusing on task-related visual regions and significantly suppressing irrelevant background noise, achieving efficient fusion and accurate alignment of cross-modal information. Finally, through clearly defined separation and dimensional transformation operations, the one-dimensional refined visual sequence is successfully mapped seamlessly and decoupled back to a two-dimensional spatial dimension representing the first and second temporal phases, accurately restoring the spatiotemporal topological structure of the dual-temporal data. This lays an indispensable dense spatial feature foundation for subsequent pixel-level high-precision change difference calculation and mask prediction.

[0022] Furthermore, in some embodiments, a specific decoding and fusion mechanism is constructed to evolve from multi-scale dual-temporal features to task-specific prediction masks. First, through upsampling and channel concatenation operations, multi-scale dual-temporal perceptual features are unified and integrated into dense feature maps of the first and second temporal phases. This effectively aggregates rich global semantics and local spatial details, providing a solid and refined feature foundation for high-resolution pixel-level segmentation. Second, by utilizing explicit pixel-by-pixel subtraction operations, absolute change differences are efficiently extracted, strongly suppressing unchanged static backgrounds and redundant information, and accurately highlighting the dynamic evolution regions in the temporal sequence. Finally, the basic change feature map and task-guided features are creatively activated through dot product fusion, transforming the task features corresponding to the language instructions into dynamic "semantic filters." This accurately filters out and outputs only change masks that perfectly match the user's specific instruction intent from all physical changes, perfectly achieving fine alignment between visual spatial information and natural language instructions, as well as highly customized intelligent output. Attached Figure Description

[0023] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts, wherein:

[0024] Figure 1 An exemplary flowchart of a traffic scene change detection method according to an embodiment of this application is shown;

[0025] Figure 2 An exemplary flowchart illustrating the acquisition of task guidance features according to an embodiment of this application is shown;

[0026] Figure 3 An exemplary flowchart illustrating the process of obtaining a dual-temporal sensing feature map according to an embodiment of this application is shown;

[0027] Figure 4 An exemplary flowchart illustrating an embodiment of this application is shown for obtaining a prediction mask corresponding to the task instruction;

[0028] Figure 5 An exemplary structural block diagram of a traffic scene change detection system according to an embodiment of this application is shown. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0030] It should be understood that the terms "comprising" and "including" as used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0031] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0032] Figure 1 An exemplary flowchart of a traffic scene change detection method 100 according to an embodiment of this application is shown.

[0033] like Figure 1 As shown, in step S110, the multimodal coding unit in the pre-built unified change detection model is used to convert the dual-temporal image pairs and task instructions corresponding to the change detection dataset into task guidance features.

[0034] In embodiments of this application, the change detection dataset includes a binary change detection dataset and a semantic change detection dataset.

[0035] Specifically, due to the relative difficulty in collecting datasets, the datasets are divided into two parts: open-source datasets and self-collected datasets.

[0036] Specifically, the open-source change detection datasets include the SYSU-CD dataset, the CDD dataset, and the SECOND dataset.

[0037] Specifically, SYSU-CD and CDD are binary change detection datasets. "Binary" change refers to models making only "black and white" judgments: they only distinguish whether a region "has changed" or "has not changed," without considering what it has become. For example, SYSU-CD covers changes in the appearance or disappearance of roads and vehicles in urban construction. CDD covers changes in the form of vehicles and roads caused by seasonal changes; their goal is simply to identify these changed areas. The SECOND dataset is a semantic change detection dataset. Semantic change detection goes a step further than binary change detection; it not only requires identifying "where it changed," but also precisely identifying "what specific category it became." For example, this dataset not only marks a changed area but also provides detailed labels indicating whether it "changed from grassland to road" or "changed to building."

[0038] Specifically, self-collected datasets require a great deal of manpower and time to perform fine semantic annotation. Therefore, the datasets mainly consist of binary change detection datasets, such as simple state transition scenarios like vehicles "entering" or "exiting" in roadside parking traffic scenarios.

[0039] In the embodiments of this application, in order to break down the incompatibility barrier between previous binary detection and semantic detection models and enable the model to handle both tasks simultaneously, the system defines a standard input format: the image input includes a pair of time-series... and Captured bi-temporal images (i.e., two photographs taken at the same location at different times) are defined as follows: (The input image is a three-dimensional digital matrix consisting of height H, width W, and color channels C. Text input is defined as...) It provides specific task instructions for the model. These instructions cover both general binary queries (such as "please segment the changed region") and fine-grained semantic queries (such as "please segment the semantic mask of the changed region").

[0040] In the embodiments of this application, the specific process of converting the bi-temporal image pairs corresponding to the change detection dataset and the task instructions into task-guided features can be found in [reference needed]. Figure 2 .

[0041] Figure 2 An exemplary flowchart illustrating the acquisition of task guidance features according to an embodiment of this application is shown.

[0042] like Figure 2 As shown, in step S210, the task instructions are converted into a text embedding sequence by the text encoder in the multimodal coding unit. In step S220, the bi-temporal image pair and the text embedding sequence are input into the multimodal large language model to generate a response sequence. In step S230, the state embedding representation of the response sequence in the last hidden layer of the multimodal large language model is extracted. In step S240, the state embedding representation is mapped to the visual feature alignment space through a multilayer perceptron network to obtain the task guidance features.

[0043] In the embodiments of this application, the multimodal large language model includes, but is not limited to, models with strong text and image comprehension capabilities such as LLaVA and MiniGPT-4.

[0044] In the embodiments of this application, during the execution of step S220, firstly, multiple semantic tags are preset in the vocabulary of the multimodal large language model to solve the problem of incompatibility between binary and semantic change detection data and the inability to train them simultaneously. Then, the multimodal large language model embeds the corresponding semantic tags at preset positions in the generated response sequence according to the intent of the task instruction.

[0045] Specifically, the semantic tags include [C], [T1] and [T2], where [C] indicates the changing region of the image, [T1] indicates the target state in the first time phase image, and [T2] indicates the target state in the second time phase image.

[0046] Specifically, the expression for the response sequence is: = , For the response sequence, For multimodal large language models, This is the first phase image. This is the second phase image.

[0047] In the embodiments of this application, the visual feature alignment space is also a feature space compatible with the downstream vision module. After the projection operation of the multilayer perceptron network, an abstract text instruction is completely transformed into a "conditioning signal" that the vision module can directly understand and call. The downstream vision module can then use this projected feature as a query vector in the attention mechanism to accurately guide feature decoding and mask prediction in specific changing regions of the dual-temporal image.

[0048] In the embodiments of this application, the expression for the task guidance feature is: ,in, ( ) represents a multilayer perceptron network. This is a task-guided feature. The response sequence is represented by the state embedding in the last hidden layer of a multimodal large language model.

[0049] After completing step S110, in step S120, the visual feature extraction unit in the unified change detection model is used to obtain the multi-scale physical feature representation of the dual-temporal image pair.

[0050] In the embodiments of this application, during the process of acquiring multi-scale physical feature representations of dual-temporal image pairs, the dual-temporal image pairs are processed by combining a visual backbone network with a feature pyramid to generate physical feature representations corresponding to multiple different resolution scales.

[0051] In the embodiments of this application, the core function of the visual backbone network is as a feature encoder for dual-temporal images. This network employs a dual-Siamese architecture, meaning it uses a dual-Siamese architecture for the first temporal image... Second phase image They share the same network weights. In this way, the model can ensure that the physical features of two phases are represented in the same metric space, thereby eliminating spurious changes caused by inconsistent weights.

[0052] In the embodiments of this application, a feature pyramid network is constructed to capture changes in land features with significant scale differences in remote sensing images (e.g., detecting both minute changes in vehicles and the evolution of large buildings or forest cover). It utilizes a max pooling layer to downsample the original features, progressively expanding the receptive field and extracting deep abstract features. A transposed convolutional layer or upsampling layer is then used to fuse deep semantic features with shallow spatial details. Through this hierarchical design, the model can generate four feature representations at different resolution scales for each temporal phase, covering full-dimensional information from low-level texture details to high-level semantic attributes.

[0053] Max pooling layers ensure the translation invariance and robustness of features, while transposed convolutional layers guarantee pixel-level precision in subsequent decoding stages. These feature representations serve as the underlying visual input, and are deeply fused with the task instructions from step S110 in subsequent modules to achieve precise change detection for task perception.

[0054] In the embodiments of this application, the expression for the multi-scale physical feature representation is: , , This represents the physical features of the first temporal image at the i-th layer of the feature pyramid. This represents the physical features of the second-phase image at the i-th layer of the feature pyramid. For visual backbone network, This is the first phase image. This is the second phase image.

[0055] After completing step S120, in step S130, the decoder in the unified change detection model is used to enhance and reconstruct the spatiotemporal dimension of the multi-scale physical feature representation using task-guided features as semantic queries, thereby obtaining a dual-temporal perception feature map.

[0056] The specific process involved in step S130 in the embodiments of this application can be found in [reference needed]. Figure 3 .

[0057] Figure 3 An exemplary flowchart illustrating the process of obtaining a dual-temporal-aware feature map according to an embodiment of this application is shown.

[0058] like Figure 3As shown, in step S310, multiple sub-decoders in the decoder process the physical feature representations and task-guided features at corresponding scales. In step S320, within each sub-decoder, a cross-attention mechanism is used with the task-guided features as the query vector to refine the physical feature representations at the corresponding scale, and this is combined with a feedforward neural network to obtain refined visual sequences at different scales. In step S330, the refined visual sequences are reconstructed into spatial dimension feature maps corresponding to the dual-temporal images through separation and dimensionality transformation operations, resulting in dual-temporal perception feature maps.

[0059] In the embodiments of this application, the decoder adopts a hierarchical parallel architecture, which contains four independent and structurally symmetrical sub-decoders. Each sub-decoder is designed to specifically process the physical feature representations at the corresponding scale in the feature pyramid and the task-guided features projected from the large model. This parallel processing mechanism ensures that everything from the fine-grained texture at the bottom layer to the macroscopic semantic information at the top layer can be accurately modulated by the task instructions.

[0060] In the embodiments of this application, within each sub-decoder, the core computational unit consists of a cross-attention mechanism and a feed-forward network (FFN). The cross-attention mechanism uses task-guided features as query vectors and corresponding scale-based dual-temporal physical feature representations as key and value vectors. By calculating the association weights between the query vectors and visual features, the model can accurately locate task-related feature points (such as identifying changes in "buildings") from a massive number of visual pixels, thereby achieving in-depth feature refinement. The refined vector sequence undergoes nonlinear transformation and dimensionality integration via the feed-forward network (FFN), ultimately outputting a refined visual sequence containing rich semantic constraints. i This sequence is a key intermediate representation for the subsequent generation of high-precision change detection masks.

[0061] In the embodiments of this application, after obtaining the refined visual sequence, the system remaps the serialized representation back to the physical space dimension through a split operation and a reshape operation. Since the refined visual sequence is often flattened into a one-dimensional sequence to fit the Transformer structure, step S330 utilizes the Split / Reshape operator to refine the visual sequence according to the topological structure of the original feature map. The sequences are rearranged into tensors with H×W spatial resolution. Then, a separation operation is performed to decouple the information fused in the sequence, restoring it to the perceptual feature maps corresponding to the first and second time phases, respectively.

[0062] In the embodiments of this application, the expression for the dual-temporal sensing feature map is:

[0063] ,in, This is the perceptual feature map at the i-th scale corresponding to the first temporal image. This is the perceptual feature map at the i-th scale corresponding to the second temporal image. For separation operations and dimension transformation operations, For the refined visual sequence at the i-th scale.

[0064] After step S130 is completed, in step S140, the time-phase change difference features are calculated based on the dual-temporal-aware feature map, and the change difference features are fused and activated with the task guidance features to obtain the prediction mask corresponding to the task instruction.

[0065] The specific process involved in step S140 in the embodiments of this application can be found in [reference needed]. Figure 4 .

[0066] Figure 4 An exemplary flowchart illustrating how to obtain a prediction mask corresponding to a task instruction according to an embodiment of this application is shown.

[0067] like Figure 4 As shown, in step S410, the dual-temporal sensing feature maps are unified to the same scale using an upsampling operation, and features are concatenated along the channel dimension to obtain a first-temporal dense feature map and a second-temporal dense feature map. In step S420, a pixel-by-pixel subtraction operation is performed on the first-temporal dense feature map and the second-temporal dense feature map to obtain a change feature map. In step S430, the change feature map is multiplied by the corresponding task guidance feature to obtain the final prediction mask corresponding to the task instruction.

[0068] In the embodiments of this application, four scales of perceptual feature maps are obtained in step S130. Since the resolutions of each scale are different, global computation cannot be performed directly. Therefore, in step S410, firstly, upsampling operations (such as bilinear interpolation or deconvolution) are used to uniformly scale the perceptual feature maps of different resolutions to the same dense spatial scale (usually the scale of the original input image or a fixed proportion thereof). Then, the feature maps after unifying the scale are stitched together along the channel dimension, fusing cross-scale details and semantics. Finally, two dense feature maps representing the full amount of information are generated: a first temporal dense feature map and a second temporal dense feature map.

[0069] In the embodiments of this application, the offset of the image on the feature manifold is directly calculated by subtracting the first temporal dense feature map from the second temporal dense feature map pixel by pixel. This method can effectively filter out static features in the background that have not changed.

[0070] In the embodiments of this application, the expression for the change feature map is: , For the change feature map, This is a dense feature map of the first time phase. This is a dense feature map of the second time phase.

[0071] In the embodiments of this application, the generated change feature map is subjected to a dot product operation with the task guidance feature. In this operation, the task guidance feature acts as a high-dimensional semantic filter, in which the semantic meaning of the change target (such as "building") will strongly resonate with the corresponding feature channel in the change feature map (high activation value), while changes unrelated to the instruction (such as "vegetation") will be suppressed.

[0072] In the embodiments of this application, the expression for the prediction mask of the task instruction is:

[0073] , For the prediction mask of the task instructions, For the change feature map, This is a task-guided feature.

[0074] After completing step S140, in step S150, the parameters of the unified change detection model are jointly iteratively optimized based on the multi-task loss function that includes instruction understanding loss and mask prediction loss, and the change detection results corresponding to the traffic scene to be detected are obtained based on the optimized unified change detection model.

[0075] In the embodiments of this application, the expression for the multi-task loss function is: Where L is the multi-task loss, To predict loss using a mask, Loss of understanding instructions Weighting coefficients for instruction comprehension loss.

[0076] In the embodiments of this application, , For cross-entropy loss, These are the weighting coefficients for the cross-entropy loss. For Dice's loss, These are the weighting coefficients for the Dice loss.

[0077] In embodiments of this application, cross-entropy loss optimizes the model's prediction confidence by penalizing the pixel-level difference between the predicted probability map and the binary ground truth label. Dice loss is used to evaluate the overlap between the predicted mask and the ground truth mask, and is particularly effective in mitigating the impact of class imbalance inherent in change detection tasks.

[0078] In the embodiments of this application, the instruction understanding loss adopts the standard autoregressive cross-entropy loss applied to the generation sequence of a large language model.

[0079] In summary, through the traffic scene change detection scheme provided above, this application embodiment constructs a unified change detection model, trains the model using a multi-source change detection dataset, and deeply integrates dual-temporal images with natural language task instructions. This breaks the limitations of traditional models that can only perform indiscriminate change detection or are limited to a single task. First, it endows the model with strong versatility and intelligence, enabling it to flexibly and simultaneously handle binary change detection and semantic change detection tasks according to different instructions within a unified architecture. Second, it utilizes task-guided features as semantic query enhancement multi-scale visual-physical features, achieving deep cross-modal interaction and accurate reconstruction of spatiotemporal dimensions, effectively overcoming background interference in complex traffic scenes and enabling the model to accurately focus on the change targets of the instructions. Third, the fusion activation of temporal difference features and task features greatly improves the matching degree between the predicted mask and the user intent, as well as the accuracy of boundary localization. Finally, through joint iterative optimization using multi-task losses including instruction understanding and mask prediction, it coordinates the model's language cognition ability and visual segmentation ability, significantly enhancing the model's generalization, robustness, and comprehensive detection performance when facing multi-source data and various complex and changing traffic scenes.

[0080] Furthermore, in some embodiments, a core mechanism for task-guided features is constructed. First, by fully leveraging the powerful cross-modal cognitive and reasoning capabilities of large language models, natural language instructions are deeply integrated with dual-temporal visual contexts, and precisely projected into the visual feature alignment space through a multilayer perceptron network, thereby effectively bridging the semantic gap between abstract text instructions and concrete image features. Second, pre-defined, proprietary semantic tags are creatively introduced and dynamically embedded, enabling the model to explicitly, directionally, and accurately distinguish the specific target state and its spatiotemporal evolution process in a specific temporal phase when executing complex instructions. This significantly improves the fine-grained resolution capability, multimodal alignment accuracy, and the accuracy and controllability of responses to specific intentions in traffic scene change detection tasks.

[0081] Furthermore, in some embodiments, dual-temporal perceptual feature maps are reconstructed based on multi-scale decoders. First, by constructing a multi-scale processing architecture containing multiple sub-decoders, the model can comprehensively capture rich visual information from global semantics to local details, effectively handling targets with varying scales in complex traffic scenarios. Second, a cross-attention mechanism is cleverly employed within each sub-decoder, using task-guided features as query vectors to perform deep feature refinement of multi-scale physical features. This allows the network to strictly follow the user's linguistic instructions, dynamically focusing on task-related visual regions and significantly suppressing irrelevant background noise, achieving efficient fusion and accurate alignment of cross-modal information. Finally, through clearly defined separation and dimensional transformation operations, the one-dimensional refined visual sequence is successfully mapped seamlessly and decoupled back to a two-dimensional spatial dimension representing the first and second temporal phases, accurately restoring the spatiotemporal topological structure of the dual-temporal data. This lays an indispensable dense spatial feature foundation for subsequent pixel-level high-precision change difference calculation and mask prediction.

[0082] Furthermore, in some embodiments, a specific decoding and fusion mechanism is constructed to evolve from multi-scale dual-temporal features to task-specific prediction masks. First, through upsampling and channel concatenation operations, multi-scale dual-temporal perceptual features are unified and integrated into dense feature maps of the first and second temporal phases. This effectively aggregates rich global semantics and local spatial details, providing a solid and refined feature foundation for high-resolution pixel-level segmentation. Second, by utilizing explicit pixel-by-pixel subtraction operations, absolute change differences are efficiently extracted, strongly suppressing unchanged static backgrounds and redundant information, and accurately highlighting the dynamic evolution regions in the temporal sequence. Finally, the basic change feature map and task-guided features are creatively activated through dot product fusion, transforming the task features corresponding to the language instructions into dynamic "semantic filters." This accurately filters out and outputs only change masks that perfectly match the user's specific instruction intent from all physical changes, perfectly achieving fine alignment between visual spatial information and natural language instructions, as well as highly customized intelligent output.

[0083] This application also provides a traffic scene change detection system, which can use the aforementioned traffic scene change detection method 100 to detect traffic scene changes, or other methods to detect traffic scene changes, which are not limited herein.

[0084] Figure 5 An exemplary structural block diagram of a traffic scene change detection system according to an embodiment of this application is shown.

[0085] like Figure 5As shown, the system 500 includes a unified change detection model 510 and a model optimization module 520. In the embodiments of this application, the unified change detection model 510 and the model optimization module 520 may be separate units or may be set in the same controller; this application does not impose any restrictions here.

[0086] Specifically, the unified change detection model 510 includes a multimodal coding unit 511, a visual feature extraction unit 512, and a decoder 513.

[0087] Specifically, the multimodal coding unit 511 is used to convert the bi-temporal image pairs corresponding to the change detection dataset and the task instructions into task guidance features.

[0088] Specifically, the visual feature extraction unit 512 is used to acquire multi-scale physical feature representations of dual-temporal image pairs.

[0089] Specifically, the decoder 513 is used to enhance and reconstruct the spatiotemporal dimension of the multi-scale physical feature representation using the task guidance features as semantic queries, to obtain a dual-temporal-aware feature map, and to calculate the change difference features between the temporal phases based on the dual-temporal-aware feature map, and to fuse and activate the change difference features with the task guidance features to obtain the prediction mask corresponding to the task instructions.

[0090] Specifically, the model optimization module 520 is used to perform joint iterative optimization of the parameters of the unified change detection model based on a multi-task loss function that includes instruction understanding loss and mask prediction loss.

[0091] Specifically, the change detection results corresponding to the traffic scene to be detected are obtained based on the optimized unified change detection model 510.

[0092] When system 500 uses the aforementioned traffic scene change detection method 100 to detect changes in the traffic scene, the multimodal coding unit 511 executes the aforementioned step S110, the visual feature extraction unit 512 executes the aforementioned step S120, the decoder 513 executes the aforementioned steps S130 and S140, and the model optimization module 520 executes the aforementioned step S150. The specific execution process can be found above and will not be repeated here.

[0093] While numerous embodiments of this application have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will arise for those skilled in the art without departing from the spirit and intent of this application. It should be understood that various alternatives to the embodiments of this application described herein may be employed in the practice of this application. The appended claims are intended to define the scope of protection of this application and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A method for detecting changes in a traffic scene, characterized in that, include: By using the multimodal coding unit in the pre-built unified change detection model, the bi-temporal image pairs corresponding to the change detection dataset and the task instructions are transformed into task guidance features; The visual feature extraction unit in the unified change detection model is used to obtain the multi-scale physical feature representation of the dual-temporal image pair; Using the decoder in the unified change detection model, and taking the task-guided features as semantic queries, the multi-scale physical feature representation is enhanced and reconstructed in spatiotemporal dimension to obtain a dual-temporal perception feature map. Based on the dual-temporal-aware feature map, the temporal-phase change difference features are calculated, and the change difference features are fused and activated with the task guidance features to obtain a prediction mask corresponding to the task instruction. The parameters of the unified change detection model are jointly iteratively optimized based on a multi-task loss function that includes instruction understanding loss and mask prediction loss, and the change detection results corresponding to the traffic scene to be detected are obtained based on the optimized unified change detection model.

2. The method for detecting changes in traffic scenarios according to claim 1, characterized in that, The change detection dataset includes a binary change detection dataset and a semantic change detection dataset.

3. The method for detecting changes in traffic scenarios according to claim 1, characterized in that, In the process of converting the bi-temporal image pairs corresponding to the change detection dataset and the task instructions into task-guided features, the following steps are performed: The task instructions are converted into a text embedding sequence by the text encoder in the multimodal coding unit; The dual-temporal image pairs and the text embedding sequence are input into a multimodal large language model to generate a response sequence; Extract the state embedding representation of the response sequence in the last hidden layer of the multimodal large language model; The state embedding representation is mapped to the visual feature alignment space through a multilayer perceptron network to obtain task guidance features.

4. The method for detecting changes in traffic scenarios according to claim 3, characterized in that, The following steps are performed during the generation of the response sequence: In the vocabulary of the multimodal large language model, multiple semantic tags are preset, including [C], [T1] and [T2]. [C] is used to indicate the changing region of the image, [T1] is used to indicate the target state in the first time phase image, and [T2] is used to indicate the target state in the second time phase image. The multimodal large language model embeds corresponding semantic tags at preset positions in the generated response sequence according to the intent of the task instruction.

5. The method for detecting changes in traffic scenarios according to claim 1, characterized in that, In the process of acquiring the multi-scale physical feature representation of the dual-temporal image pair, the dual-temporal image pair is processed by combining a visual backbone network with a feature pyramid to generate physical feature representations corresponding to multiple different resolution scales. The expression for the multi-scale physical feature representation is as follows: , This represents the physical features of the first temporal image at the i-th layer of the feature pyramid. This represents the physical features of the second-phase image at the i-th layer of the feature pyramid. For visual backbone network, This is the first phase image. This is the second phase image.

6. The method for detecting changes in traffic scenarios according to claim 1, characterized in that, In the process of obtaining the dual-temporal sensing feature map, the following steps are performed: The physical feature representations and task guidance features at corresponding scales are processed by multiple sub-decoders in the decoder. Within each sub-decoder, the physical feature representation at the corresponding scale is refined using the task-guided features as the query vector through a cross-attention mechanism, and then processed in conjunction with a feedforward neural network to obtain refined visual sequences at different scales. The refined visual sequence is reconstructed into a spatial dimension feature map corresponding to the dual-temporal image by separation and dimensional transformation operations, thus obtaining the dual-temporal perception feature map.

7. The method for detecting changes in traffic scenarios according to claim 6, characterized in that, The expression for the dual-temporal sensing feature map is: ,in, This is the perceptual feature map at the i-th scale corresponding to the first temporal image. This is the perceptual feature map at the i-th scale corresponding to the second temporal image. For separation operations and dimension transformation operations, For the refined visual sequence at the i-th scale.

8. The method for detecting changes in traffic scenarios according to claim 1, characterized in that, In obtaining the prediction mask corresponding to the task instruction, the following steps are performed: By using upsampling operations to unify the dual-temporal sensing feature maps to the same scale, and performing feature stitching along the channel dimension respectively, a first temporal dense feature map and a second temporal dense feature map are obtained. By performing a pixel-by-pixel subtraction operation on the first and second temporal dense feature maps, a change feature map is obtained, wherein the expression of the change feature map is: , For the change feature map, This is a dense feature map of the first time phase. This is a dense feature map of the second time phase; The change feature map is multiplied by the corresponding task guidance feature to obtain the final prediction mask corresponding to the task instruction. The expression for the prediction mask of the task instruction is as follows: , For the prediction mask of the task instructions, For the change feature map, This is a task-guided feature.

9. The method for detecting changes in a traffic scene according to claim 1, characterized in that, The expression for the multi-task loss function is: Where L is the multi-task loss, To predict loss using a mask, Loss of understanding instructions Weighting coefficients for instruction comprehension loss; , For cross-entropy loss, These are the weighting coefficients for the cross-entropy loss. For Dice's loss, These are the weighting coefficients for the Dice loss.

10. A traffic scene change detection system, characterized in that, The system employs the traffic scene change detection method as described in any one of claims 1-9 to detect changes in a traffic scene, and the system includes: a unified change detection model and a model optimization module; The unified change detection model includes a multimodal coding unit, a visual feature extraction unit, and a decoder; The multimodal coding unit is used to convert the bi-temporal image pairs corresponding to the change detection dataset and the task instructions into task guidance features; The visual feature extraction unit is used to obtain multi-scale physical feature representations of the dual-temporal image pair; The decoder is used to enhance and reconstruct the spatiotemporal dimension of the multi-scale physical feature representation using the task guidance feature as a semantic query, to obtain a dual-temporal-aware feature map, and to calculate the change difference feature between the temporal phases based on the dual-temporal-aware feature map, and to fuse and activate the change difference feature with the task guidance feature to obtain a prediction mask corresponding to the task instruction. The model optimization module is used to perform joint iterative optimization of the parameters of the unified change detection model based on a multi-task loss function that includes instruction understanding loss and mask prediction loss; Specifically, the optimized unified change detection model is used to obtain the change detection results for the traffic scene to be detected.

Images