A method and device for establishing an incremental target density estimation model of a semantic excitation pattern representation

Abstract

The application discloses a kind of semantic excitation map representation incremental target density estimation model establishment method and device, first to the image in training data set is preprocessed;Then construct the incremental target density estimation model of semantic excitation map representation, including main network, structure anchoring modulation adapter, semantic excitation map representation module, regression head and classification head;Main network keeps frozen in the whole training process, for feature extraction;Structure anchoring modulation adapter is embedded in multiple Transformer layers of visual encoder, for dynamically modulating the fusion weight of task-specific adapter according to current input feature;Semantic excitation map representation module is used to enhance visual feature expression by visual-semantic bimodal map interaction;Regression head is used to convert enhanced visual feature map into density map;Classification head is used to assist class discrimination in incremental learning, help model distinguish whether current image belongs to new class, old class or background.The application can efficiently complete class incremental target detection and counting task, and the performance is more superior than prior art.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and specifically to a method and apparatus for establishing an incremental target density estimation model based on semantic excitation graph representation. Background Technology

[0002] Target density estimation is a fundamental task in computer vision, which aims to automatically estimate the number of targets of a specific category from an image or video frame. It has wide value in practical applications such as traffic monitoring, urban planning, precision agriculture, and crowd analysis.

[0003] Traditional target density estimation methods are mostly designed for single categories. When the target category changes dynamically in practical applications (e.g., agricultural monitoring systems need to count different crops or pests sequentially according to the season, and urban management systems need to count different categories of targets such as vehicles, pedestrians, and buildings), training a separate counting model for each new category consumes a lot of computational resources and time, making it difficult to meet the needs of actual deployment. Therefore, incremental target density estimation techniques have emerged. These techniques require the model to continuously learn new target categories while maintaining the ability to count already learned categories, avoiding the catastrophic forgetting problem.

[0004] Existing incremental target density estimation methods are mainly divided into two categories:

[0005] The first category is based on incremental object detection paradigms, which achieve counting by locating and aggregating objects. These methods typically employ a two-stage strategy: first, detecting the location of each object instance in the image, and then obtaining the total count by counting the number of detection boxes or aggregating instance segmentation results. However, when the scene contains densely packed, heavily occluded, and small-scale objects, the performance of the object detector itself degrades significantly, making it difficult to meet practical requirements for counting accuracy. Furthermore, detection-based methods require expensive bounding box annotations, which are extremely costly in real-world applications.

[0006] The second category consists of specially designed incremental counting networks. For example, EoCNet learns new categories by integrating a class-independent masking module and a class-incrementing module, and retains old knowledge using knowledge distillation. This type of method is directly designed for counting tasks, avoiding the difficulty of precisely locating each target in detection methods. However, existing methods typically require full parameter tuning of the backbone network during incremental learning to adapt to new categories. In practical applications, the extremely high cost of annotation results in only a few hundred samples per category, and full parameter tuning easily leads to overfitting of the model to a limited training distribution. This not only causes mutual interference between old and new knowledge in the backbone network but also severely impairs the model's ability to generalize to unknown domains.

[0007] In recent years, some works have attempted to introduce pre-trained visual-language models (such as CLIP) to assist counting tasks, achieving zero-shot or few-shot counting through fine-tuning via text prompts. However, existing methods still utilize multimodal information in a relatively coarse manner, failing to fully consider the fine-grained interaction between visual features and text semantics, resulting in the ineffective guidance role of semantic information. Furthermore, in incremental learning scenarios, there is currently a lack of effective technical solutions for utilizing multimodal semantic information to alleviate feature degradation under few-shot conditions.

[0008] In summary, there is an urgent need in the existing technology for a target density estimation method that can efficiently utilize multimodal semantic information in incremental learning scenarios while avoiding catastrophic forgetting and feature degradation. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the prior art and propose a method and apparatus for establishing an incremental target density estimation model based on semantic excitation graph representation. By introducing a structure anchoring modulation adapter and a semantic excitation graph representation module into the frozen pre-trained backbone network, it achieves efficient adaptation to new categories and effective preservation of knowledge of old categories during incremental learning. At the same time, it utilizes visual-semantic bimodal graph interaction to enhance visual feature expression, thereby improving the accuracy and stability of target density estimation.

[0010] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0011] A method for establishing an incremental target density estimation model based on semantic activation graph representation, the method comprising the following steps:

[0012] Step S1: Divide the dataset containing several images to be counted into a training dataset, a validation dataset, and a test dataset, and determine the learning order of different categories of images in the dataset;

[0013] Step S2: Preprocess the images in the training dataset;

[0014] Step S3: Construct an incremental target density estimation model based on semantic activation graph representation, including a backbone network, a structure anchoring modulation adapter, a semantic activation graph representation module, a regression head, and a classification head. The backbone network remains frozen throughout the training process for feature extraction. The structure anchoring modulation adapter is embedded in multiple Transformer layers of the visual encoder to dynamically modulate the fusion weights of the task-specific adapter based on the current input features. The semantic activation graph representation module is used to enhance visual feature representation through visual-semantic bimodal graph interaction. The regression head is used to convert the enhanced visual feature map into a density map. The classification head is used to assist in category discrimination during incremental learning, helping the model distinguish whether the current image belongs to a new category, an old category, or the background.

[0015] Step S4: Obtain the preprocessed training data in sequence and train the incremental target density estimation model represented by the semantic activation graph;

[0016] Step S5: Select and save the incremental target density estimation model with the highest prediction accuracy from the semantic activation graph representation by verifying the dataset;

[0017] Step S6: Test the saved model using the test dataset to obtain the target density estimation accuracy for the current category;

[0018] Step S7: Return to step S4 until all classes have been trained, and obtain the incremental target density estimation model of the semantic activation graph representation after training.

[0019] To optimize the above technical solution, the specific measures also include:

[0020] In step S1, the method for determining the learning order of different categories of images in the dataset is as follows: select an image of a specific category from the dataset as the base class, and combine it with the background class image for the first stage of training. The image data of the remaining categories are used as the incremental classes for subsequent training.

[0021] In step S2, the image preprocessing method is as follows:

[0022] Step S2-1: Adjust the scale of the image so that the image size is not smaller than the preset cropping size, and then randomly crop it to a fixed resolution;

[0023] Step S2-2: Perform data augmentation on the cropped image, including:

[0024] S2-2-1: Randomly flip the image horizontally with a preset probability;

[0025] S2-2-2: Perform random gamma correction on the image with a preset probability;

[0026] S2-2-3: Perform random grayscale processing on the image with a preset probability.

[0027] In step S3, the backbone network uses a pre-trained CLIP model, which includes an image encoder and a text encoder. The image encoder uses a ViT-B / 16 structure, and the text encoder uses a Transformer structure paired with CLIP. Throughout the entire training process, the parameters of both encoders remain frozen.

[0028] The structure-anchored modulation adapter in step S3 achieves dynamic modulation in the following way:

[0029] Take the first The output of the self-attention layer in each Transformer block is denoted as . ,in For the mark length, The feature dimension is defined as follows: a linear layer, randomly initialized and frozen with a set of parameters, is mapped to a global vector. The feature dimension of this global vector is then mapped to the number of adapters. An activation function is then used to dynamically generate a set of adapter weights to adjust the strength of different adapter classes. The specific formula is as follows:

[0030]

[0031] In the formula, For parameters to be frozen, For the frozen matrix, For the first A low-rank adapter is generated for each task; then the weighted adapter output is added to the feedforward network layer of each Transformer layer, as shown in the following formula:

[0032]

[0033] In the formula, It is a feedforward network layer.

[0034] The semantic activation graph representation module in step S3 specifically includes:

[0035] First, a visual graph is constructed based on the relationships between the tags. ,in and Let each represent a set of nodes and a set of edges, respectively, representing image features. Each marker in the graph is defined as a graph node. and The label sequence length and feature dimension are respectively used, and then the similarity between all nodes is calculated to form a similarity matrix. For each node, a selection is made... The least dissimilar nodes are taken as its difference nodes, which are represented as follows: The edge feature is defined as the vector difference between each original node and its differing nodes, i.e. The final visual representation is composed of original node features, differing node features, and edge features. Then, multi-head attention is used to mine the latent dependencies within it. The specific formula is as follows:

[0036]

[0037] In the formula, This indicates concatenation along the channel dimension. and For linear projection, This indicates that the focus is on the bulls.

[0038] Text graph construction mainly includes the following:

[0039] First, average pooling is performed on the sequence dimension of the visual graph representation to obtain global reference features. Then, the similarity between each visual marker and the reference feature is calculated, and the marker with the highest similarity is selected. These markers serve as visual anchors, imposing visual constraints on the text information. This prompts the text graph to prioritize the activation and reorganization of text concepts strongly related to the current image content, while filtering out irrelevant text noise. Finally, the selected visual anchors are aggregated with all text markers to form nodes in the text graph, enabling text features to assist in visual tasks. The specific formula is as follows:

[0040]

[0041] in, The text is embedded by the text encoder outputting the text corresponding to the category of the input image.

[0042] Text images are constructed similarly to visual images, using the following formula:

[0043]

[0044] In the formula, and These represent the differences between nodes and edge features in the text graph, respectively. and For linear projection,

[0045] Cross-graph interaction and feature enhancement mainly include the following:

[0046] The global feature mean is obtained by averaging the text graph representation along the sequence dimension, and feature-level weights are generated by ReLU activation, which are then used to... Reweighting:

[0047]

[0048] In the formula, Indicates average pooling. This indicates element-wise multiplication.

[0049] The regression head consists of two 3*3 convolutions and one 1*1 convolution, ultimately generating a density map with one channel. The classification head consists of a global average pooling layer and a fully connected layer. It takes the enhanced visual features extracted by the semantic activation map representation module as input and outputs the probability that the current image belongs to each incremental category.

[0050] Step S4 specifically includes:

[0051] Step S4-1: Set training parameters. If the current stage is the base class training stage, initialize the incremental target density estimation model represented by the semantic activation graph with a normal distribution. If it is the incremental training stage, directly use the network parameters obtained from the previous training stage for initialization. For parameters newly extended for the current category, still use a normal distribution for initialization.

[0052] Step S4-2: If the current sample belongs to the new class, select several representative images from the training data belonging to the old class, and combine these images with the images belonging to the new class to form the new class training dataset; otherwise, proceed to step S4-3.

[0053] Step S4-3: Input the training set into the semantic activation graph representation network, and calculate the total network loss based on the network output, the corresponding ground truth density map, and the class labels. Additional distillation loss for old category samples And based on this loss, the Adam optimizer is used to optimize the trainable parameters for the current stage;

[0054] Step S4-4: Use the validation set to select the optimal network parameters;

[0055] Steps S4-5: After the current stage of training is completed, the herding strategy is used to select representative images from the new category training set and add them to the playback memory, and the current model is saved as the old model for the next stage.

[0056] Step S4-3, the process of calculating the total loss of the model includes the following steps:

[0057] S4-3-1: Calculate the counting loss according to the following formula. :

[0058]

[0059] In the formula, The number of marked points. Indicates the first The difference between the predicted result and the true value of each labeled point is calculated using Bayesian loss. It is a binary mask;

[0060] S4-3-2: Calculate the classification loss according to the following formula. :

[0061]

[0062] In the formula, Indicates the number of samples. Indicates the number of categories. Indicates the first In the nth sample Real labels for each category The model represents the first The sample belongs to the first The predicted probabilities of each category;

[0063] S4-3-3: Calculate distillation loss according to the following formula :

[0064]

[0065] In the formula, Represents a representative sample set. This represents the old model. This represents the model at the current stage.

[0066] S4-3-4: Calculate the total loss of the model according to the following formula. :

[0067]

[0068] In the formula, This represents hyperparameters.

[0069] An apparatus for establishing an incremental target density estimation model based on semantic activation graph representation includes a processor and a computer program stored in memory and executable on the processor. When the processor executes the program, it implements the method described above.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] First, this invention proposes a structure-anchored modulation adapter. Unlike traditional incremental learning methods that fine-tune all parameters of the entire backbone network, this invention compresses and statistically analyzes input features using frozen random mappings, dynamically generating modulation weights. Only a small number of trainable parameters are needed to achieve efficient model adaptation to new tasks. This mechanism fundamentally avoids the mutual interference between old and new knowledge in the backbone network caused by full-parameter fine-tuning, significantly mitigating the catastrophic forgetting problem. Furthermore, because the backbone network remains frozen, the computational cost of this invention during incremental learning is far less than that of the full-parameter fine-tuning scheme, making it more suitable for practical deployment scenarios.

[0072] Second, this invention proposes a semantic activation graph representation method. Unlike existing simple feature concatenation or multimodal feature alignment methods, this invention refines visual features using textual semantic information by constructing visual and textual graphs and performing cross-graph interaction. Specifically, the visual graph effectively captures discriminative structural information in the image through a "difference node" strategy, the textual graph selectively activates textual concepts through visual anchor point constraints, and cross-graph interaction injects textual semantics into the visual representation in a feature reweighting manner. This fine-grained multimodal interaction mechanism effectively compensates for representation degradation caused by limited training samples and enhances the model's discriminative ability in incremental learning.

[0073] Third, this invention organically integrates the structural anchoring modulation adapter with the semantic activation graph representation module, forming a complete incremental target density estimation solution. The two work synergistically—the adapter ensures the model continuously adapts to new categories while maintaining parameter efficiency, and the semantic activation graph module enhances feature representation at the multimodal semantic level—together achieving high-precision and high-stability target density estimation in scenarios with few samples for incremental learning. Attached Figure Description

[0074] Figure 1 This is a flowchart illustrating the method for establishing an incremental target density estimation model based on semantic activation graph representation provided by the present invention.

[0075] Figure 2 This is a schematic diagram of the incremental target density estimation model represented by the semantic activation graph of the present invention. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0077] This embodiment provides a method for establishing an incremental target density estimation model based on semantic activation graph representation, such as... Figure 1 and Figure 2 As shown, the specific steps include the following:

[0078] Step S1: Divide the dataset containing several images to be counted into a training dataset, a validation dataset, and a test dataset, and determine the learning order of different categories of images in the dataset;

[0079] This embodiment selects the EoCo dataset for the incremental target density estimation task. Specifically, images of a specific class are chosen from the dataset as the base class, and images of the background class are used for the first stage of training. Images of the remaining classes are used as the incremental classes for subsequent training.

[0080] Step S2: Preprocess the images in the training dataset;

[0081] The preprocessing methods are as follows:

[0082] Step S2-1: Adjust the image scale so that the image size is not smaller than the preset cropping size, and then randomly crop it to a fixed resolution; in this embodiment, the fixed resolution is 224*224.

[0083] Step S2-2: Perform data augmentation on the cropped image. The data augmentation method is as follows: S2-2-1: Randomly flip the image horizontally with a preset probability; S2-2-2: Perform random gamma correction on the image with a preset probability; S2-2-3: Perform random grayscale processing on the image with a preset probability. In this embodiment, the preset probabilities for random horizontal flipping, random gamma correction, and random grayscale processing are 0.5, 0.3, and 0.1, respectively.

[0084] Step S3: Construct an incremental target density estimation model based on semantic activation graph representation, including a backbone network, a structure anchoring modulation adapter, a semantic activation graph representation module, a regression head, and a classification head. The backbone network remains frozen throughout the training process for feature extraction. The structure anchoring modulation adapter is embedded in multiple Transformer layers of the visual encoder to dynamically modulate the fusion weights of the task-specific adapter based on the current input features. The semantic activation graph representation module is used to enhance visual feature representation through visual-semantic bimodal graph interaction. The regression head is used to convert the enhanced visual feature map into a density map. The classification head is used to assist in category discrimination during incremental learning, helping the model distinguish whether the current image belongs to a new category, an old category, or the background.

[0085] The backbone network specifically employs a pre-trained CLIP model, comprising an image encoder and a text encoder. The image encoder uses a ViT-B / 16 architecture, while the text encoder uses a Transformer architecture paired with CLIP. Throughout the entire training process, the parameters of both encoders remain frozen.

[0086] The structure-anchored modulation adapter is implemented in the following way:

[0087] Take the first The output of the self-attention layer in each Transformer block is denoted as . ,in For the mark length, The feature dimension is used; a linear layer, randomly initialized and frozen with a set of parameters, is mapped to a global vector, and the feature dimension of this global vector is mapped to the number of adapters. Then, an activation function dynamically generates a set of adapter weights to adjust the effectiveness of different adapter categories. The specific formula is:

[0088]

[0089] In the formula, For parameters to be frozen, For the frozen matrix, For the first A low-rank adapter is generated for each task; then the weighted adapter output is added to the feedforward network layer of each Transformer layer. The specific formula is:

[0090]

[0091] In the formula, For feedforward network layers;

[0092] The semantic activation graph representation module is used to improve the model's ability to distinguish between new and old categories, and specifically includes:

[0093] First, a visual graph is constructed based on the relationships between the tags. ,in and These represent the sets of nodes and edges, respectively. Image features. Each marker in the graph is defined as a graph node. and These represent the label sequence length and feature dimension, respectively. Then, the similarity between all nodes is calculated to form a similarity matrix, and for each node, a... The least dissimilar nodes are taken as its difference nodes, which are represented as follows: The edge feature is defined as the vector difference between each original node and its differing nodes, i.e. The final visual representation is constructed by concatenating original node features, differing node features, and edge features, and then multi-head attention is used to uncover potential dependencies. The specific formula is:

[0094]

[0095] In the formula, This indicates concatenation along the channel dimension. and For linear projection, This indicates that the focus is on the bulls.

[0096] The text graph construction mainly includes the following:

[0097] First, average pooling is performed on the sequence dimension of the visual graph representation to obtain global reference features. Then, the similarity between each visual marker and the reference feature is calculated, and the marker with the highest similarity is selected. Each text marker serves as a visual anchor. These anchors impose visual constraints on the text information, prompting the text graph to prioritize the activation and reorganization of text concepts strongly related to the current image content, while filtering out irrelevant text noise. Finally, the selected visual anchors are aggregated with all text markers to form nodes in the text graph, enabling text features to assist in visual tasks. The specific formula is as follows:

[0098]

[0099] in, The text is embedded by the text encoder outputting the text corresponding to the category of the input image.

[0100] Text images are constructed similarly to visual images, using the following formula:

[0101]

[0102] In the formula, and These represent the differences between nodes and edge features in the text graph, respectively. and It is a linear projection.

[0103] The cross-graph interaction and feature enhancement mainly include the following:

[0104] The global feature mean is obtained by averaging the text graph representation along the sequence dimension, and feature-level weights are generated by ReLU activation, which are then used to... Reweighting:

[0105]

[0106] In the formula, Indicates average pooling. This indicates element-wise multiplication.

[0107] This process enables textual semantics to stimulate visual features: concepts in the text graph that are strongly related to the current image content receive higher weights, thereby highlighting the discriminative regions in the visual features and suppressing background features that are irrelevant to the counting task.

[0108] Specifically, the regression head consists of two 3*3 convolutions and one 1*1 convolution, ultimately generating a density map with 1 channel.

[0109] The classification head shown consists of a global average pooling layer and a fully connected layer. The input is the enhanced visual features extracted by the semantic activation map representation module, and the output is the probability that the current image belongs to each incremental category.

[0110] Step S4: Obtain the preprocessed training data sequentially and train the incremental target density estimation model represented by the semantic activation graph. The training method specifically includes:

[0111] Step S4-1: Set training parameters. If the current stage is the base class training stage, initialize the incremental target density estimation model represented by the semantic activation graph with a normal distribution. If it is the incremental training stage, directly use the network parameters obtained from the previous training stage for initialization. For parameters newly extended for the current category, still use a normal distribution for initialization.

[0112] Step S4-2: If the current sample belongs to the new class, select several representative images from the training data belonging to the old class, and combine these images with the images belonging to the new class to form the new class training dataset; otherwise, proceed to step S4-3.

[0113] Step S4-3: Input the training set into the semantic activation graph representation network, and calculate the total network loss based on the network output, the corresponding ground truth density map, and the class labels. Additional distillation loss for old category samples And based on this loss, the Adam optimizer is used to optimize the trainable parameters for the current stage;

[0114] S4-3-1: Calculate the counting loss according to the following formula. :

[0115]

[0116] In the formula, The number of marked points. Indicates the first The difference between the predicted result and the true value of each labeled point is calculated using Bayesian loss. It is a binary mask;

[0117] S4-3-2: Calculate the classification loss according to the following formula. :

[0118]

[0119] In the formula, Indicates the number of samples. Indicates the number of categories. Indicates the first In the nth sample Real labels for each category The model represents the first The sample belongs to the first The predicted probabilities of each category;

[0120] S4-3-3: Calculate distillation loss according to the following formula :

[0121]

[0122] In the formula, Represents a representative sample set. This represents the old model. This represents the model at the current stage.

[0123] S4-3-4: Calculate the total loss of the model according to the following formula. :

[0124]

[0125] In the formula, This represents hyperparameters.

[0126] Step S4-4: Use the validation set to select the optimal network parameters;

[0127] Steps S4-5: After the current stage of training is completed, the herding strategy is used to select representative images from the new category training set and add them to the playback memory, and the current model is saved as the old model for the next stage.

[0128] Step S5: Select and save the incremental target density estimation model with the highest prediction accuracy from the semantic activation graph representation by verifying the dataset;

[0129] Step S6: Test the saved model using the test dataset to obtain the target density estimation accuracy for the current category;

[0130] Step S7: Return to step S4 until all classes have been trained, and obtain the incremental target density estimation model of the semantic activation graph representation after training.

[0131] This embodiment also provides an apparatus for establishing an incremental target density estimation model represented by a semantic activation graph, including a processor and a computer program stored in a memory and capable of running on the processor. When the processor executes the program, it implements the above-described method.

[0132] The quantitative results of this embodiment on the incremental target density estimation dataset EoCo Part A are compared with those of other methods, and the results are shown in Table 1. This embodiment selects two metrics to evaluate the network's counting performance: MAE (mean absolute error) and MSE (mean squared error). A lower MAE indicates more accurate prediction, while a lower MSE indicates more stable prediction results.

[0133] Table 1. Quantitative comparison results of this embodiment with other methods on EoCo Part A.

[0134]

[0135] Analysis of experimental results:

[0136] As can be seen from Table 1, the method proposed in this invention outperforms all the comparative methods in terms of MAE and MSE in all six incremental stages.

[0137] During the base class training phase (t0), the MAE of this invention is 61.62, which is about 10.4% lower than the optimal comparison method BiC (68.77), indicating that this invention has better counting accuracy even during the base class training phase.

[0138] In the subsequent incremental phases (t1 to t5), the performance advantage of this invention becomes even more pronounced. In the final phase (t5), the MAE of this invention is 22.98, a reduction of approximately 9.1% compared to the best comparative method SRNet (25.29), and a reduction of approximately 20.3% compared to EoCNet (28.82), which is specifically designed for incremental counting. This indicates that the performance degradation of this invention is smaller and it has better resistance to forgetting as the number of incremental categories increases.

[0139] In terms of stability (MSE index), the present invention also outperforms all the comparison methods at all stages, with an MSE of 55.07 at the final stage (t5), which is significantly better than SRNet (58.18) and EoCNet (64.72), indicating that the counting results of the present invention are more stable and reliable.

[0140] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0141] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for establishing an incremental target density estimation model based on semantic activation graph representation, characterized in that: The method includes the following steps: Step S1: Divide the dataset containing several images to be counted into a training dataset, a validation dataset, and a test dataset, and determine the learning order of different categories of images in the dataset; Step S2: Preprocess the images in the training dataset; Step S3: Construct an incremental target density estimation model based on semantic activation graph representation, including a backbone network, a structure anchoring modulation adapter, a semantic activation graph representation module, a regression head, and a classification head. The backbone network remains frozen throughout the training process for feature extraction. The structure anchoring modulation adapter is embedded in multiple Transformer layers of the visual encoder and is used to dynamically modulate the fusion weights of the task-specific adapter based on the current input features. The semantic activation graph representation module is used to enhance visual feature representation through visual-semantic bimodal graph interaction. The regression head is used to convert the enhanced visual feature map into a density map. The classification head is used to assist in category discrimination during incremental learning, helping the model distinguish whether the current image belongs to a new category, an old category, or the background. Step S4: Obtain the preprocessed training data in sequence and train the incremental target density estimation model represented by the semantic activation graph; Step S5: Select and save the incremental target density estimation model with the highest prediction accuracy from the semantic activation graph representation by verifying the dataset; Step S6: Test the saved model using the test dataset to obtain the target density estimation accuracy for the current category; Step S7: Return to step S4 until all classes have been trained, and obtain the incremental target density estimation model of the semantic activation graph representation after training.

2. The method for establishing an incremental target density estimation model based on semantic activation graph representation according to claim 1, characterized in that: In step S1, the method for determining the learning order of different categories of images in the dataset is as follows: select an image of a specific category from the dataset as the base class, and combine it with images of the background class for the first stage of training. The image data of the remaining categories are used as incremental classes for subsequent training.

3. The method for establishing an incremental target density estimation model based on semantic activation graph representation according to claim 1, characterized in that: In step S2, the image preprocessing method is as follows: Step S2-1: Adjust the scale of the image so that the image size is not smaller than the preset cropping size, and then randomly crop it to a fixed resolution; Step S2-2: Perform data augmentation on the cropped image, including: S2-2-1: Randomly flip the image horizontally with a preset probability; S2-2-2: Perform random gamma correction on the image with a preset probability; S2-2-3: Perform random grayscale processing on the image with a preset probability.

4. The method for establishing an incremental target density estimation model based on semantic activation graph representation according to claim 1, characterized in that: In step S3, the backbone network adopts a pre-trained CLIP model, which includes an image encoder and a text encoder. The image encoder uses a ViT-B / 16 structure, and the text encoder uses a Transformer structure paired with CLIP. During the entire training process, the parameters of both encoders remain frozen.

5. The method for establishing an incremental target density estimation model based on semantic activation graph representation according to claim 1, characterized in that: The structure-anchored modulation adapter in step S3 achieves dynamic modulation in the following way: Take the first The output of the self-attention layer in each Transformer block is denoted as . ,in For the mark length, For feature dimensions; A linear layer, randomly initialized and frozen with a set of parameters, is mapped to a global vector. The feature dimensions of this global vector are then mapped to the number of adapters. An activation function is then used to dynamically generate a set of adapter weights to adjust the strength of different adapter classes. The specific formula is as follows: In the formula, For parameters to be frozen, For the frozen matrix, For the first A low-rank adapter is generated for each task; then the weighted adapter output is added to the feedforward network layer of each Transformer layer, as shown in the following formula: In the formula, It is a feedforward network layer.

6. The method for establishing an incremental target density estimation model based on semantic activation graph representation according to claim 1, characterized in that: The semantic activation graph representation module in step S3 specifically includes: First, a visual graph is constructed based on the relationships between the tags. ,in and Let each represent a set of nodes and a set of edges, respectively, representing image features. Each marker in the graph is defined as a graph node. and The label sequence length and feature dimension are respectively used, and then the similarity between all nodes is calculated to form a similarity matrix. For each node, a selection is made... The least dissimilar nodes are taken as its difference nodes, which are represented as follows: The edge feature is defined as the vector difference between each original node and its differing nodes, i.e. The final visual representation is composed of original node features, differing node features, and edge features. Then, multi-head attention is used to mine the latent dependencies within it. The specific formula is as follows: In the formula, This indicates concatenation along the channel dimension. and For linear projection, This indicates the bulls' self-attention. The text graph construction mainly includes the following: First, average pooling is performed on the sequence dimension of the visual graph representation to obtain global reference features. Then, the similarity between each visual marker and the reference feature is calculated, and the marker with the highest similarity is selected. These markers serve as visual anchors, imposing visual constraints on the text information. This prompts the text graph to prioritize the activation and reorganization of text concepts strongly related to the current image content, while filtering out irrelevant text noise. Finally, the selected visual anchors are aggregated with all text markers to form nodes in the text graph, enabling text features to assist in visual tasks. The specific formula is as follows: in, The text is embedded as the text of the category corresponding to the input image, processed by a text encoder. Text images are constructed similarly to visual images, using the following formula: In the formula, and These represent the differences between nodes and edge features in the text graph, respectively. and For linear projection, The cross-graph interaction and feature enhancement mainly include the following: The global feature mean is obtained by averaging the text graph representation along the sequence dimension, and feature-level weights are generated by ReLU activation, which are then used to... Reweighting: In the formula, Indicates average pooling. This indicates element-wise multiplication.

7. The method for establishing an incremental target density estimation model based on semantic activation graph representation according to claim 1, characterized in that: The regression head consists of two 3*3 convolutions and one 1*1 convolution, ultimately generating a density map with one channel. The classification head consists of a global average pooling layer and a fully connected layer, taking the enhanced visual features extracted by the semantic activation map representation module as input and outputting the probability that the current image belongs to each incremental category.

8. The method for establishing an incremental target density estimation model based on semantic activation graph representation according to claim 1, characterized in that: Step S4 specifically includes: Step S4-1: Set training parameters. If the current stage is the base class training stage, initialize the incremental target density estimation model represented by the semantic activation graph with a normal distribution. If it is the incremental training stage, directly use the network parameters obtained from the previous training stage for initialization. For parameters newly extended for the current category, still use a normal distribution for initialization. Step S4-2: If the current sample belongs to the new class, select several representative images from the training data belonging to the old class, and combine these images with the images belonging to the new class to form the new class training dataset; otherwise, proceed to step S4-3. Step S4-3: Input the training set into the semantic activation graph representation network, and calculate the total network loss based on the network output, the corresponding ground truth density map, and the class labels. Additional distillation loss for old category samples And based on this loss, the Adam optimizer is used to optimize the trainable parameters for the current stage; Step S4-4: Use the validation set to select the optimal network parameters; Steps S4-5: After the current stage of training is completed, the herding strategy is used to select representative images from the new category training set and add them to the playback memory, and the current model is saved as the old model for the next stage.

9. The method for establishing an incremental target density estimation model based on semantic activation graph representation according to claim 8, characterized in that: In step S4-3, the process of calculating the total loss of the model includes the following steps: S4-3-1: Calculate the counting loss according to the following formula. : In the formula, The number of marked points. Indicates the first The difference between the predicted result and the true value of each labeled point is calculated using Bayesian loss. It is a binary mask; S4-3-2: Calculate the classification loss according to the following formula. : In the formula, Indicates the number of samples. Indicates the number of categories. Indicates the first In the nth sample Real labels for each category The model represents the first The sample belongs to the first The predicted probabilities of each category; S4-3-3: Calculate distillation loss according to the following formula : In the formula, Represents a representative sample set. This represents the old model. This represents the model at the current stage. S4-3-4: Calculate the total loss of the model according to the following formula. : In the formula, This represents hyperparameters.

10. An apparatus for establishing an incremental target density estimation model based on semantic activation graph representation, comprising a processor and a computer program stored in memory and executable on the processor, characterized in that: When the processor executes the program, it implements the method described in any one of claims 1-9.

Images