Large model adaptation method based on parameter efficient fine-tuning and light-weight counter head

Abstract

The application relates to the technical field of image processing, and discloses a large model adaptation method based on parameter efficient fine tuning and a light counting head, which comprises the following steps: cutting and normalizing an original crowd image and inputting the image into a visual encoder; extracting an unbiased deviation of a feature tensor to generate a dynamic activation coefficient; using the dynamic activation coefficient to control a low-rank adaptive structure to obtain a multi-layer local feature tensor; performing weighted summation on the multi-layer local feature tensor to generate a two-dimensional fusion feature map; using a hollow space pyramid pooling and a sub-pixel convolution to process the fusion feature map to obtain an up-sampling feature map; inputting the up-sampling feature map into three parallel prediction branch networks to correspondingly output an original prediction density map, an optimal transmission probability field map and a foreground and background segmentation mask map; using the mask map to multiply the first two to cut off and remove non-target background responses; and combining real labeled data to calculate a loss and update parameters. The application can reduce the algorithm power consumption of model fine tuning, and inhibit false positive prediction errors from the aspects of structure and optimization.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method for adapting large models based on efficient parameter fine-tuning and a lightweight counting head. Background Technology

[0002] With the development of multimodal large model technology, its zero-shot generalization and deep semantic understanding capabilities have been applied in visual analysis tasks. In dense crowd counting scenarios, traditional small-scale networks rely on a large amount of locally labeled data, and their generalization performance is limited when faced with cross-domain dynamic environments and complex backgrounds. Therefore, introducing large model fine-tuning and cross-modal knowledge transfer has become a research direction for solving the dense counting problem. Existing related studies usually introduce visual language pre-trained models into unsupervised or zero-shot crowd counting frameworks. By constructing text prompts and comparing them with image encoders for fine-tuning or feature alignment, the semantic association between text prompts and crowd-related visual regions is strengthened, thereby achieving cross-modal knowledge transfer and estimating the number of people.

[0003] However, traditional large-scale models are biased towards low-resolution feature classification or text generation in their initial structural design, which can easily lead to the loss of high-frequency spatial information when directly applied to dense regression tasks. Full fine-tuning not only consumes enormous computational resources but also triggers a catastrophic forgetting phenomenon that destroys the general knowledge of the pre-trained model. In complex visual environments with dense architectural textures, dramatic lighting changes, or numerous distractions, existing efficient parameter fine-tuning methods or basic large-scale model application strategies struggle to balance the prediction requirements of macroscopic semantic understanding and microscopic detail regression. Existing technologies suffer from insufficient extraction of high-frequency spatial features and poor adaptive perception capabilities during the feature extraction stage, while their prediction networks are highly susceptible to interference from complex textures. Under conditions of limited parameter updates and computational resources, these technical deficiencies cause counting models to generate a large number of false positive prediction errors, failing to achieve high-precision pixel-level crowd distribution estimation and hindering the effective activation of the visual perception potential of large-scale models in dense scenes. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method for adapting large models based on efficient parameter fine-tuning and a lightweight counting head. This method solves the problems of high computational overhead for fine-tuning large models in dense crowd counting tasks, and the susceptibility to interference from complex backgrounds, which can lead to a large number of false positive prediction errors.

[0005] To achieve the above objectives, the present invention provides a method for adapting a large model based on efficient parameter fine-tuning and a lightweight counting head, comprising the following steps: The original crowd image is sliced ​​and normalized to generate a normalized image feature tensor sequence; The normalized image feature tensor sequence is input into the visual encoder. The unbiased variance of the input feature tensor sequence is extracted to generate dynamic activation coefficients. The dynamic activation coefficients are used to control the low-rank adaptive structure to obtain multi-layer local feature tensors. Calculate the spatial fusion weight matrix corresponding to the multi-layer local feature tensor, and perform weighted summation to generate a two-dimensional fusion feature map; The two-dimensional fused feature map is processed using the dilated spatial pyramid pooling module and sub-pixel convolution operation to output an upsampled feature map. The upsampled feature map is input into three parallel prediction branch networks, which output the original prediction density map, the optimal transmission probability field map, and the foreground-background segmentation mask map, respectively. The original prediction density map and the optimal transmission probability field map are truncated by multiplying the foreground and background segmentation mask map respectively, and the target prediction density map and the target probability field map are output. The loss is calculated and summed based on the target prediction density map, target probability field map, foreground / background segmentation mask map, and real labeled data, and the parameters of the low-rank adaptive structure and the three parallel prediction branches are updated.

[0006] Furthermore, in the normalization process of the original crowd image segments, the target fill width remainder and target fill height remainder are calculated based on the unit block size parameters preset by the visual encoder. A corresponding number of zero-pixel values ​​are appended to the edges of the original crowd image to generate a fill image, which is then divided into non-overlapping image segments. For each image segment, the original pixel tensor is extracted and a linear transformation normalization calculation is performed independently, outputting a spatially aligned normalized image feature tensor sequence.

[0007] Furthermore, in the control process of the low-rank adaptive structure, unbiased variance extraction is performed on the input feature tensor sequence to generate a variance distribution vector. This variance distribution vector is then input into a single-layer linear projection network and a nonlinear activation function, outputting dynamic activation coefficients. The input feature tensor sequence is then multiplied sequentially by the first low-rank dimensionality reduction matrix and the second low-rank dimensionality increase matrix in the low-rank adaptive structure, followed by element-wise multiplication with the dynamic activation coefficients. The multiplication result is then element-wise added to the tensor data output from the pre-trained weight matrix in a frozen state. This process adaptively adjusts the transmission strength of the low-rank features using the statistical variance characteristics of the input data.

[0008] Furthermore, in the process of generating the two-dimensional fused feature map, a linear transformation layer containing a concatenated weight matrix and a nonlinear activation function are used to calculate the feature mapping scores of each multi-layer local feature tensor for the same spatial coordinate position. The feature mapping scores of each depth are then exponentially calculated and normalized across the network depth dimension to generate the spatial fusion weight matrix corresponding to each depth layer. At the aligned spatial coordinate positions, the spatial fusion weight matrix is ​​multiplied by the corresponding multi-layer local feature tensor using scalar multiplication, and a cross-depth accumulation operation is performed to achieve spatial-level fusion of features at different depths.

[0009] Furthermore, in the feature decoding and upsampling process, the two-dimensional fused feature map is input into the dilated spatial pyramid pooling module, which includes a standard convolution branch and multiple dilated convolution branches with different dilation coefficients. Multi-scale feature tensors are extracted synchronously and concatenated with the global feature tensor restored by upsampling after global average pooling in the channel dimension. The number of channels in the concatenated feature tensor is expanded by preset upsampling network nodes to obtain an intermediate high-dimensional feature tensor. Following spatial block mapping rules, the values ​​of the intermediate high-dimensional feature tensor in the channel dimension are physically moved and interleaved to the adjacent pixel coordinates in the plane's width and height dimensions, completing the sub-pixel convolution operation and generating the upsampled feature map.

[0010] Furthermore, the three parallel prediction branch networks contain different network operator structures: the first branch network contains a first convolutional layer, a linear rectified activation layer, and a second convolutional layer in series, used to output the original prediction density map; the second branch network contains a backbone convolutional layer and an identity mapping shortcut, which outputs the optimal transmission probability field map by adding the backbone tensor and the bypass tensor element by element; the third branch network contains a channel feature mapping convolutional layer and a nonlinear binary classification activation network layer, used to output a foreground-background segmentation mask map.

[0011] Furthermore, in the feature truncation process, a Hadamard product operation is performed. Taking advantage of the characteristic that pixel values ​​at corresponding coordinate positions in the foreground-background segmentation mask map approach 0, the floating-point values ​​of corresponding non-target background regions in the original predicted density map and the optimal transmission probability field map are forcibly truncated and attenuated. This operation mechanism directly eliminates invalid response features caused by non-target background regions during the forward propagation stage.

[0012] Furthermore, the loss summation operation upon which the parameter update is based includes performing a linear weighted summation on four independent losses. The calculation logic for the four losses is as follows: 1) Calculate the pixel-level density fitting loss based on the target prediction density map and Gaussian smoothed density labels; 2) Perform global pixel summation on the target prediction density map and calculate the global absolute error loss with the scalar label of the total number of real people; 3) For the data imbalance state where the number of background pixels without targets is much greater than the number of foreground pixels containing targets, add penalty weights to the pixel coordinates with high classification error rates between the binarized region label and the foreground / background segmentation mask map to generate the foreground segmentation focus classification loss; 4) Flatten the target probability field map into a one-dimensional distribution vector, calculate the squared Euclidean distance between the physical coordinates corresponding to each pixel in the one-dimensional distribution vector and each real coordinate point in the real point set label to construct a distance cost matrix, and then solve for the transmission probability matrix that minimizes the total transmission cost under edge distribution constraints. Finally, calculate and sum the Hadamard product of the distance cost matrix and the transmission probability matrix to generate the dynamically optimal transmission distance loss.

[0013] This invention provides a method for adapting large models based on efficient parameter fine-tuning and a lightweight counting head. It offers the following advantages: 1. This invention generates dynamic activation coefficients by extracting the unbiased variance of the input feature tensor sequence and uses the dynamic activation coefficients to control the low-rank adaptive structure. This feature adaptively adjusts the transmission intensity of nonlinear features by utilizing the spatial distribution differences of the input data. It completes the visual feature adaptation of large models in dense prediction tasks under the condition of updating only a small proportion of model parameters, thereby reducing the memory usage and computing power overhead in the model fine-tuning process.

[0014] 2. This invention inputs the upsampled feature map into three parallel prediction branch networks and outputs the original prediction density map, the optimal transmission probability field map, and the foreground-background segmentation mask map. The foreground-background segmentation mask map is used to multiply and truncate the first two. This feature establishes a parallel and decoupled prediction structure, which directly eliminates the invalid response features caused by non-target background regions during the forward propagation stage, and suppresses the false positive prediction error caused by the background region from the perspective of network physical structure.

[0015] 3. This invention calculates the sum of multiple losses based on the prediction results and the real labeled data to update the parameters. In the calculation of the foreground segmentation focus classification loss, for the state of imbalance between the number of background pixels and foreground pixels, the penalty weight is increased to the position with a higher classification error rate. This feature, together with the dynamic optimal transmission distance loss, suppresses the interference of background pixels from the parameter optimization level, reduces the mean absolute error and root mean square error of the model in dense crowd counting scenarios, and improves the generalization robustness of the model. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram illustrating the overall implementation process and data flow of the present invention; Figure 3 This is a schematic diagram of the input data preprocessing and lossless dynamic slicing process of the present invention; Figure 4 This is a schematic diagram of the structure and calculation process of the spatial variance-driven dynamic gated low-rank adaptive module of the present invention. Figure 5 This is a schematic diagram of the forward propagation calculation process for the multiplicative truncation hard constraint of the present invention; Figure 6 This diagram illustrates the performance comparison of different counting methods of the present invention on a large, dense population dataset. Detailed Implementation

[0017] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See attached document Figure 1 This invention provides a method for adapting large models based on efficient parameter fine-tuning and a lightweight counting head, applicable to computing devices including processors and memory. The method includes: A computing device includes a central processing unit (CPU), a graphics processing unit (GPU), and RAM. The RAM stores computer programs and intermediate tensor data generated during execution. The CPU and GPU execute the computer programs to implement the method steps described in detail later.

[0019] The graphics processing unit (GPU) has dedicated video memory for loading large visual language models with frozen parameters and performing large-scale tensor matrix multiplication operations. The computing device runs an operating system that includes parallel computing acceleration libraries and a deep learning framework, providing low-level instruction support for the model's forward propagation and backpropagation gradient calculations.

[0020] Obtain the original crowd image, and fill the right and bottom edges of the original crowd image with zero pixel values ​​according to the input size standard set by the large visual language model to obtain a filled image whose size meets the preset conditions.

[0021] The filled image is divided into multiple fixed-size image blocks, and the mean is subtracted and the standard deviation is divided for all image blocks in the channel dimension, outputting the corresponding normalized image feature tensor sequence.

[0022] The normalized image feature tensor sequence is input into a large-scale visual language model containing a multi-layered visual encoder. The input sequence tensor is obtained at a pre-defined fully connected layer of the visual encoder. The variance distribution vector of the input sequence tensor along the sequence dimension is calculated.

[0023] The variance distribution vector is mapped to dynamic activation coefficients in spatial dimension through linear mapping and nonlinear activation functions. These dynamic activation coefficients are then used to control the output ratio of the low-rank adaptation matrix. The updated feature sequence of the preset fully connected layer is obtained by summing the original frozen matrix output of the preset fully connected layer with the output of the low-rank adaptation matrix after ratio control.

[0024] Feature extraction nodes are set at preset network depths in the visual encoder. Local feature tensors corresponding to different network depths are synchronously obtained from the feature extraction nodes of each layer. The local feature tensors of each network depth are then input into the spatial feature fusion module.

[0025] The spatial feature fusion module calculates the corresponding score values ​​of the local feature tensors at each spatial location through linear transformation. Normalization is performed on the scores at each spatial location across different network depth dimensions to generate spatial fusion weight matrices for each depth layer. The spatial fusion weight matrices are then element-wise multiplied with the corresponding depth's local feature tensors and summed to generate a two-dimensional fusion feature map.

[0026] The 2D fused feature map is input into a network module with a dilated spatial pyramid pooling structure and a sub-pixel convolutional upsampling structure to obtain an upsampled feature map with channel dimensionality reduction and spatial resolution restoration. The upsampled feature map is then input into three independent parallel prediction branch networks.

[0027] The first branch network performs convolution processing on the upsampled feature map and outputs the original predicted density map. The second branch network extracts features from the upsampled feature map and outputs the optimal transmission probability field map. The third branch network performs binary classification mapping on the upsampled feature map and outputs a foreground / background segmentation mask map.

[0028] The target prediction density map is output by multiplying each pixel value in the original predicted density map with the probability value at the corresponding position in the foreground / background segmentation mask map. Similarly, the target probability field map is output by multiplying each pixel value in the optimal transmission probability field map with the probability value at the corresponding position in the foreground / background segmentation mask map.

[0029] Pixel-level density fitting loss is calculated based on the target predicted density map and the corresponding true Gaussian density labels of the image. Global counting absolute error loss is calculated based on the spatial integral value of the target predicted density map and the true total population label. Foreground segmentation focal classification loss is calculated based on the foreground / background segmentation mask map and the true binarized region labels. Dynamically optimal transmission distance loss is calculated based on the target probability field map and the true set of head coordinates.

[0030] The pixel-level density fitting loss, global counting absolute error loss, foreground segmentation focus classification loss, and dynamic optimal transmission distance loss are linearly weighted and summed to obtain the total loss value. The gradient parameters of the network computation graph are calculated based on the total loss value, and the trainable weight parameters in the low-rank adaptation matrix, spatial feature fusion module, and three parallel prediction branches are updated according to the gradient parameters.

[0031] See attached document Figure 2 The original crowd image is obtained. To preserve the pixel-level size features in the original crowd image and avoid the loss of small target features due to scaling operations, the original crowd image is padded with zero pixels at the boundaries according to the input block size standard pre-set by the visual encoder of the large visual language model.

[0032] The filled image is segmented into fixed-size image blocks. For each image block, mean and variance normalization are performed along the channel dimension to generate a normalized image feature tensor sequence. This normalized image feature tensor sequence is then input into the visual encoder for forward propagation computation.

[0033] The visual encoder consists of multiple cascaded network layers, where most of the pre-trained weights are configured to be frozen, halting gradient updates. A low-rank adaptation structure is bypassed into the pre-defined network layers. This low-rank adaptation structure comprises a dimensionality-reducing matrix with decreasing parameter count and an dimensionality-increasing matrix with increasing parameter count.

[0034] Obtain the input sequence tensor corresponding to the preset network layer. Calculate the variance distribution vector of the input sequence tensor along the sequence dimension. The values ​​of the variance distribution vector reflect the numerical dispersion and fluctuation of the tensor features at the corresponding spatial locations.

[0035] The variance distribution vector is input into a lightweight projection network containing linear mappings and activation functions, outputting dynamic activation coefficients with values ​​ranging from zero to one. The tensor data output by the low-rank adaptive structure is multiplied by the dynamic activation coefficients, and then element-wise added to the tensor data output by the frozen parameters in the preset network layers. The dynamic activation coefficients control the numerical accumulation ratio of different sequence positions in the low-rank adaptive structure.

[0036] Feature interception nodes are set at multiple predetermined depth locations in the visual encoder. Processed local feature tensors are extracted synchronously from each node. These local feature tensors encompass both the underlying texture structure tensor and the deep abstract semantic tensor.

[0037] A spatial feature router is constructed to process the extracted multi-layer local feature tensors. For each spatial location, a multi-layer linear transformation structure is used to map the local feature tensors at each depth, and the corresponding numerical scalar score is calculated.

[0038] At the same spatial location, exponential operations are performed on the numerical scalar scores output by network layers of different depths and then summed. The weight of each depth layer's score in the total score is calculated, generating a spatial weight matrix across depth dimensions. The spatial weight matrix is ​​then used to perform element-wise weighted summation on the local feature tensors corresponding to each depth layer. Through tensor concatenation and dimension reshaping, a two-dimensional fused feature map containing multi-layer information is output.

[0039] The 2D fused feature map is input into the feature decoding network module. The feature decoding network module contains a dilated spatial pyramid pooling structure. By setting multiple parallel dilated convolution branches with different dilation coefficients, context tensors under different receptive fields are obtained and concatenated to achieve channel-dimensional feature compression.

[0040] In the feature upsampling stage, subpixel convolution operations are used. Through a physical rearrangement mechanism, pixel data in the channel dimension is moved and arranged to the spatial dimension, so that the output upsampled feature map is proportionally enlarged in both width and height dimensions, while maintaining the conservation of total pixel information.

[0041] The upsampled feature maps are distributed to three parallel output network branches. The first branch calculates spatial features using standard convolutional layers, outputting a raw floating-point matrix recording the crowd density at each pixel location. The second branch contains dilated convolutions and residual connections, outputting an optimal transport probability field matrix representing the coordinate alignment probability. The third branch performs pixel-level binary classification prediction, outputting a probability mask matrix indicating whether each location belongs to the foreground target region or the background interference region.

[0042] Perform matrix-level multiplication truncation. Element-wise multiplication is performed between the original floating-point numerical matrix and the optimal transmission probability field matrix and the probability mask matrix, respectively. By using the near-zero values ​​in the background region of the probability mask matrix, the numerical output at the background location is constrained and filtered to obtain the target prediction density map and the target probability field map.

[0043] During the backpropagation phase, pixel-level error loss and absolute error integral loss are calculated based on the target prediction density map and preset Gaussian density labels. Classification focus loss is calculated based on the probability mask matrix and binarized labels. A cost matrix is ​​constructed based on the target probability field map and discrete coordinate labels, and the optimal transmission allocation loss is calculated. The above loss values ​​are summed to calculate the gradient matrix of each trainable node in the graph. The parameters of the low-rank adaptive structure, the spatial feature router, and the three parallel output network branches are updated based on the gradient matrices.

[0044] See attached document Figure 3 This embodiment specifically describes the computational processing flow from acquiring the original crowd images to inputting them into a large-scale visual language model.

[0045] To avoid losing information about tiny target pixels during resolution scaling of the original crowd image, a zero-pixel edge padding operation is performed on the input original crowd image. The initial pixel width and initial pixel height values ​​of the original crowd image are obtained. The preset cell block size parameters of the visual encoder of the large visual language model are read.

[0046] Based on the initial pixel width, initial pixel height, and unit block size parameters, the target fill width remainder and target fill height remainder are calculated. Zero pixel values ​​corresponding to the column number of the target fill width remainder are appended to the rightmost pixel column edge of the original crowd image, and zero pixel values ​​corresponding to the row number of the target fill height remainder are appended to the bottom pixel row edge of the original crowd image, generating the corresponding filled image.

[0047] The width and height values ​​of the filled image generated by the above operations are both integer multiples of the unit block size parameter. Following a pre-defined fixed matrix size, the filled image is sequentially divided into multiple non-overlapping image blocks in spatial dimensions. The relative pixel coordinates within each image block remain unchanged.

[0048] For each image segment obtained from the segmentation, extract its original pixel tensors for each color channel. Perform a linear transformation normalization calculation independently on each original pixel tensor; the mathematical expression of this calculation step is as follows: ; In the above formula, This represents the normalized image feature tensor of the computed output; Represents the original pixel tensor corresponding to the current image slice; This represents the pixel mean parameter pre-calculated on the corresponding data channel for a given global dataset; This represents the pixel standard deviation parameter pre-calculated on the corresponding data channel for a given global dataset.

[0049] The normalization operation described above is performed sequentially on all the divided image blocks, outputting the corresponding normalized image feature tensor sequence. This normalized image feature tensor sequence is then used as input data and transmitted to the visual encoder for subsequent tensor forward propagation calculations.

[0050] See attached document Figure 4 This embodiment specifically illustrates the calculation process of controlling the update ratio of the low-rank matrix using the spatial numerical discreteness of the feature tensor under the condition of freezing the main parameters of a large visual language model.

[0051] In the pre-defined fully connected layer of the visual encoder, a branch containing the original weight matrix with frozen parameters and a branch containing the low-rank adaptation matrix with trainable parameters are configured. The low-rank adaptation matrix branch consists of a concatenated first low-rank dimensionality reduction matrix and a second low-rank dimensionality increase matrix. The input feature tensor sequence is obtained and fed into the pre-defined fully connected layer.

[0052] To achieve differentiated tensor update calculations based on the numerical fluctuations at different spatial locations within the feature tensor, unbiased variance extraction is performed on the input feature tensor sequence along the sequence dimension. By calculating the numerical dispersion of pixel features at different spatial locations within the same feature channel, the corresponding variance distribution vector is output. The calculation formula for this step is as follows: ; In the above formula, This represents the variance distribution vector of the calculated output; The input feature tensor sequence represents the input. This represents the function for calculating unbiased variance. This indicates that the tensor dimension for performing variance calculation is the sequence space dimension.

[0053] The calculated variance distribution vector is passed to a single-layer linear projection network. A linear mapping is performed using the trainable mapping weight matrix and bias vector in the single-layer linear projection network, and then input into a nonlinear activation function. The numerical range of the mapping result is restricted to between 0 and 1, and the dynamic activation coefficients corresponding to each spatial location are output. The calculation process is as follows: ; In the above formula, This represents the output dynamic activation coefficients; Represents a nonlinear activation function; This represents the weight matrix of a single-layer linear projection network. This represents the bias vector of a single-layer linear projection network.

[0054] During forward propagation feature update, the output tensors of the original weight matrix branch and the low-rank adaptive matrix branch are calculated separately. In the low-rank adaptive matrix branch, the input feature tensor sequence is multiplied sequentially by the first low-rank dimension reduction matrix and the second low-rank dimension increase matrix, and then multiplied by a constant scaling factor. Subsequently, the calculation result of the low-rank adaptive matrix branch is multiplied element-wise with the dynamic activation coefficients to control the tensor output ratio of this branch at different spatial locations. Finally, the controlled low-rank branch output tensor is added element-wise with the output tensor of the original weight matrix branch to generate the output feature tensor of the preset fully connected layer. The overall feature update formula is as follows: ; In the above formula, This represents the updated output feature tensor; This represents the pre-trained weight matrix that is in a frozen state within the original weight matrix branch; This represents the first low-rank dimension-reduced matrix; This represents the second low-rank, increased-dimensional matrix; This represents the pre-defined rank of a low-rank matrix; This represents a pre-defined low-rank scaling constant factor.

[0055] Through the above structure and calculation steps, the output of the low-rank adaptive matrix branch is accumulated at a higher proportion in spatial locations with larger eigenvariant variance values, and at a lower proportion in spatial locations with smaller eigenvariant variance values. When calculating the gradient during backpropagation, only the following updates are performed. , , and The parameter matrix, The matrix does not participate in gradient update operations.

[0056] This embodiment specifically illustrates the feature tensor extraction across network depths and the adaptive weighted combination calculation process based on spatial location.

[0057] To obtain pixel feature structures under different receptive field ranges, feature extraction operators are set in multiple preset network depth layers of the visual encoder. During the forward propagation computation of the model, when tensor data flows through preset network depth layers, the corresponding feature tensors are simultaneously truncated by the feature extraction operators, obtaining local feature tensors of different network depths for a total preset number of layers.

[0058] The extracted local feature tensors at each depth are input into the spatial feature routing network module. For the [specific depth]... The feature map extracted by the deep layer network is used to read its local feature vector at a specific spatial coordinate position. A linear transformation layer containing a concatenated weight matrix and a nonlinear activation function are then used to perform matrix multiplication on the local feature vector to calculate the current spatial position at the [missing information - likely a specific position or location]. The feature map score of the layer. The calculation formula for this step is as follows: ; In the above formula, Indicates the first The layer local feature tensor in spatial coordinate position The feature mapping score calculated at the location; Indicates the first Feature maps extracted by layer network nodes in spatial coordinates Local feature vectors at; This represents the first-layer trainable weight matrix of the spatial feature routing network module. This represents the trainable weight matrix of the second layer of the spatial feature routing network module. This represents the Gaussian error linear unit activation function.

[0059] After calculating the feature mapping scores for each level and spatial location, for the same spatial coordinate location... The feature map scores of all extraction layers are exponentially calculated and normalized across the network depth dimension to generate the spatial fusion weights for each layer. The calculation steps are as follows: ; In the above formula, Indicates the first The layer local feature tensor in spatial coordinate position The spatial fusion weights are calculated at the point where the weight values ​​are between 0 and 1 and the sum of the weights at each level is 1. This represents the total number of layers of feature tensors extracted by the feature extraction operator; Indicates the first The layer local feature tensor in spatial coordinate position Feature mapping score at the location; Indicates the extraction The calculation results of each network layer are summed. This represents an exponential function with the natural constant as its base.

[0060] After obtaining the spatial fusion weights corresponding to each network depth, element-wise scalar multiplication is performed between the spatial fusion weight values ​​and the corresponding local feature tensors at aligned spatial coordinate positions. Subsequently, the multiplication results from each network depth at the same spatial coordinate position are accumulated, outputting the recombined two-dimensional fusion feature map. The formula for this recombining and merging calculation is as follows: ; In the above formula, This represents the recombined two-dimensional fusion feature map as the final calculated output; Indicates the first The original local feature tensor matrix extracted from the layer.

[0061] After performing the tensor operations of cross-layer feature extraction and pixel-wise normalized weighted summation, the generated reconstructed two-dimensional fusion feature map constitutes a single-layer high-dimensional tensor. This reconstructed two-dimensional fusion feature map simultaneously possesses high-resolution local pixel texture tensor features output from the shallow network of the visual encoder, as well as low-resolution global context tensor features output from the deep network.

[0062] This embodiment specifically illustrates the calculation process of performing spatial multi-scale feature extraction and physical resolution restoration on the fused deep features.

[0063] Obtain the reconstructed 2D fusion feature map output from the preprocessor module. Input the reconstructed 2D fusion feature map in parallel into a dilated spatial pyramid pooling module containing multiple branches. The dilated spatial pyramid pooling module includes one standard convolutional branch and three dilated convolutional branches. The standard convolutional branch uses a convolutional layer with a 1×1 kernel to extract basic pixel features. The three dilated convolutional branches all use convolutional layers with 3×3 kernels, but each is configured with a different dilation coefficient.

[0064] In the forward propagation computation of each dilated convolution branch, the sampling points of the convolution kernel are spread out in a grid pattern in the spatial dimension according to the corresponding dilation coefficient. Through this structure, each branch can obtain feature tensor matrices with different physical spans on the input feature map. The feature tensors output by the above four independent branches, as well as the global feature tensor restored by upsampling after global average pooling, are concatenated and merged in the channel dimension.

[0065] The concatenated feature tensors are input into a channel compression layer configured with a 1×1 convolution kernel. A multi-scale fused feature map with a fixed number of output channels is calculated through matrix multiplication. This multi-branch parallel extraction and dimensionality reduction structure can simultaneously acquire tensor data containing target features of different sizes without increasing the input image resolution, while controlling the memory usage of subsequent modules.

[0066] The multi-scale fused feature map is input into an upsampling network node based on the sub-pixel convolution principle. The upsampling network node first calls a standard convolutional layer to expand the number of channels of the multi-scale fused feature map to a square of the preset upsampling ratio parameter, thereby obtaining an intermediate high-dimensional feature tensor.

[0067] A periodic physical coordinate rearrangement operation is performed on the intermediate high-dimensional feature tensor. Feature values ​​in each channel of the tensor along the depth direction are extracted, and according to the defined spatial block mapping rules, the values ​​in each channel dimension are sequentially moved and interleaved to the adjacent pixel coordinate positions in the plane's width and height dimensions. During this stage, the system does not perform pixel value inference calculations using nearest neighbor interpolation or bicubic interpolation.

[0068] After the aforementioned cross-dimensional coordinate rearrangement calculation, the depth dimension data of the input tensor is transformed into the spatial dimension. The height and width values ​​of the output feature tensor are increased proportionally according to a preset upsampling ratio parameter, generating an upsampled feature map with restored spatial resolution. This purely physical displacement operation maintains a constant total value of the feature tensor, preventing local feature weakening caused by interpolation algorithms when processing matrix regions containing dense small targets. The generated upsampled feature map is output and used as shared input data for subsequent prediction networks.

[0069] This embodiment specifically illustrates the process of allocating upsampled feature maps to independent network structures and performing density estimation, probability field generation, and region classification calculations respectively.

[0070] Obtain the upsampled feature map output by the preceding upsampled network nodes. Transmit the tensor data of the upsampled feature map synchronously and separately to the structurally independent first, second, and third branch networks. These three branch networks are arranged in parallel and each contains an independent trainable weight parameter matrix.

[0071] The first branch network is configured as a density distribution extraction module. It comprises a first convolutional layer, a linear rectified activation layer, and a second convolutional layer, all connected in series. The first convolutional layer performs channel feature extraction calculations, and the second convolutional layer's output channel count is set to 1. After the upsampled feature map is input into the first branch network, it undergoes forward propagation matrix multiplication operations as described above, and a non-negative numerical mapping is performed at the network's end, outputting a two-dimensional single-channel original predicted density map. The floating-point values ​​of each pixel in the original predicted density map represent the estimated number of targets contained within the corresponding spatial region of the original input image.

[0072] The second branch network is configured as a target location probability distribution extraction module. To avoid gradient vanishing during backpropagation while extracting local feature tensors, the second branch network includes a convolutional module configured with residual connections. The residual connection structure consists of a cascaded backbone convolutional layer and a parallel identity mapping shortcut. Upsampled feature maps are input to the backbone convolutional layer for feature computation to generate the backbone tensor, while the upsampled feature maps are simultaneously propagated along the identity mapping shortcut to generate a bypass tensor. The backbone tensor and the bypass tensor are then added element-wise. The resulting fused tensor is passed through a convolutional layer with a terminal output channel of 1 to output a two-dimensional single-channel optimal transmission probability field map. The pixel values ​​of the optimal transmission probability field map represent the relative probability weights of the existence of the target center point at the corresponding spatial coordinate position.

[0073] To address the misclassification issue caused by non-target regions (such as building surface structures, tree branches, etc.) resembling crowd textures in images, a third branch network is configured. This third branch network serves as a binary classification module for foreground and background. It comprises channel feature mapping convolutional layers and nonlinear binary classification activation network layers. The upsampled feature maps, after channel feature mapping calculations, are input into the nonlinear binary classification activation network layer.

[0074] The nonlinear binary classification activation network layer employs the Sigmoid activation function, performing exponential and mapping calculations on the channel feature scalars corresponding to each independent pixel coordinate. This calculation strictly limits the output value at each pixel coordinate to a real number between 0 and 1, outputting a foreground / background segmentation mask for each pixel. Pixel values ​​close to 1 in the foreground / background segmentation mask indicate a higher probability that the spatial region corresponding to that coordinate belongs to a foreground region containing a crowd target, while pixel values ​​close to 0 indicate a higher probability that the spatial region corresponding to that coordinate belongs to a background region without a target. Through the above three parallel branch structures, independent extraction and output of tensors with different physical meanings for the same input feature are achieved.

[0075] See attached document Figure 5 This embodiment specifically illustrates the matrix operation process of performing pixel-level spatial filtering on the density distribution prediction result and the probability field prediction result using the foreground-background segmentation mask image.

[0076] The original prediction density map output by the first branch network, the optimal transmission probability field map output by the second branch network, and the foreground / background segmentation mask map output by the third branch network are obtained. All three input data are two-dimensional single-channel feature tensors, and their height and width values ​​are strictly consistent in the spatial dimension.

[0077] Tensor element-wise multiplication is performed to generate the final density distribution. The floating-point value corresponding to each spatial pixel coordinate in the original predicted density map is multiplied by the classification probability value at the same spatial pixel coordinate position in the foreground / background segmentation mask image, outputting the target predicted density map. The mathematical expression of this calculation process is as follows: ; In the above formula, This represents the calculated target prediction density map; This represents the original prediction density map output by the first branch network; This represents the foreground / background segmentation mask output by the third branch network. This represents the Hadamard product operation, which is the calculation of multiplying elements in the same spatial position within a matrix one by one.

[0078] Synchronously perform tensor element-wise multiplication calculations to generate the final probability field. Multiply the floating-point value corresponding to each spatial pixel coordinate in the optimal transmission probability field map with the classification probability value at the same spatial pixel coordinate position in the foreground / background segmentation mask map, and output the target probability field map. The mathematical expression of this calculation process is as follows: ; In the above formula, This represents the calculated target probability field diagram; This represents the optimal transmission probability field diagram output by the second branch network.

[0079] During the forward propagation calculation described above, the pixel values ​​of each pixel in the foreground-background segmentation mask are within a closed interval of 0 to 1. For spatial regions determined by the network to have no target background, the pixel values ​​at the corresponding coordinate positions in the foreground-background segmentation mask approach 0. Through the Hadamard product operation described above, the floating-point values ​​in the original predicted density map and the optimal transport probability field map located in this background region are forcibly truncated and decayed to near 0 after being multiplied by a scalar that approaches 0.

[0080] The aforementioned multiplication operations constitute a hard numerical constraint on the output tensor. During forward propagation, invalid response features generated in non-target background regions are directly eliminated at the numerical level. When the computation graph enters the backpropagation stage to calculate the gradient, according to the chain rule of calculus, the classification error gradient backpropagation generated in the background region is confined to the third branch network. This ensures that the first and second branch networks do not need to handle false positive errors caused by similar background textures, guaranteeing mathematical decoupling of the parallel branch networks when calculating the target distribution weight update.

[0081] This embodiment specifically illustrates the calculation process of calculating various error losses based on the network output feature tensor and preset real labels, and updating network node parameters through backpropagation.

[0082] Obtain pre-constructed ground truth annotation data corresponding to the input image. Ground truth annotation data includes Gaussian smooth density labels generated based on ground truth coordinate points, scalar labels of the total number of ground truth populations, binarized region labels generated based on coordinate point dilation, and labels of ground truth point sets containing discrete two-dimensional coordinate data.

[0083] Based on the target predicted density map and Gaussian smoothed density labels, a pixel-by-pixel mean square error calculation is performed to generate a pixel-level density fitting loss. This calculation process evaluates the local numerical difference between the predicted and true values ​​in the spatial distribution, and its mathematical expression is as follows: ; In the above formula, This represents the pixel-level density fitting loss of the calculated output; This indicates the total number of pixels contained in the target prediction density map; Indicating the target prediction density map Predict floating-point values ​​for each pixel; Indicating Gaussian smoothing density labels The actual floating-point value of each pixel.

[0084] Based on the target predicted density map and the scalar label of the actual total population, a global spatial absolute error is calculated to generate a global count absolute error loss. All pixel values ​​in the target predicted density map are extracted and summed to obtain the predicted total scalar. The absolute value of the difference between the predicted total scalar and the actual total population scalar label is calculated. The formula for this step is as follows: ; In the above formula, This represents the global count absolute error loss in the calculated output; This represents the total number of predictions output by performing a global pixel summation operation on the target prediction density map; A scalar label representing the total number of the actual population.

[0085] To address the data imbalance where the number of background pixels without a target is significantly greater than the number of foreground pixels containing a target, a foreground segmentation focal point classification loss is calculated based on the foreground-background segmentation mask and binarized region labels. Penalty weights are added to the pixel coordinates with high classification error rates; the calculation formula is as follows: ; In the above formula, This represents the foreground segmentation and focus classification loss calculated from the output. Indicates the binarized region label in Each pixel represents a real floating-point value, which can be either 0 or 1. This indicates the foreground / background segmentation mask image in The actual floating-point value of each pixel; This represents the preset hard sample adjustment index parameter; This represents the preset positive and negative sample class balance weight parameter; This represents a logarithmic function with the natural constant as its base.

[0086] Based on the target probability field map and the real point set labels, the cost of point-to-point matching is calculated to generate the dynamically optimal transmission distance loss. The target probability field map is flattened into a one-dimensional distribution vector in the spatial dimension. The squared Euclidean distance between the physical coordinate space position corresponding to each pixel in the one-dimensional distribution vector and each real coordinate point in the real point set labels is calculated to construct a two-dimensional distance cost matrix. An iterative algorithm is used to solve for the transmission probability matrix that minimizes the total transmission cost under edge distribution constraints. The Hadamard product of the distance cost matrix and the transmission probability matrix is ​​calculated and summed. The calculation formula is as follows: ; In the above formula, This represents the dynamically optimal transmission distance loss calculated from the output. This indicates the total number of coordinate points contained in the set of real points. Represents the distance cost matrix in the nth The coordinates of the actual floating-point value of the nth pixel and the nth The squared Euclidean distance between the actual floating-point coordinates of each pixel; This represents the value of the corresponding element in the optimal transmission probability matrix output through iterative solution.

[0087] A linear weighted summation is performed on the four independently calculated loss values ​​to generate the total loss value used for backpropagation. The calculation steps are as follows: ; In the above formula, This represents the total loss value generated; These represent the preset constant hyperparameter weights for each loss term.

[0088] Based on the total loss value, the automatic differentiation engine of the deep learning framework is activated. During backpropagation, the gradient matrix of each trainable weight parameter matrix within the network computation graph is calculated according to the chain rule. After obtaining the gradient matrix, a preset parameter optimization algorithm is invoked to iteratively update the low-rank adaptation matrix, the mapping weights in the spatial feature fusion module, and the convolutional kernel parameters in the three parallel prediction branches. The original pre-trained parameter matrix in the large visual language model does not receive gradient backpropagation; its values ​​remain frozen and unchanged.

[0089] Specific application example: High-precision monitoring of dense crowds in complex urban squares Suppose this solution is applied to an intelligent security system in a large city center square. The square's environment is extremely complex, containing extensive areas of architectural brickwork textures, swaying tree branches and leaves, and dramatic changes in light and shadow. During holidays, the square is extremely crowded, and due to camera perspective limitations, pedestrians in the foreground and those in the background appear at vastly different scales in the images. In this scenario, traditional small-scale surveillance networks are highly susceptible to misinterpreting distant leaves or brickwork cracks as human heads, resulting in numerous false positives.

[0090] Once the surveillance camera captures a high-resolution image of the original crowd in the square, the system first fills its edges with zero pixels and divides it into multiple image blocks, which are then input into a large visual language model deployed on the backend computing device.

[0091] During the forward propagation phase of the model, the unique dynamic gating low-rank adaptation module of this invention begins to play a crucial role. Because the features of the plaza background (sky, flat ground) change gradually, while the features of the densely populated area change drastically, the system calculates the variance distribution vector by extracting the unbiased variance along the sequence dimension: ; This formula extracts the input feature tensor sequence. The spatial fluctuation characteristics are observed. In densely populated areas, the variance is high, the dynamic activation coefficient approaches 1, and the low-rank adaptation matrix is ​​fully activated to extract complex head features. In flat background areas without targets, the variance is low, the dynamic activation coefficient approaches 0, and the low-rank matrix is ​​hardly processed. This significantly reduces memory and computing power costs while achieving adaptive capture of local details in large models.

[0092] After feature extraction and upsampling, the feature tensor is fed into three parallel prediction branch networks. To completely suppress interference from building textures and foliage, the third branch network outputs a foreground-background segmentation mask that strictly defines the foreground and background. The system performs a multiplicative truncation hard-constraint forward propagation, transforming the original predicted density map output by the first branch network, which includes some background noise, into a single image. Perform Hadamard product operation with the mask image: ; Through the calculation of this formula, all floating-point values ​​identified as background areas of building brick walls or leaves are forcibly truncated to 0 by multiplying them by a mask probability approaching 0, thus outputting an extremely pure target prediction density map. Ultimately, for By performing a global integral, the precise total number of people in the square can be output.

[0093] To verify the authenticity and superiority of the proposed solution, quantitative experiments were conducted on mainstream large-scale dense population datasets (including dataset A and dataset B). The evaluation metrics used were mean absolute error (MAE) and root mean square error (RMSE), with lower values ​​indicating more accurate predictions and stronger generalization ability.

[0094] Compare to baseline: Traditional small-scale networks: classic dedicated counting models based on VGG backbones, lacking prior knowledge of large models.

[0095] Full fine-tuning of large models: Directly connect large visual language models to ordinary regression heads and perform brute-force full parameter fine-tuning and updates.

[0096] The method of this invention adopts a design of dynamic gated LoRA, spatial cross-layer feature routing, and three-branch decoupled truncation prediction.

[0097] Analysis of experimental results: See attached document Figure 6 ,exist Figure 6 In the figure, the horizontal axis lists the MAE and RMSE error metrics for datasets A and B, respectively, while the vertical axis represents the specific error values.

[0098] The dark black bars represent traditional small-scale networks, which exhibit extremely high errors on complex datasets (e.g., RMSE exceeds 220 for dataset B), revealing their lack of generalization robustness when facing complex and ever-changing scenarios.

[0099] The dark gray bars represent full fine-tuning of the large model. Although the error has decreased thanks to the global semantic understanding of the large model, the full fine-tuning destroys the pre-training knowledge and lacks a mechanism to protect high-frequency spatial details, resulting in a high MAE.

[0100] The light gray bars represent the method of this invention, and their height is the lowest among all indicators. Through the optimization strategy of segmentation-gated hard truncation in physical structure and suppression of background pixels, this invention completely suppresses false positive predictions, reducing the MAE to 52.3 on dataset A and the RMSE to 142.8 on the extremely dense dataset B. Experimental data strongly demonstrate that this scheme overcomes the counting bottleneck under complex background interference while reducing computational cost.

Claims

1. A method for adapting large models based on efficient parameter fine-tuning and lightweight counting heads, characterized in that: Includes the following steps: The original crowd image is sliced ​​and normalized to generate a normalized image feature tensor sequence; The normalized image feature tensor sequence is input into a visual encoder, the unbiased variance of the input feature tensor sequence is extracted to generate dynamic activation coefficients, and the dynamic activation coefficients are used to control a low-rank adaptive structure to obtain multi-layer local feature tensors. Calculate the spatial fusion weight matrix corresponding to the multi-layer local feature tensor, and perform weighted summation to generate a two-dimensional fusion feature map; The two-dimensional fused feature map is processed using a hollow spatial pyramid pooling module and sub-pixel convolution operations to output an upsampled feature map. The upsampled feature map is input into three parallel prediction branch networks, which output the original prediction density map, the optimal transmission probability field map, and the foreground-background segmentation mask map, respectively. The original prediction density map and the optimal transmission probability field map are multiplied and truncated using the foreground and background segmentation mask map, respectively, to output the target prediction density map and the target probability field map. Based on the target prediction density map, target probability field map, foreground / background segmentation mask map, and real labeled data, the loss is calculated and summed to update the parameters of the low-rank adaptive structure and the three parallel prediction branch networks.

2. The method for adapting a large model based on efficient parameter fine-tuning and a lightweight counting head according to claim 1, characterized in that, The step of normalizing the original crowd image segments to generate a normalized image feature tensor sequence includes: Based on the preset unit block size parameters of the visual encoder, calculate the target fill width remainder and the target fill height remainder; A corresponding number of zero-pixel value data is appended to the edges of the original crowd image to generate a filled image, and the filled image is divided into non-overlapping image blocks. For each of the image segments, the original pixel tensors are extracted and linear transformation normalization calculations are performed independently to output the normalized image feature tensor sequence.

3. The method for adapting a large model based on efficient parameter fine-tuning and a lightweight counting head according to claim 1, characterized in that, The step of extracting the unbiased variance of the input feature tensor sequence to generate dynamic activation coefficients, and using the dynamic activation coefficients to control the low-rank adaptive structure includes: Unbiased variance extraction is performed on the input feature tensor sequence to generate a variance distribution vector. The variance distribution vector is then input into a single-layer linear projection network and a nonlinear activation function to output the dynamic activation coefficients. The input feature tensor sequence is multiplied sequentially by the first low-rank dimensionality reduction matrix and the second low-rank dimensionality increase matrix in the low-rank adaptive structure, then multiplied element-wise by the dynamic activation coefficients, and the multiplication result is added element-wise to the tensor data output by the pre-trained weight matrix in the frozen state.

4. The method for adapting a large model based on efficient parameter fine-tuning and a lightweight counting head according to claim 1, characterized in that, The step of generating a two-dimensional fused feature map by weighted summation of the multi-layer local feature tensors based on the spatial fusion weight matrix corresponding to each depth layer includes: Using a linear transformation layer containing a concatenated weight matrix and a nonlinear activation function, the feature mapping scores of each of the multi-layer local feature tensors are calculated for the same spatial coordinate position. The feature mapping scores of each depth are subjected to exponential operation and normalization across the network depth dimension to generate the spatial fusion weight matrix corresponding to each depth layer. At the aligned spatial coordinates, the spatial fusion weight matrix is ​​multiplied by the corresponding multi-layer local feature tensor, and a cross-depth accumulation operation is performed to output the two-dimensional fusion feature map.

5. The method for adapting a large model based on efficient parameter fine-tuning and a lightweight counting head according to claim 1, characterized in that, The step of processing the two-dimensional fused feature map using the dilated spatial pyramid pooling module and sub-pixel convolution operations to output the upsampled feature map includes: The two-dimensional fused feature map is input into the dilated spatial pyramid pooling module, which contains a standard convolution branch and multiple dilated convolution branches with different dilation coefficients. Multi-scale feature tensors are extracted synchronously and then concatenated with the global feature tensor that has been upsampled and restored after global average pooling in the channel dimension. An intermediate high-dimensional feature tensor is obtained by expanding the number of channels of the concatenated and merged feature tensor through preset upsampling network nodes. According to the spatial block mapping rule, the values ​​of the intermediate high-dimensional feature tensor in the channel dimension are physically moved and interleaved to the adjacent pixel coordinate positions in the plane width and height dimensions to complete the sub-pixel convolution operation and generate the upsampled feature map.

6. The method for adapting a large model based on efficient parameter fine-tuning and a lightweight counting head according to claim 1, characterized in that, The steps of inputting the upsampled feature map into three parallel prediction branch networks and outputting the original prediction density map, the optimal transmission probability field map, and the foreground / background segmentation mask map include: The upsampled feature map is transmitted to a first branch network containing a first convolutional layer, a linear rectified activation layer, and a second convolutional layer in series, and the original predicted density map is output. The upsampled feature map is transmitted to the second branch network, which includes the backbone convolutional layer and the identity mapping shortcut. The backbone tensor and the bypass tensor are added element by element, and the optimal transmission probability field map is output. The upsampled feature map is transmitted to a third branch network containing a channel feature map convolutional layer and a nonlinear binary classification activation network layer to output the foreground / background segmentation mask map.

7. The method for adapting a large model based on efficient parameter fine-tuning and a lightweight counting head according to claim 1, characterized in that, The steps of multiplying and truncating the original prediction density map and the optimal transmission probability field map using the foreground-background segmentation mask map to output the target prediction density map and the target probability field map include: The Hadamard product operation is performed, and taking advantage of the fact that the pixel values ​​at the corresponding coordinate positions in the foreground-background segmentation mask are close to 0, the floating-point values ​​of the corresponding non-target background regions in the original prediction density map and the optimal transmission probability field map are forcibly truncated and attenuated, and the invalid response features generated in the non-target background regions are erased during forward propagation.

8. The method for adapting a large model based on efficient parameter fine-tuning and a lightweight counting head according to claim 1, characterized in that, The real labeled data includes Gaussian smooth density labels, real total population scalar labels, binarized region labels, and real point set labels; The loss calculation based on the target prediction density map, target probability field map, foreground / background segmentation mask map, and ground-labeled data includes performing a linear weighted summation operation on the following four loss terms: Pixel-level density fitting loss calculated based on the target predicted density map and the Gaussian smoothed density label; The global count absolute error loss is calculated by summing the global pixels of the target predicted density map and comparing it with the scalar label of the total real population. The foreground segmentation focus classification loss is calculated based on the foreground-background segmentation mask and the binarized region label; The dynamic optimal transmission distance loss is calculated based on the target probability field map and the real point set labels.

9. The method for adapting a large model based on efficient parameter fine-tuning and a lightweight counting head according to claim 8, characterized in that, The steps for calculating the dynamic optimal transmission distance loss based on the target probability field map and the real point set labels include: Flatten the target probability field map into a one-dimensional distribution vector, calculate the squared Euclidean distance between the physical coordinates of each pixel in the one-dimensional distribution vector and each real coordinate point in the real point set label, and construct a distance cost matrix. Under the constraint of edge distribution, solve for the transmission probability matrix that minimizes the total transmission cost, calculate the Hadamard product of the distance cost matrix and the transmission probability matrix and sum them to generate the dynamic optimal transmission distance loss.

10. The method for adapting a large model based on efficient parameter fine-tuning and a lightweight counting head according to claim 8, characterized in that, The steps for calculating the foreground segmentation focus classification loss based on the foreground-background segmentation mask and the binarized region labels include: To address the data imbalance where the number of background pixels without a target is much greater than the number of foreground pixels containing a target, penalty weights are added to the pixel coordinates with higher classification error rates between the binarized region label and the foreground / background segmentation mask, thus generating the foreground segmentation focus classification loss.

Images