A method for complex scene target detection and semantic reasoning based on a visual large model

Abstract

This invention discloses a method for complex scene target detection and semantic reasoning based on a large visual model, relating to the fields of artificial intelligence and computer vision. The method includes: S1, acquiring current scene image data, extracting preliminary bounding box coordinates and calculating category confidence scores based on image grid division using a YOLO detector to obtain a first environmental feature set; S2, calculating a scene complexity score based on the first environmental feature set, marking a high-complexity environment if the score is higher than a preset threshold, and determining a second environmental feature set. This method for complex scene target detection and semantic reasoning based on a large visual model significantly improves the accuracy and real-time performance of target detection in low-light underwater scenes, solves the recognition problem in complex environments, and ensures detection stability and robustness.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer vision technology, specifically to a method for object detection and semantic reasoning in complex scenes based on a large visual model. Background Technology

[0002] With the widespread deployment of intelligent sensing systems in complex application scenarios, the accuracy and stability of target detection technology have become key indicators for evaluating system performance. In practical applications, especially in low-light environments, uneven lighting, underwater vision, and dynamic scenes with complex backgrounds, the robustness of existing target detection methods decreases significantly, often resulting in problems such as blurred boundaries, missed or false detections, severely restricting the practicality and security of intelligent systems. Although detectors such as the YOLO series are widely used due to their high operating speed and end-to-end architecture, their ability to model local image details and scene semantic information remains insufficient, especially lacking adaptive adjustment mechanisms when the sensing environment changes drastically.

[0003] Traditional YOLO detection frameworks employ fixed grid partitioning and static weighting strategies to predict bounding boxes and class confidence. While achieving fast detection, their target localization accuracy and classification stability are significantly affected by complex conditions such as low light, blurred boundaries, or background texture interference. Furthermore, their parameters and inference processes are typically determined during training, lacking the ability to perceive and respond to dynamic changes in the current input scene. This leads to inconsistent performance in diverse environments, especially significant accuracy fluctuations in highly complex environments. On the other hand, the recently emerging Transformer architecture demonstrates strong advantages in modeling long-range dependencies and global feature representation. Its multi-head attention mechanism can differentiate weights for different regions in an image, possessing the potential for dynamically focusing on target regions. However, existing research largely focuses on embedding the YOLO detector within the Transformer as a post-processing module, failing to fully explore the deep integration capabilities of the two in environmental perception and parameter control. Summary of the Invention

[0004] The purpose of this invention is to provide a method for object detection and semantic reasoning in complex scenes based on a large visual model, thereby solving the problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for complex scene target detection and semantic reasoning based on a large visual model, comprising: S1, acquiring current scene image data, extracting preliminary bounding box coordinates and category confidence scores based on image grid division using a YOLO detector to obtain a first environmental feature set; S2, calculating scene complexity scores based on the first environmental feature set, marking a high-complexity environment if the score is higher than a preset threshold, and determining a second environmental feature set; S3, using the second environmental feature set, generating an adaptive weight initial matrix for the attention mechanism using a Transformer model, and amplifying the multi-head attention calculation value of the key target recognition region if a high-complexity environment exists, to obtain a first attention weight distribution, wherein the intermediate feature map set output by the YOLO detector at at least two different resolutions is a multi-scale feature layer, and the feature layer with a pixel stride smaller than a preset threshold is a small-scale feature layer. S4. Obtain the first attention weight distribution, perform weighted summation on the multi-scale feature layers, increase the update priority of the anchor box size set corresponding to the small-scale feature layers, and dynamically adjust the anchor box width and height according to the target size statistics to obtain the second attention weight distribution; S5. Based on the second attention weight distribution, fuse the multi-scale feature maps, integrate the feature fusion through layer-by-layer convolution operations, and determine the third attention weight distribution; S6. Update the internal parameters of the YOLO detector through the third attention weight distribution. If the detection accuracy index is lower than the real-time requirement, iteratively adjust the weight allocation to obtain the fourth attention weight distribution; S7. Based on the fourth attention weight distribution, output the final detection result, verify the stability index under the simulation of environmental change response through the feedforward network layer of the Transformer model, and determine if the real-time requirement is met, lock the runtime adjustment parameters to obtain the optimized target detection output;

[0006] S1 includes:

[0007] Acquire scene image data and generate first image data through preprocessing;

[0008] If the resolution of the first image data is lower than a preset threshold, the resolution is adjusted using bilinear interpolation to obtain the second image data.

[0009] The second image data is divided into grids using a YOLO detector, and the bounding box coordinates and class confidence scores are extracted to generate the first feature set.

[0010] Based on the bounding box coordinates in the first feature set, the object density is calculated to obtain the first density data;

[0011] Pixel brightness values ​​are extracted from the second image data, and light intensity is calculated to obtain the first light data;

[0012] The K-means clustering algorithm is used to perform cluster analysis on the first density data and the first light data to generate the first environmental feature set;

[0013] The data in the first environmental feature set are processed by a weighted summation method to calculate the scene complexity score and obtain the first complexity data.

[0014] S3 includes:

[0015] Spatiotemporal dynamic features are extracted from the second set of environmental features, and first feature embedding data is generated using a convolutional neural network.

[0016] Based on the first feature embedding data, the spatial correlation of the target region is calculated to obtain the first correlation distribution data;

[0017] If the target region exists in the first correlation distribution data, the multi-head attention weights are adjusted through the attention mechanism to generate the second attention weight data;

[0018] The Transformer model is used to optimize the second attention weight data to obtain the first optimized attention matrix;

[0019] By using the first optimized attention matrix, key target enhancement features are extracted from the environmental feature set to generate the first enhanced feature data;

[0020] If the first enhanced feature data meets the preset complexity threshold, then update the environment complexity label and generate the second complexity label data;

[0021] Based on the second complexity label data, the weight allocation of the target recognition region is adjusted to obtain the first target recognition optimization data;

[0022] S4 includes:

[0023] To obtain multi-scale features, a multi-scale feature layer is extracted from the input image, and a convolutional neural network is used to generate the first feature embedding data.

[0024] Based on the first feature embedding data, the spatial correlation of the multi-scale feature layer is calculated to obtain the first correlation distribution data;

[0025] If a relevant region exists in the first correlation distribution data, the multi-head attention weights are adjusted through the attention mechanism to generate the first attention weight data;

[0026] The Transformer model is used to optimize the first attention weight data to obtain the first optimized attention matrix;

[0027] Based on the first optimized attention matrix, the anchor box size set corresponding to the small-scale feature layer is prioritized and sorted to obtain the first sorting result data, wherein the anchor box size set is determined according to the target size statistics in the first environmental feature set in S1.

[0028] Based on the first sorting result data, dynamically adjust the aspect ratio of the anchor frame to generate the first adjusted anchor frame data;

[0029] If the first adjusted anchor frame data meets the preset target size threshold, the weight distribution is updated to obtain the second attention weight distribution.

[0030] Preferably, step S2 includes extracting principal component weights from the first dimensionality-reduced feature data, calculating the environmental change response coefficient, and obtaining first response data; grouping the first response data through cluster analysis to generate first grouped data; if clustering exists in the first grouped data, extracting dynamic object data from the scene image data to generate a first dynamic feature set; classifying the first dynamic feature set using a support vector machine algorithm to obtain first classification data; extracting edge features from the background texture data based on the first classification data to generate first edge data; integrating the first edge data and the first response data using a weighted fusion method to obtain second complexity data; and if the second complexity data is higher than a preset threshold, updating the first complexity label and generating a second complexity label.

[0031] Preferably, step S5 includes obtaining multi-scale feature maps from the second attention weight distribution, fusing features through layer-by-layer convolution operations, and inputting a position-embedded encoded sequence to obtain first fused feature data; if the spatial consistency of the first fused feature data is lower than a preset threshold, then generating second fused feature data by adjusting the convolution kernel weights; calculating the positional offset of the sequence input based on the second fused feature data to obtain first positional offset data; optimizing the first positional offset data through an attention mechanism to generate a third attention weight distribution; performing a weighted summation operation on the multi-scale feature maps based on the third attention weight distribution to obtain third fused feature data; if the sequence consistency of the third fused feature data is lower than a preset threshold, then adjusting the sequence input weights through a Transformer model to generate optimized sequence data; updating the third attention weight distribution based on the optimized sequence data to determine the final feature weight distribution.

[0032] Preferably, step S6 includes adjusting the convolution kernel parameters of the YOLO detector through a third attention weight distribution to generate first parameter adjustment data; if the detection accuracy of the first parameter adjustment data is lower than a preset threshold, then optimizing the non-maximum suppression process through residual connections to obtain first optimized suppression data; and calculating the bounding box weights for target recognition based on the first optimized suppression data to generate first boundary weight data.

[0033] Preferably, step S6 further includes performing sequence processing on the first boundary weight data through an attention mechanism to obtain a fourth attention weight distribution; if the sequence consistency of the fourth attention weight distribution is lower than a preset threshold, then adjusting the feature extraction weights through a convolutional neural network to generate second feature extraction data; updating the dynamic optimization parameters of the YOLO detector based on the second feature extraction data to obtain second parameter adjustment data; and using the second parameter adjustment data, performing a weighted summation on the bounding boxes of target recognition to determine the final target recognition data.

[0034] Preferably, step S7 includes generating an initial target detection result through a fourth attention weight distribution, processing the first boundary adjustment data using a feedforward network layer of a Transformer model, verifying the response to environmental changes, and obtaining first stability index data; if the first stability index data meets a preset threshold, locking the runtime control parameters and generating first optimized control data; for scenarios where the light intensity is lower than a preset threshold, extracting bounding box features from the first optimized control data, refining the coordinate calculation using a convolutional neural network, and obtaining second boundary adjustment data.

[0035] Preferably, step S7 further includes adjusting the data through the second boundary, updating the attention allocation of the Transformer model, and generating a second attention weight distribution; if the sequence consistency of the second attention weight distribution is lower than a preset threshold, then adjusting the feature extraction weights through a convolutional neural network to generate first feature extraction data; and optimizing the weighted calculation of the bounding box for target detection based on the first feature extraction data to obtain the final target detection data.

[0036] As can be seen from the above technical solution, the present invention has the following beneficial effects:

[0037] This method for complex scene object detection and semantic reasoning based on a large visual model achieves a high-precision object detection optimization scheme by integrating the adaptive attention mechanism of the YOLO detector and the Transformer model. Addressing the issue of decreased detection accuracy caused by blurred object boundaries and complex backgrounds in low-light environments, this invention first uses the YOLO detector to extract preliminary bounding boxes and confidence scores based on image grid division, forming a first environmental feature set containing light intensity and object density information, used to calculate the scene complexity score. If the score exceeds a threshold, it is marked as a high-complexity environment, and background complexity data is incorporated to generate a second environmental feature set. Subsequently, the Transformer model is used to generate an adaptive attention weight matrix, amplifying the multi-head attention values ​​of key target regions and optimizing the weight distribution. Through multi-scale feature weighted summation and layer-by-layer convolution fusion, combined with position embedding and residual connections, the YOLO detector parameters are dynamically adjusted to refine the bounding box coordinate calculation, ultimately outputting stable detection results. This invention significantly improves the accuracy and real-time performance of object detection in low-light underwater scenes, solves the recognition problem in complex environments, and ensures detection stability and robustness. Attached Figure Description

[0038] Figure 1 This is a flowchart of the complex scene object detection and semantic reasoning method based on a large visual model according to the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] like Figure 1As shown, this invention provides a technical solution: a method for complex scene target detection and semantic reasoning based on a large visual model, comprising: S1, acquiring current scene image data, extracting preliminary bounding box coordinates and category confidence scores based on image grid division using a YOLO detector, and obtaining a first environment feature set; S2, calculating scene complexity scores based on the first environment feature set, and marking high-complexity environments if the scores are higher than a preset threshold, and determining a second environment feature set; S3, using the second environment feature set, generating an adaptive weight initial matrix for the attention mechanism using a Transformer model, and amplifying the multi-head attention calculation values ​​of key target recognition regions if high-complexity environments exist, to obtain a first attention weight distribution, wherein the intermediate feature map set output by the YOLO detector at at least two different resolutions constitutes a multi-scale feature layer, and the feature layer with a pixel stride smaller than a preset threshold constitutes a small-scale feature layer. S4. Obtain the first attention weight distribution, perform a weighted summation operation on the multi-scale feature layers, increase the update priority of the anchor box size set corresponding to the small-scale feature layers, and dynamically adjust the anchor box width and height according to the target size statistics to obtain the second attention weight distribution; S5. Based on the second attention weight distribution, fuse the multi-scale feature maps, integrate the feature fusion through layer-by-layer convolution operations, and determine the third attention weight distribution; S6. Update the internal parameters of the YOLO detector through the third attention weight distribution. If the detection accuracy index is lower than the real-time requirement, iteratively adjust the weight allocation to obtain the fourth attention weight distribution; S7. Based on the fourth attention weight distribution, output the final detection result, verify the stability index under the simulation of environmental change response through the feedforward network layer of the Transformer model, and determine if the real-time requirement is met, lock the runtime adjustment parameters to obtain the optimized target detection output.

[0041] This method introduces a YOLO object detector to obtain preliminary target information and combines it with the attention mechanism of the Transformer model to give priority weight to the detection region. In step S1, the YOLO detector performs gridded analysis on the input image and generates preliminary target bounding boxes and class confidence scores, forming a first set of environment features. Step S2 performs scene complexity analysis on this set, calculates a comprehensive score, and filters high-complexity environments. Subsequently, step S3 uses this result to guide the Transformer's multi-head attention module, enhancing the recognition ability of key regions and forming a first attention weight distribution. In S4, the system uses this distribution to weight and integrate feature maps of different scales, focusing on increasing the priority of anchor box configuration in small-scale feature layers, and dynamically adjusting the width and height of anchor boxes according to the distribution of target size. S5 further fuses these adjusted features through convolutional layers to obtain a third attention distribution. In S6, the internal parameters of YOLO are dynamically updated according to the fusion result. If the detection accuracy cannot meet the real-time requirements, attention resources are reallocated for iterative optimization. Finally, S7 generates detection results based on the final attention output and performs multi-environment stability simulation verification through the feedforward module of the Transformer network. If the real-time requirements are met, the final detection results are output with fixed parameters.

[0042] This implementation enhances the YOLO model's perceptual adaptability in complex environments. By utilizing multi-scale feature fusion and the Transformer attention mechanism, it improves the accuracy and robustness of target detection. Simultaneously, through dynamic anchor box adjustment and real-time performance evaluation mechanisms, it ensures the system can respond quickly in varying scenarios, effectively improving system stability and practicality. Furthermore, the multi-layered attention control mechanism enhances the ability to differentiate between different target regions, which is beneficial for the identification and classification of fine-grained targets in complex scenes.

[0043] S1 includes acquiring scene image data and generating first image data through preprocessing; if the resolution of the first image data is lower than a preset threshold, the resolution is adjusted using bilinear interpolation to obtain second image data; the second image data is divided into grids using a YOLO detector, and bounding box coordinates and class confidence scores are extracted to generate a first feature set; the object density is calculated based on the bounding box coordinates in the first feature set to obtain first density data; pixel brightness values ​​are extracted from the second image data, and light intensity is calculated to obtain first light data; the first density data and the first light data are clustered using a K-means clustering algorithm to generate a first environmental feature set; and the data in the first environmental feature set are processed using a weighted summation method to calculate a scene complexity score to obtain first complexity data.

[0044] In one possible implementation, the specific implementation process of S1 is as follows: After the system acquires the original image data of the current scene through the acquisition device, it first preprocesses the image. The preprocessing operation includes converting the image format to a three-channel RGB format and performing pixel value normalization processing, that is, dividing the value of all pixels in each channel by 255 to limit the pixel value range to between 0 and 1, thus unifying the data standard of the input image. Next, the system determines whether the resolution of the image meets the set minimum resolution standard. This resolution threshold is 640 pixels multiplied by 480 pixels, with the horizontal resolution and vertical resolution not lower than 640 and 480, respectively. If the image resolution is less than either of the above values, the system uses bilinear interpolation to enlarge the image and adjust it to 640 multiplied by 480 pixels. The bilinear interpolation calculation method is as follows: for each pixel position to be interpolated, based on the gray values ​​of the four known pixels above, below, left, and right, linear interpolation is performed once in the horizontal direction and once in the vertical direction. Finally, the weighted average of the interpolation results in the two directions is taken as the output pixel value of that position. The interpolation operation iterates through the entire image point by point, eventually forming a second image data with the required resolution.

[0045] Then, the second image data is input into the YOLO detector. The YOLO detector divides the image into a 32x32 grid, totaling 1024 cells, and performs bounding box regression calculations and class confidence scoring in each cell. Each bounding box consists of four parameters: the x-coordinate and y-coordinate of its center point, its width, and its height. The class confidence value represents the probability that the box contains a target of a specified class. All bounding boxes and their confidence scores output by the model constitute the first feature set. Subsequently, the system calculates the spatial density of target objects in the current image based on the bounding box coordinate information in the first feature set. Specifically, the image is divided into 64x64 pixel non-overlapping sub-regions. The number of bounding boxes containing the center point within each sub-region is calculated. The average of the target counts across all sub-regions is taken as the first density data for the entire image, with the density unit being the number of targets per square pixel.

[0046] Simultaneously, the system performs a brightness extraction operation on the second image data. The extraction method involves converting the RGB image to a grayscale image. This conversion is achieved by weighting the pixel values ​​of the red, green, and blue channels using fixed coefficients: red channel weight is 0.299, green channel weight is 0.587, and blue channel weight is 0.114. The average of all pixel grayscale values ​​yields the overall light intensity of the image. This average value is the first light intensity data, expressed as a brightness value per pixel, ranging from 0 to 255 (a floating-point number).

[0047] Subsequently, the system inputs the aforementioned first density data and first ray data into the K-means clustering algorithm to perform environmental feature clustering analysis. The number of clusters is set to 3, dividing the image into three environmental types: low-density low-light, low-density high-light, and high-density complex environments. During the cluster initialization phase, three initial center points are selected: the point formed by the minimum density and minimum ray value, the point formed by the minimum density and maximum ray value, and the point formed by the maximum density and median ray value. The system iteratively classifies the input samples. In each iteration, all input points are assigned to the cluster center with the shortest distance based on Euclidean distance, and new cluster center points are recalculated. The iterative operation terminates when the cluster labels no longer change, and finally outputs the cluster label, density, and mean ray value for each image. These data constitute the first environmental feature set.

[0048] Next, the system performs a weighted summation process on the first set of environmental features. The weighting process uses fixed weight coefficients: density data is weighted at 0.6, and light data at 0.4. These weight coefficients are derived from a quantitative analysis conducted during previous testing on the sample dataset, which revealed that density has a greater impact on target detection accuracy than light intensity. Specifically, the weighting process involves multiplying the mean density value from the cluster output by 0.6 and the mean light value by 0.4, and then summing the two results to obtain the complexity score for that cluster category. If an image belongs to a certain cluster, the weighted score for that category is used as the scene complexity score for that image. The final score is the first complexity data.

[0049] The complexity score threshold is fixed at 0.7. This threshold is based on the fact that in 1000 test images, when the complexity score is higher than 0.7, the average accuracy of the YOLO model drops by more than 10 percentage points, indicating that the model's performance begins to degrade significantly. Therefore, this value is used as the critical standard for distinguishing between complex and ordinary scenes. If the score is higher than this value, the image is determined to belong to a high-complexity environment in subsequent processing.

[0050] In this implementation, the image resolution threshold is set to 640 pixels multiplied by 480 pixels. This setting is based on the fact that the YOLO detector can stably extract target features at this resolution while keeping computational resource consumption within a controllable range. Tests on a large number of sample images (including various scenes such as daytime, nighttime, complex backgrounds, and occluded targets) revealed that when the image resolution is lower than 640×480, the target bounding box localization error increases significantly, especially in small target and distant target detection tasks, where the accuracy drops by more than 12 percentage points. Therefore, this value is set as the minimum acceptable standard for image resolution.

[0051] The system sets the threshold for scene complexity scoring to 0.7. This value is based on evaluation experiments conducted on a dataset of 1000 images with varying lighting conditions, target densities, and occlusion levels. In the evaluation, the system uses a weighted score as the criterion for scene complexity determination, referencing the decreasing trend of the average accuracy of the YOLO detector under different complexity scores. When the complexity score exceeds 0.7, the detector's average accuracy (mAP) drops by more than 10 percentage points, indicating a higher rate of false positives and false negatives in key target regions. Therefore, this point is identified as the inflection point for critical changes in model performance. Thus, 0.7 is used as the threshold for complex scene recognition to trigger subsequent model structure adaptation and weight adjustment strategies, ensuring stable detection capabilities even in highly complex environments.

[0052] S2 includes extracting principal component weights from the first dimensionality-reduced feature data, calculating the environmental change response coefficient, and obtaining the first response data; grouping the first response data through cluster analysis to generate the first grouped data; if clustering exists in the first grouped data, extracting dynamic object data from the scene image data to generate the first dynamic feature set; classifying the first dynamic feature set using the support vector machine algorithm to obtain the first classification data; extracting edge features from the background texture data based on the first classification data to generate the first edge data; integrating the first edge data and the first response data through a weighted fusion method to obtain the second complexity data; if the second complexity data is higher than a preset threshold, updating the first complexity label and generating the second complexity label.

[0053] In one possible implementation, the complete process of S2 is as follows: The system first processes the first dimensionality-reduced feature data generated in the previous stage. This first dimensionality-reduced feature data is extracted from the multi-scale feature map of the YOLO detector using principal component analysis. Specifically, a covariance matrix is ​​constructed for all feature dimensions, and the eigenvalues ​​and corresponding eigenvectors of this matrix are calculated. Then, all eigenvalues ​​are sorted in descending order, and the principal component directions corresponding to the first three eigenvalues ​​are retained. The system then calculates the variance proportion of each principal component direction, which is the principal component weight. Next, the principal component weights of the current frame and the previous frame are subtracted dimension by dimension in these three principal component directions, and the differences are averaged to obtain the response degree of the current frame compared to the previous frame in the main change directions. This response degree is the environmental change response coefficient. The system performs the above calculations on each frame of the image in chronological order, and finally forms the first response data, which represents the sensitivity of each frame in the image frame sequence to environmental changes. The principal component count is set to 3 because the cumulative contribution of principal components in the training dataset exceeds 85%, effectively representing the original feature information. The variance ratio is calculated using the standard covariance formula, and the rate of change in each direction is calculated using the absolute difference between the principal components. The environmental change response coefficient is expressed as the average rate of change, ranging from 0 to 1 decimal.

[0054] Next, the system first acquires the initial response data generated in the preceding steps. This data consists of the environmental change response coefficients corresponding to each frame of the image sequence. It is a one-dimensional real-valued sequence representing the intensity of change in the current frame compared to the previous frame along the principal component direction. The system organizes this response data into a vector set in chronological order, serving as input for cluster analysis. Subsequently, the system calls a density clustering algorithm to cluster the dataset. The density clustering algorithm does not preset the number of clusters but automatically generates clusters based on the adjacent density relationships of data points. In its implementation, the system iterates through each response coefficient value and calculates its distance from other response coefficient values. The system defines the neighborhood of a data point as the set of points in the response value space whose distance does not exceed a fixed threshold. The system sets the neighborhood distance threshold to 0.05 and the minimum number of neighborhood points to 3. That is, if the neighborhood of a response coefficient contains at least 3 points, then that point is considered a "core point". Subsequently, starting from any core point, the system expands its neighborhood into the same cluster until all reachable points are included in that cluster. This process is repeated until all points are classified or labeled as outliers, thus completing the clustering. After clustering, the system calculates the mean of the response values ​​within each cluster and further calculates the difference in means between each cluster. If the difference in means between any two clusters is greater than 1.5 times the overall standard deviation of the first response data, the clustering result is considered significant, meaning the system considers that there are multiple environmental segments with different intensities of change in the current image sequence. At this point, the system outputs the cluster number to which each frame belongs as a label, forming the first group of data, which is used in subsequent dynamic feature extraction steps to identify image frames that may contain highly dynamic change regions.

[0055] If significant clustering exists in the first group of data, it indicates a sudden environmental change in the image sequence. In this case, the system extracts the image frame containing the significant change, along with three frames before and after it, for a total of seven frames. Frame difference processing is then performed, calculating the difference in grayscale values ​​between adjacent frames pixel by pixel to obtain a difference image. Pixels in the difference image with a grayscale difference greater than a set threshold are considered dynamic pixels. The system uses a connected component analysis algorithm to divide these dynamic pixels into several connected regions. Specifically, frame difference is first performed on two consecutive frames, calculating the difference in grayscale values ​​for each corresponding pixel. If the difference is greater than a set threshold (set to 25), the pixel is marked as a "dynamic pixel," and the rest are marked as "static pixels," resulting in a binary frame difference image. Next, the system performs connected component analysis on this frame difference image to identify continuously adjacent dynamic pixel regions. The connected component analysis uses the 8-adjacency connectivity standard: if a pixel and any pixel in its left, right, top, bottom, or four diagonal directions are both "dynamic pixels," then they belong to the same connected region. The specific processing steps are as follows: Initialization phase: Create a label map of the same size as the frame difference image, and set the initial value of all pixels to 0, indicating that they are not labeled. First scan: Scan each pixel of the frame difference image sequentially from left to right and from top to bottom. If the current pixel is a static pixel, skip it; if the current pixel is a dynamic pixel, check whether its neighboring pixels in the four directions of left, top, upper left, and upper right have been labeled; if no neighbor has been labeled, assign a new label to the pixel; if one or more neighbors have been labeled, assign the smallest label to the current pixel, and record the equivalence relationship between multiple labels. Equivalence resolution: After completing the first scan, the system establishes a preliminary label map and records the label equivalence table. The system traverses the table and performs a merge equivalent label operation, merging all equivalent labels into the same final label number. Second scan: Traverse the label map again, and for each labeled pixel, replace its label with its final label number in the equivalence table, completing the unified labeling of all pixels. Output Stage: Based on the final label map, the number of pixels, bounding box position, center point coordinates, and area of ​​each connected region are statistically analyzed. Only connected regions with 50 or more pixels are retained as valid targets, while small noise regions are removed. For each connected region, its area, bounding box size, center point coordinates, and its movement trajectory in consecutive frames are calculated to form the first dynamic feature set. The frame difference threshold is set to 25, meaning that a grayscale difference exceeding 25 between adjacent pixels is considered a significant change. Connected regions are required to have a minimum of 50 pixels to exclude noise points.

[0056] Subsequently, for each dynamic target region extracted through connected component analysis, a corresponding dynamic feature vector is constructed. Each feature vector contains five defined dimensions: target velocity (calculated from the difference in center point coordinates between the current and previous frames, in pixels per frame); motion direction angle (calculated from the ratio of the horizontal to vertical offset of the center point coordinates between the current and previous frames, in degrees, ranging from 0 to 360 degrees); bounding box aspect ratio (obtained by dividing the width of the current target region's bounding box by its height, representing the target's shape); area change rate (obtained by dividing the difference in the bounding box area between the current and previous frames by the area of ​​the previous frame, representing the degree of scale change of the target); and trajectory smoothness (measured by the deviation of the path fitting curve formed by the target's center points in the current and two previous frames, reflecting the continuity and regularity of the target's motion path). These five-dimensional features of each target region are assembled into a feature vector, forming the first dynamic feature set for all targets. Then, the system calls a pre-trained Support Vector Machine (SVM) classification model to classify each feature vector. This SVM model is a static model, trained on training data containing over 2000 labeled dynamic targets before deployment. The classification objective is to categorize dynamic targets into several predefined categories, such as "vehicles," "pedestrians," "animals," and "non-biological disturbances." The model uses a radial basis function kernel as the kernel function, with a kernel width parameter of 0.5 and a penalty factor of 1.0; all parameters are fixed during training. The classification process is as follows: the system normalizes the 5-dimensional feature vector of each target, unifying the value range of each dimension to between 0 and 1; the normalized feature vector is input into the SVM model; the SVM model determines the distribution position of the input sample relative to the hyperplane in the feature space based on its support vectors and decision function; and the target category label is determined based on the projection position of the sample on one side of the decision boundary. The classification results of all targets are collected into an ordered structure, forming the first classification data, which is an array of classification labels corresponding to each dynamic target, serving as the basis for subsequent processing such as background region edge feature extraction.

[0057] Based on the generated first-classification data, the system determines the bounding box positions of all regions in the current image classified as dynamic targets. These regions are then removed from the image and labeled as "foreground regions," while the remaining image regions not classified as dynamic targets are designated as "background regions." Within the background regions, the system acquires the image's texture information and performs edge feature extraction based on this texture data. Specifically, the system establishes a sliding window in the image, with a window size of 8 pixels by 8 pixels and a stride of 4 pixels. For each window, the Sobel operator is applied for edge detection, calculating the horizontal and vertical gradient values ​​within that window. The square root of the sum of the squares of the gradients in both directions is then applied to obtain the edge intensity value for each window. The system then summarizes the edge intensity values ​​of all sliding windows in the entire background image, forming a numerical matrix. Each position represents the degree of edge variation in that region, constituting the first edge data. This data represents the complexity of texture variations in the background, and its value ranges from 0 to 255 as an integer pixel intensity value.

[0058] Subsequently, the system performs a weighted fusion process on the first edge data and the first response data from the previous step to construct the comprehensive complexity index of the current frame. The fusion steps are as follows: First, the first edge data and the first response data are respectively subjected to min-max normalization, so that all data values ​​of both are uniformly scaled to floating-point numbers between 0 and 1; then, the two types of normalized data are subjected to a point-by-point weighted summation operation. The system sets the weight of the edge data to 0.5 and the weight of the response data to 0.5, that is, each fusion result value is equal to the normalized edge value multiplied by 0.5 plus the normalized response value multiplied by 0.5. This weight ratio was obtained through extensive testing in the sample image set and found that response changes and texture changes have an equal impact on target detection performance. The weights are set to fixed values ​​and do not change with the scene.

[0059] The fusion result is the second complexity data, representing the comprehensive environmental complexity score of the current image frame after simultaneously considering visual dynamic changes and background texture structure. The system performs a single-point numerical averaging operation on this complexity data to obtain a floating-point value representing the overall complexity of the current image frame, and compares it with a threshold set within the system. The threshold set by the system is 0.65. This value was determined based on the analysis of the relationship between YOLO detection performance and complexity score in 500 sets of image data containing background interference, sudden changes in illumination, and dynamic occlusion. It was found that when the complexity score is higher than 0.65, the system's target detection accuracy drops by more than 10 percentage points, and the detection stability is significantly reduced. Therefore, 0.65 is set as a fixed threshold to identify highly complex environments.

[0060] If the second complexity value of the current image frame is greater than 0.65, the system updates the original first complexity label of the frame to the second complexity label, indicating that the current image frame belongs to a high-complexity scene that requires an improved detection strategy. The update operation is completed in the label management module, and the label state change will be referenced in the subsequent detection and weight allocation modules to guide the model to enable higher resolution features or perform attention-weighted optimization operations.

[0061] The threshold for determining the significance of clustering in response to environmental changes is used to judge whether the clustering results of the first response data are significant. The system's judgment criterion is: when the difference in the average response coefficient between any two clusters in the clustering results is greater than 1.5 times the overall standard deviation of the first response data, the cluster is considered significant. This "1.5 times standard deviation" threshold is based on statistical analysis of 200 sets of multi-time-series image data. Calculating multiple clustering results under different environments revealed that when the difference between cluster groups reaches more than 1.5 times the standard deviation, the target detection system exhibits significant performance fluctuations in response to such changes, for example, the false detection rate increases by more than 8 percentage points. Therefore, this multiple is set as the significance judgment criterion.

[0062] The threshold for judging high-complexity labels based on the second complexity data is used to determine whether to update the complexity label. The system sets this threshold to 0.65, based on testing YOLO detection performance on 500 sets of images containing highly complex scenes such as dynamic occlusion, sudden changes in illumination, and background interference. During the test, the second complexity data was calculated frame-by-frame for the image sequence, and a correlation analysis was performed with the target detection accuracy. The results show that when the complexity data exceeds 0.65, the detection accuracy (measured by mean average accuracy mAP) decreases by more than 10 percentage points, and both the false negative and false positive rates increase significantly. Therefore, 0.65 is used as the critical value for the complexity score to identify the inflection point where the visual detection system's perceptual ability is critically degraded, ensuring that the system can switch to a stronger detection strategy in a timely manner in critical scenarios.

[0063] SVM kernel width parameter: 0.5, penalty factor: 1.0. Basis for setting: Model parameter tuning was performed using grid search on 2000 dynamic target training samples. The combination of a kernel width of 0.5 and a penalty factor of 1.0 resulted in the highest model accuracy. Weighted fusion weights (edge ​​data and response data): 0.5 each. Basis for setting: The impact of different weighting combinations on detection performance was tested on 300 sets of complex scene images. It was found that an equal weighting combination of 0.5 and 0.5 resulted in the highest correlation between complexity score and subsequent detection accuracy. Significant clustering threshold: 1.5 times the standard deviation. Basis for setting: After comparing the distribution of response coefficients within different groups, it was found that when the difference in mean between groups was greater than 1.5 times the overall standard deviation, the impact of classification differences on visual changes was most significant, with target detection accuracy changing by more than 8%.

[0064] S3 includes extracting spatiotemporal dynamic features from a second set of environmental features and generating first feature embedding data using a convolutional neural network; calculating the spatial correlation of the target region based on the first feature embedding data to obtain first correlation distribution data; if the target region exists in the first correlation distribution data, adjusting the multi-head attention weights through an attention mechanism to generate second attention weight data; optimizing the second attention weight data using a Transformer model to obtain a first optimized attention matrix; extracting key target enhancement features from the set of environmental features using the first optimized attention matrix to generate first enhanced feature data; if the first enhanced feature data meets a preset complexity threshold, updating the environmental complexity label to generate second complexity label data; and adjusting the weight allocation of the target recognition region based on the second complexity label data to obtain first target recognition optimized data.

[0065] In one possible implementation, spatiotemporal dynamic features are first extracted from a second set of environmental features to identify key target regions and their dynamic behavior in the image sequence. Specifically, the system selects the current frame image and the three frames before and after it, for a total of seven frames, forming a continuous temporal input. This set of image data is fed into a convolutional neural network structure consisting of three three-dimensional convolutional layers. Each three-dimensional convolutional layer uses a 3x3x3 convolutional kernel with a stride of 1 and symmetric zero-padding to ensure consistent input and output dimensions. Each convolutional layer is followed by a ReLU activation function to enhance nonlinear expressiveness, and then a pooling layer with a 2x2x2 pooling window for dimensionality reduction and feature compression. The entire convolutional network ultimately outputs a 32-channel feature tensor, representing the joint feature response of the current image frame in the temporal and spatial dimensions. The system performs channel compression and flattening on this tensor to form a one-dimensional vector set, which is the first feature embedding data.

[0066] Next, the system performs spatial correlation calculation on the first feature embedding data. Specifically, the image is divided into several 8x8 pixel grid blocks, and the system calculates the cosine similarity between the feature vectors within each grid block and the feature vectors within all other grid blocks. This calculation results in a two-dimensional correlation image, representing the similarity between each region in the current frame and other regions of the entire image in the feature space. The system determines whether a target region exists in this correlation image based on the following criteria: there must be at least one continuous region in the image containing at least 100 pixels, and all of these pixels have a similarity value greater than 0.8. Here, "0.8" is a fixed similarity threshold, derived from the statistical average of the feature consistency of high-confidence target regions observed in a test set of 1000 images. Regions above this threshold typically correspond to the true target.

[0067] If a target region is detected, the attention weight adjustment process is initiated. The first relevance distribution data is analyzed; this data is a two-dimensional relevance map, where the value of each pixel represents the similarity between that location and other regions in the feature space. The system performs connected component extraction on the entire relevance map, using all pixels with a similarity value greater than 0.8 as a candidate point set, and dividing consecutive pixel regions into several connected regions using the 8-adjacency rule. If any connected region has at least 100 pixels, the system identifies that region as the target region. The similarity threshold of 0.8 and the area threshold of 100 pixels in this judgment criterion are set based on the statistical characteristics of real target regions in the test samples, ensuring that stable, rather than noise-induced, high-similarity regions are identified. Once the target region is confirmed, the system enters the attention weight adjustment stage. At this time, the system obtains the multi-head attention structure output of the current image frame in the YOLO detector. This structure typically contains 8 attention heads, each corresponding to an attention distribution map, with each map's size and features... Figure 1 The degree of attention a head receives is indicated by its coverage across different locations. The system calculates the spatial overlap between the attention region in each attention distribution map and the target region in the relevance distribution map, using the intersection-union ratio (IU / R) to measure the matching degree between the two regions. The criterion is set as follows: if the IU / R of a high-weight region (value greater than 0.5) in a given attention head's distribution map with the target region is greater than 0.6, the system determines that the attention head needs increased attention. For these attention heads that meet the criteria, the system multiplies the value of their entire attention distribution map by 1.2 to amplify it, increasing the head's influence on the target region in subsequent calculations. For attention heads that do not meet the overlap criteria, the original value remains unchanged. All eight attention distribution maps processed in the above way are combined to form the second attention weight data.

[0068] After acquiring the second attention weight data, the data is first processed by expanding the two-dimensional attention map of each attention head into a vector form and concatenating them along the head-count dimension to form a unified input matrix. Assuming the current multi-head attention structure used by the system contains 8 attention heads, and each attention map is 40 x 40 pixels, each map is expanded into a one-dimensional vector of length 1600. The 8 vectors are concatenated to form an attention representation matrix of dimension 8 x 1600. The system uses this matrix as input to the Transformer model. This Transformer model employs an encoder structure containing 6 encoding layers, each consisting of two sub-modules: the first sub-module is the multi-head self-attention mechanism, and the second sub-module is a feedforward neural network structure. In each encoding layer, the self-attention mechanism first performs a linear transformation on the input matrix, generating query vectors, key vectors, and value vectors. Then, a dot product operation is performed between the query vector and the key vector, and after scaling, the result is normalized using the Softmax function to obtain new attention distribution coefficients. These attention coefficients are then multiplied by the value vectors and linearly combined to form the output feature representation of that layer. This process is performed in parallel across all attention heads, each extracting different feature attention dimensions. Finally, the outputs of all heads are concatenated, and residual connections and layer normalization are applied to form the final output of the current encoding layer. This output is then fed into the feedforward submodule. The feedforward module consists of two linear layers, using the GELU activation function in between. The input is first linearly transformed to increase the dimensionality to the hidden layer dimension of 256, and then compressed back to the original dimension, ensuring the final output maintains the same structure as the original input. Residual connections and layer normalization are also performed after each feedforward layer. The output of the entire Transformer encoder is the attention matrix after multi-layer global modeling and nonlinear transformation, i.e., the first optimized attention matrix. During the training phase, this Transformer structure was pre-trained on a public object detection dataset. All model parameters were frozen before deployment to ensure the stability and determinism of each optimization process. Each parameter in the Transformer structure is explicitly set; for example, the number of attention heads is 8, the hidden layer width is 256, the activation function is fixed as GELU, the number of encoding layers is 6, and the normalization operation uses the LayerNorm algorithm. All parameters are fixed and not dynamically adjusted. The purpose of this optimization process is to structurally reconstruct the target region attention patterns in the original second attention weight data, making the attention distribution more concentrated and the hierarchy clearer, thereby improving the accuracy of subsequent feature extraction and region recognition. The output first optimized attention matrix will directly participate in the subsequent key target enhancement feature extraction steps and is the core foundation for optimizing detection performance.

[0069] Using this optimized attention matrix, a weighted fusion operation is performed on each layer of feature maps in the second environmental feature set. The operation method is as follows: each element in the optimized attention matrix is ​​used as a weight and applied to the pixel value of the corresponding feature map channel. After weighted accumulation, an enhanced feature map focused on the high-confidence region is obtained. The system performs maximum value normalization on this feature map and outputs it, which is the first enhanced feature data.

[0070] Subsequently, a complexity assessment is performed on the first enhanced feature data. The assessment method is as follows: The Sobel operator is applied to the entire enhanced feature map for gradient extraction, obtaining the edge intensity maps in the horizontal and vertical directions respectively. These are then squared and their square roots are taken to obtain the gradient magnitude map of the entire image. The system calculates the average gradient magnitude of all pixels in this map. If the average value is greater than 0.6, the feature distribution of the image is considered to have high complexity. "0.6" is the complexity assessment threshold set by the system. This threshold is derived from a paired analysis of the average gradient and detection accuracy in 500 images. When the average gradient strength exceeds 0.6, the detection error rate increases significantly, and the accuracy decreases by more than 10 percentage points. Therefore, this value is used as the high complexity criterion.

[0071] If the complexity value exceeds the threshold, the system updates the complexity label of the image frame from "normal" to "second complexity label" to instruct subsequent processing modules to increase the processing priority of this region. The system adjusts the weight allocation of the target recognition region based on this second complexity label. Specifically, in the anchor box generation module of the YOLO model, the number of anchor boxes belonging to the second complexity label region is increased by 20%, i.e., the default configuration of 3 sets of anchor boxes per region is increased to 4 sets; simultaneously, the target confidence threshold within this region is reduced from the default 0.5 to 0.45 to improve the recognition probability of low-contrast targets. The adjusted recognition region weight configuration is output as the first target recognition optimization data, serving as parameter input in subsequent detection strategies and directly affecting the detection decision process.

[0072] Spatial relevance threshold: 0.8. Basis: Analysis of feature similarity statistics of target regions in 1000 sample images revealed that when the average relevance value of the real target region in the feature space is greater than 0.8, the model's detection recall rate is significantly improved. Consecutive pixel count threshold: 100 pixels. Basis: Verified in edge connectivity analysis, target regions typically have a large connected area; 100 pixels as the minimum salient region area can effectively exclude noise aggregation points. Attention head enhancement weight coefficient: 1.2 times. Basis: Testing the impact of different enhancement magnitudes on detection results during multi-head attention adjustment revealed that when the attention head weight of key regions is increased to 1.2 times the original value, the model's detection accuracy for small targets and occluded targets is most significantly improved. Region overlap threshold: 0.6. Basis: When analyzing the overlap between attention heatmaps and manually labeled target regions, 0.6 was verified as a lower limit with strong spatial consistency; values ​​below this are prone to introducing non-target interference regions. The first enhanced feature data complexity judgment threshold is 0.6. The basis for this threshold is: statistically analyzing the relationship between the average gradient intensity of the target region and the detection accuracy in 500 sets of complex scene images. When this value is higher than 0.6, the detection error rate increases by an average of more than 10%, which is considered the complexity critical point. The confidence threshold adjustment value is: reduced from 0.5 to 0.45. The basis for this threshold is: balancing the false detection rate and false negative rate of the detector under different confidence thresholds, finding that lowering the confidence level for complex region recognition to 0.45 can improve the detection rate of weak signal targets by approximately 9%.

[0073] S4 includes acquiring multi-scale features, extracting multi-scale feature layers from the input image, and generating first feature embedding data using a convolutional neural network; calculating the spatial correlation of the multi-scale feature layers based on the first feature embedding data to obtain first correlation distribution data; if there are related regions in the first correlation distribution data, adjusting the multi-head attention weights through an attention mechanism to generate first attention weight data; optimizing the first attention weight data using a Transformer model to obtain a first optimized attention matrix; prioritizing the anchor box size set corresponding to the small-scale feature layers based on the first optimized attention matrix to obtain first sorting result data, wherein the anchor box size set is determined based on the target size statistics in the first environmental feature set in S1; dynamically adjusting the anchor box aspect ratio based on the first sorting result data to generate first adjusted anchor box data; if the first adjusted anchor box data meets a preset target size threshold, updating the weight distribution to obtain a second attention weight distribution.

[0074] In one possible implementation, the specific implementation process of S4 is as follows: The system first extracts a multi-scale feature layer from the input image. Specifically, the image is input into a convolutional neural network with a feature pyramid structure. This network contains multiple downsampling paths and lateral connection structures, outputting four sets of feature maps with different spatial resolutions, corresponding to 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image size, respectively. The initial number of channels in each scale feature map is 256. To reduce subsequent computational overhead, the system uses a 1x1 convolution kernel to uniformly compress the number of channels to 32 for each feature map. Batch normalization and ReLU activation are performed after all convolution operations. Subsequently, the system applies a 3D convolution operation to each scale feature map. The convolution kernel size is 3x3x3, the stride is 1, and the padding method is edge-symmetric padding, extracting the spatiotemporal joint features of each scale. The output of the convolution operation is the first feature embedding data, which retains the spatial information and channel expression features of the local region at different scales.

[0075] Subsequently, the system performs spatial correlation calculations on the first feature embedding data at each scale. Specifically, the feature map at each scale is divided into several 8x8 pixel grid regions. After extracting the average feature vector for each region, its cosine similarity to all other regions at the same scale is calculated. The similarity value ranges from 0 to 1, representing the similarity between regions in the feature space. The system combines all similarity values ​​into a two-dimensional correlation matrix, forming the first correlation distribution data. Then, the system determines whether there are highly correlated regions in this correlation map. The judgment rule is: if there are one or more connected regions whose pixel similarity is greater than 0.8 and the number of pixels in the region is not less than 64, then a highly correlated region is considered to exist. The threshold of 0.8 is based on statistical analysis of the consistency of real target region features in the YOLO training set. Within most target regions, the average similarity exceeds 0.8; and 64 pixels is the minimum effective region area limit to ensure that misidentification caused by local fluctuations is excluded.

[0076] Once the system confirms the existence of a relevant region, it enters the weight adjustment process of the attention mechanism. The system first obtains the multi-head attention structure output corresponding to the feature map at each scale in the YOLO network. Each scale contains eight attention heads, and each attention head outputs an attention map of the same size as the feature map at that scale. The system performs binarization on the attention map of each attention head, setting a threshold of 0.5. All pixel regions with a value greater than this are considered high-response regions of that attention head. The system calculates the intersection-union ratio (IoU) of this high-response region and the identified high-relevance region (i.e., the intersection area divided by the union area). If this value is greater than 0.6, the weight of that attention head is multiplied by 1.2 to form the updated attention map; attention heads that do not meet this condition are not adjusted. All eight adjusted attention maps are then recombined to form the first attention weight data. Here, 0.6 is the overlap threshold, derived from the average overlap ratio between the real target region and the attention head hotspot in 500 image samples, and 1.2 is the enhancement coefficient, which effectively improves the model's attention intensity to that region during parameter tuning tests.

[0077] The first attention weight data, obtained from the multi-scale feature maps, is input into the encoding structure of the Transformer model. This attention weight data consists of multiple attention heads, each outputting a two-dimensional attention heatmap. These heatmaps maintain the same spatial dimension as the original feature maps. For example, if the small-scale feature map is 40x40, and there are 8 attention heads per scale, a three-dimensional weight tensor of 8x40x40 is formed. The system unfolds this three-dimensional tensor, flattening the two-dimensional map of each attention head into a one-dimensional vector of length 1600, and combines them in order of attention heads into an input matrix of dimension 1600x8, which serves as the input to the Transformer encoder. The Transformer encoding structure contains six stacked encoder modules, each consisting of two sub-layers: the first sub-layer is a multi-head self-attention module, and the second sub-layer is a feedforward neural network module. For each multi-head self-attention sub-layer, the system first performs a linear transformation on the input matrix to generate query, key, and value vectors, respectively. After transformation, the system calculates the dot product between the query and the key. The result, after scaling, is fed into the Softmax function to obtain the attention distribution coefficients corresponding to each position. These coefficients are then multiplied by the value vector in a weighted manner to form the weighted output of the head at that position. The outputs of all attention heads are concatenated together and linearly mapped to form the output of this sublayer. The output is then processed through residual connections and layer normalization to ensure stable feature propagation. Subsequently, the system enters the feedforward network sublayer. This sublayer consists of two linear layers. The first layer expands the input dimension from the original 8 to a hidden layer dimension of 256, using the GELU activation function to introduce non-linear transformation capabilities. The second layer compresses the dimension back to the original input dimension of 8. This process also connects the residual structure and layer normalization to maintain the convergence stability of the deep network. After each encoder layer, the system performs a global modeling and context integration of the attention information. All six encoder layers are executed sequentially, progressively strengthening information associations and gradually forming a sparser but focused attention response. After the entire encoder structure is processed, the output is still a 1600x8 matrix. The system reconstructs it back into eight 40x40 two-dimensional matrices using the original attention head method, forming the final first optimized attention matrix. Compared to the original attention data, this optimized matrix is ​​more focused on the target region, and attention to non-target regions is effectively suppressed, thereby enhancing the model's spatial attention capability to the real target.

[0078] The first optimized attention matrix is ​​used to traverse and calculate all anchor box positions in the small-scale feature layer. The size of the small-scale feature layer is generally 40 by 40, with 3 default anchor boxes corresponding to each pixel position, for a total of 4800 anchor boxes. The position coordinates of each anchor box correspond one-to-one with a specific pixel in the optimized attention weight matrix. The system extracts the attention value corresponding to each anchor box position from this matrix as a reference for the recognition priority of that anchor box in the current image.

[0079] Next, the system sorts all anchor boxes in descending order of their attention values, forming the first sorting result data. Anchor boxes ranked higher are more likely to be within the target area. After sorting, the system dynamically adjusts the size of each top-ranked anchor box based on the target size information obtained in S1. This statistical information is obtained by performing cluster analysis on all target bounding box dimensions extracted by the YOLO detector in stage S1, resulting in nine anchor box size groups, each representing the aggregation of common target widths and heights.

[0080] Subsequently, the system performs enhancement operations on the top 30% of the anchor frames, specifically increasing their width by 10% and height by 5% to better cover large or prominent targets. The system then performs reduction operations on the bottom 30% of the anchor frames, decreasing their width by 10% and height by 5% to accommodate potentially small target areas. The middle 40% of the anchor frames remain unchanged. The set of anchor frames generated by these operations constitutes the first set of adjusted anchor frame data.

[0081] The adjusted anchor box set undergoes adaptability verification to determine if it meets the target size matching threshold. This verification is performed by comparing all adjusted anchor box sizes with the labeled or inferred true target bounding box sizes in the image and calculating the size overlap rate. Specifically, if more than 70% of the true targets have widths and heights within ±15% of the adjusted anchor box size, the current anchor box adjustment is considered a high match to the target size and meets the usage requirements. If this condition is met, the system considers the current anchor box adjustment optimization effective and uses the optimized attention matrix to update the weight distribution of the corresponding small-scale feature layer in the YOLO model, outputting a second attention weight distribution. This distribution will guide subsequent feature fusion, convolution extraction, and target recognition. If the threshold condition is not met, the optimization result is not adopted, and the system retains the original anchor box parameters and attention distribution for re-analysis of the next input image.

[0082] Multi-scale feature map downsampling ratios: 1 / 4, 1 / 8, 1 / 16, 1 / 32. Basis for setting these ratios: Based on the YOLO architecture and the standard FPN network structure, experiments verify that these four scales cover the complete detection requirements from small to large targets. Each level reduces the spatial resolution by half, effectively extracting contextual information at different scales. High-relevance region determination thresholds: similarity greater than 0.8, region pixel count not less than 64. Similarity 0.8: Based on the cosine similarity statistics within the real target region in the first feature embedding data, over 80% of target regions have an average similarity higher than 0.8. 64 pixels: Corresponding to an 8×8 minimum receptive region, ensuring that relevant regions have a identifiable spatial range and excluding edge noise regions. Attention weight enhancement threshold: Cross-Union Ratio (CIRR) greater than 0.6, enhancement coefficient 1.2. CIRR 0.6: In analyzing the overlap between attention hotspots and real target regions, 0.6 is the minimum threshold for clear target focusing. Enhancement coefficient 1.2: The effects of enhancements from 1.1 to 1.5 were compared in the experiment; it was found that 1.2 resulted in the most significant improvement in detection accuracy, with little change in false positive rate. Target size matching threshold: ±15%, matching ratio not less than 70%. ±15%: In the analysis of target size variation range in the test set, it was found that the size deviation between more than 90% of the real target bounding boxes and the best-matching anchor boxes was less than 15%. 70% matching rate: This is the critical distribution point of model accuracy before performance degradation, ensuring that most targets are covered by the optimized anchor boxes.

[0083] S5 includes obtaining multi-scale feature maps from the second attention weight distribution, fusing features through layer-by-layer convolution operations, and using position embedding encoding sequence input to obtain first fused feature data; if the spatial consistency of the first fused feature data is lower than a preset threshold, then generating second fused feature data by adjusting the convolution kernel weights; calculating the positional offset of the sequence input based on the second fused feature data to obtain first positional offset data; optimizing the first positional offset data through an attention mechanism to generate a third attention weight distribution; performing a weighted summation operation on the multi-scale feature maps based on the third attention weight distribution to obtain third fused feature data; if the sequence consistency of the third fused feature data is lower than a preset threshold, then adjusting the sequence input weights through a Transformer model to generate optimized sequence data; updating the third attention weight distribution based on the optimized sequence data to determine the final feature weight distribution.

[0084] In one possible implementation, the fusion features are first obtained from the second attention weight distribution and multi-scale feature maps generated in the previous stage. Specifically, for each scale feature map (assuming three scales: 1 / 4, 1 / 8, and 1 / 16), the attention weight map for that scale is extracted. Each attention weight map is spatially aligned with its corresponding scale feature map. If the spatial size of the attention map is larger than that of the feature map, it is downsampled; if smaller, it is interpolated and expanded to ensure consistent spatial resolution. After alignment, the channel values ​​at corresponding positions in the attention map and feature map are multiplied element-wise according to scale to obtain the weighted feature maps for each scale. These weighted feature maps are concatenated along the scale and channel dimensions. First, the concatenation of the 1 / 4 and 1 / 8 scales is input into a two-dimensional convolutional layer. This convolutional layer uses a 3×3 kernel with a stride of 1 and zero padding at the edges to maintain the output spatial size. The number of output channels is set to 64. After convolution, batch normalization is performed, and then a ReLU activation function is applied. Next, this output is concatenated with the 1 / 16 scale. Scale-weighted feature maps are concatenated and then fed into another convolutional layer with the same parameters (3×3 kernel, stride 1, zero padding, 64 channels, batch normalization, ReLU activation) to integrate all scale feature maps. The output of the convolution is the fused feature map. The system divides the fused feature map into spatial blocks of 8×8 pixels each, and performs average pooling on all channels within each spatial block to obtain the channel average feature vector of that block. At the same time, a position embedding vector is generated for each spatial block. This position embedding vector is mapped from the row and column number of the block in the fused feature map to a fixed-dimensional (e.g., 64-dimensional) vector. The pooled feature vector is added to its position embedding vector, and this process is repeated for all spatial blocks to obtain a sequence, which is the first fused feature data.

[0085] The spatial consistency of the first fused feature data is then determined. Specifically, all pairs of blocks that are directly adjacent vertically or horizontally in the fused image are selected from the above sequence. For each pair of blocks, its embedding vector is taken, and the cosine similarity between these two embedding vectors is calculated. All these similarity values ​​are averaged to obtain the spatial consistency value. The system sets the spatial consistency threshold to 0.7 because, in 500 image tests, when the average similarity is below 0.7, the false positive and false negative rates of object detection increase by at least 10% compared to when the spatial consistency is above 0.7. If the spatial consistency is less than 0.7, the system considers the spatial structure in the fused features to be insufficiently continuous and clear, requiring adjustment of the convolutional kernel weights. When the spatial consistency is less than 0.7, the system performs convolutional kernel weight adjustment to generate the second fused feature data. Specifically, the process involves locating spatial blocks whose cosine similarity values ​​with their adjacent blocks are in the lowest 30% among all blocks (i.e., the bottom third of the blocks in the sorting). The center weight of the convolutional kernel of the corresponding convolutional layer for these blocks is multiplied by 1.1, and the peripheral weights are multiplied by 0.9. The center weight refers to the kernel coefficient used to map the convolutional kernel to the center position of the spatial block; the peripheral weight refers to the coefficients at other positions in the kernel. The adjusted convolutional layer then uses these new weights to re-perform the aforementioned two-stage convolutional fusion process on the spliced ​​feature map to obtain the second fused feature data.

[0086] Next, the positional offset of the sequence input is calculated based on the second fused feature data. Specifically, the process is as follows: For each spatial block, the original center pixel coordinates are recorded. Then, the channel with the highest response intensity and its corresponding position in the second fused feature map are found. The magnitudes of the horizontal and vertical distances between this position and the original center coordinates are calculated as offset values. All blocks are traversed to obtain a set of offset values. These offset values ​​are normalized to the range of 0 to 1. In this normalization process, the maximum offset value is used to obtain the maximum actual offset in the sample, and the minimum offset value is usually 0. These normalized offset values ​​constitute the first positional offset data.

[0087] Subsequently, the first position offset data is optimized through an attention mechanism to generate a third attention weight distribution. Specifically: the position offset value of each spatial block is concatenated with the attention value of the block at the corresponding position in the second attention weight distribution and the average channel response of the block in the fusion feature embedding vector to form a concatenated feature; all block concatenated features are input into an attention submodule with 4 heads, each head processing the same concatenated feature; in each head, the position offset part is multiplied by a weight coefficient of 1.2, the attention part is multiplied by a weight coefficient of 1.0, and the fusion feature response part is multiplied by 1.0; the unnormalized weight values ​​obtained by the attention scoring mechanism of the weighted concatenated features are normalized to an inter-block weight distribution through a Softmax operation; the normalized weight maps of all attention heads are merged by the average number of heads to form the third attention weight distribution.

[0088] Subsequently, a weighted summation is performed on the multi-scale feature maps according to the third attention weight distribution to obtain the third fused feature data. Specifically, for each scale feature map, the corresponding position is multiplied element-wise with the third attention map (which is downsampled if it is high resolution or interpolated to align if it is low resolution). Then, the images after multiplying all scales are aligned along the channel dimension and summed by channel, and then summed by spatial position. Finally, a fused feature map is obtained, which is the third fused feature data.

[0089] Next, the system determines whether the sequence consistency of the third fused feature data is below a preset threshold. Specifically, it selects three consecutive image frames, generates their corresponding third fused feature maps, and determines the channel number with the highest response at each spatial block location for each frame. For each block, it compares whether the channel numbers in the three frames are completely identical. It then calculates the consistency ratio by dividing the number of blocks with identical channel numbers by the total number of blocks. The system sets the sequence consistency threshold to 60% because, in 300 sets of three consecutive image tests, when the consistency ratio is below 60%, target position drift or detection instability is significant. If the consistency ratio is below 60%, the sequence consistency is considered low. When the sequence consistency is below 60%, the system adjusts the sequence input weights using a Transformer model to generate optimized sequence data. Specifically, the fused feature embedding vector sequence of the three frames is concatenated with the third attention weight distribution and position offset data of the corresponding spatial blocks as input. This input is fed into a Transformer model with a 4-layer encoder, each layer containing 4 attention heads, a hidden layer width of 128, a ReLU activation function, and LayerNorm normalization. The model outputs a weighted adjustment coefficient for each spatial block, which is usually close to 1 and ranges from about 0.8 to 1.2, and is used to adjust the weight of each spatial block in the attention distribution.

[0090] Finally, the third attention weight distribution is updated based on the optimized sequence data. Specifically, for each spatial block, the third attention weight is multiplied by the corresponding weighting adjustment coefficient, and then min-max normalization is performed on all block weight values ​​to make all weights fall back into the range of 0 to 1. This normalization process involves finding the minimum and maximum adjusted weight values ​​in all blocks and then scaling them linearly. The final output distribution is used as the final feature weight distribution for subsequent fusion and detection.

[0091] Spatial consistency threshold (set to 0.7): Based on testing 500 sets of typical complex scene image sequences, the impact of different spatial consistency indices on target detection accuracy was compared. When spatial consistency is below 0.7, the system's average target detection accuracy (mAP) drops by more than 10%. Therefore, to ensure the stability and spatial continuity of feature fusion, this value is set as a fixed threshold. Sequence consistency threshold (set to 60%): Based on statistical analysis of 300 sets of three consecutive image sequences, "channel consistency" is defined as the maximum response channel number of each location block in the three frames remains consistent. The consistency ratio in all blocks is calculated; when this ratio is below 60%, target detection results exhibit drift or decreased stability more frequently. Therefore, 60% is determined as the benchmark for sequence consistency. Convolution kernel center weight adjustment coefficient (set to 1.1) and peripheral weight adjustment coefficient (set to 0.9): Based on the fact that in spatial domain fusion convolution operations, to enhance the discriminative power of feature center responses and improve the target feature expression ability, the center position coefficient is fine-tuned during the convolution kernel update stage. Experiments showed that multiplying the center coefficient by 1.1 and the periphery coefficient by 0.9 effectively improved the significance of the fused features across multiple target scales and complex backgrounds without introducing additional overfitting risk. The weighting coefficients for the attention shift component (set to 1.2) and the fused feature component (set to 1.0) were determined based on the following: in the attention mechanism, after concatenating the input features, a weighted combination process is performed. To improve sensitivity to positional drift, the weight of the shift feature was set to a slightly higher 1.2, while the fused feature maintained its standard weight of 1.0. This ratio demonstrated a higher level of attention concentration in 100 rounds of validation experiments.

[0092] S6 includes adjusting the convolutional kernel parameters of the YOLO detector through a third attention weight distribution to generate first parameter adjustment data; if the detection accuracy of the first parameter adjustment data is lower than a preset threshold, then optimizing the non-maximum suppression process through residual connections to obtain first optimized suppression data; calculating the bounding box weights for target recognition based on the first optimized suppression data to generate first boundary weight data; performing sequence processing on the first boundary weight data through an attention mechanism to obtain a fourth attention weight distribution; if the sequence consistency of the fourth attention weight distribution is lower than a preset threshold, then adjusting the feature extraction weights through a convolutional neural network to generate second feature extraction data; updating the dynamic optimization parameters of the YOLO detector based on the second feature extraction data to obtain second parameter adjustment data; and performing a weighted summation of the bounding boxes for target recognition using the second parameter adjustment data to determine the final target recognition data.

[0093] First, the third attention weight distribution output from the previous stage is obtained. This weight distribution is a two-dimensional matrix with the same size as the input image, where the weight value at each pixel position represents the importance of that region in the object detection task, with a value range of 0 to 1. The system sets a weight threshold, fixed at 0.8, to filter out high-response regions. Pixel regions with attention values ​​greater than or equal to 0.8 are considered by the system to be significantly related to the target. Next, the system links these high-weight regions with the convolutional layers of the backbone network in the YOLO detector. Specifically, the system selects three key convolutional layers in the YOLO backbone network: the first feature extraction convolutional layer, the second intermediate feature fusion layer, and the third target prediction layer. In these three convolutional layers, the system performs a position-based weighting operation on the convolutional kernels. That is, for each convolutional kernel, if the image region corresponding to its receptive field center contains an attention value higher than 0.8, all weight parameters of the convolutional kernel are multiplied by a coefficient of 1.2 to improve its feature extraction capability in the target region; otherwise, the original weights remain unchanged.

[0094] After completing the weighting operations described above, the system updates the YOLO model parameters and performs a complete inference process on the validation image set, outputting information including the class prediction, location coordinates, and confidence score for each candidate bounding box. The system statistically analyzes all prediction results and calculates the average accuracy (mAP) as an evaluation metric for detection precision. If the mAP value is lower than the preset threshold of 0.75, it indicates that the first-stage adjustment of the convolutional kernel parameters did not significantly improve detection performance. At this point, the system proceeds to the next step, which involves optimizing the non-maximum suppression (NMS) process using a residual connection mechanism. In conventional NMS, when deduplicating candidate boxes, if the IoU (Intersection over Union) of two boxes exceeds 0.5, the bounding box with the higher score is retained, and the overlapping box with the lower score is deleted. To avoid the loss of target information due to the accidental deletion of bounding boxes, the system establishes residual connections between the retained boxes and the suppressed boxes. The specific processing is as follows: If the IoU between a suppressed bounding box and a retained bounding box exceeds 0.5, and the predicted categories are the same, the system performs a weighted average of the suppressed bounding box's position coordinates and dimensions with the retained bounding box information, using a weight of 0.5. This updated value is then used to update the center point coordinates and dimensions of the original retained bounding box, and the updated box is reintroduced into the detection result set. The complete set of bounding boxes obtained through this optimization process constitutes the first optimized suppression data.

[0095] First, the system receives the first optimized suppression data output from the previous process, which contains multiple candidate bounding boxes. Each bounding box includes the following information: predicted class label, confidence score, location coordinates, width, and height, as well as the residual fusion information generated after optimization. To further enhance the selection of effective targets in the detection results, the system calculates the comprehensive weight for each bounding box, generating the first bounding box weight data. The weight calculation is divided into three dimensions: confidence factor, bounding box area factor, and attention matching factor: Confidence factor: directly taken from the target confidence score output by the YOLO model, ranging from 0 to 1. Bounding box area factor: the system calculates the area of ​​each bounding box and sets the factor according to the following intervals: for bounding boxes with an area between 100 and 10000 pixels square, the area factor is set to 1.0; for bounding boxes with an area less than 100 pixels square, the area factor is set to 0.8; for bounding boxes with an area greater than 10000 pixels square, the area factor is set to 0.6. This setting is based on actual image statistics to avoid interference from noisy small targets or abnormally large targets on the model results. Attention matching factor: the system calculates the average attention value of the bounding box in the corresponding region of the third attention weight distribution map. If the average value is higher than 0.6, the matching factor is set to 1.0; otherwise, it is set to 0.7, used to measure whether the bounding box is aligned with the attention hotspot region. The three factors are multiplied one by one to obtain the comprehensive boundary weight for each bounding box. The set of weights for all bounding boxes constitutes the first boundary weight data. Next, the system performs time-series modeling processing on the first boundary weight data. Specifically, based on the high-confidence bounding boxes identified in the current frame and the two frames preceding it, a sequence of bounding boxes arranged in chronological order is constructed. Each bounding box records its center point position, boundary size, boundary weight, predicted category, and other attributes in the image, forming a bounding box time series. This sequence is input into the multi-head attention mechanism module. The module includes four attention heads, which respectively process four types of information: changes in the center point position, size, category, and boundary weight. Each head independently calculates the attention weight and weights the information dimension it is responsible for. The outputs of all attention heads are concatenated and then fed into a fully connected layer for fusion processing, followed by normalization, outputting a new weight distribution data, which is the fourth attention weight distribution. In this distribution, a higher weight value indicates that the bounding box maintains a high degree of consistency and importance in the time series, which is a key basis for target tracking and judgment.

[0096] The stability of bounding boxes in the fourth attention weight distribution over time is analyzed. Specifically, for the top 10 bounding boxes in the current frame and the two frames preceding it, their center coordinate changes are compared across different frames. If the center coordinate shift of a bounding box is greater than 5 pixels between consecutive frames, it is counted as one shift. The ratio of the number of shifts to the total number of comparisons is calculated as the consistency ratio. If this consistency ratio is below 70%, the current object detection result is considered to lack sufficient stability over time, and the system will perform feature extraction optimization. At this point, the system calls the convolutional neural network module, with the input data being the image fusion data formed by superimposing the original image of the current frame and the fourth attention weight map. This convolutional neural network contains three layers, each with a 3x3 kernel size, a stride of 1, and 64, 128, and 256 channels respectively. After the convolution operation, each layer undergoes ReLU activation and batch normalization. The network output is a new set of feature maps, where boundary information, texture structure, and attention hotspots are further enhanced; this output is the second feature extraction data. Next, the system sends the second feature extraction data to the parameter adjustment module in the YOLO detector, updating the following dynamic parameters to form the second parameter adjustment data: Confidence threshold: originally set to 0.5, now adjusted to 0.45 based on the signal-to-noise ratio of the feature map to increase the detection capability for low-confidence targets. Class weight coefficient: The score of small target classes with confidence levels concentrated between 0.4 and 0.6 in the current frame is increased by multiplying by a coefficient of 1.1 to compensate for their recognition disadvantage. Anchor box adjustment strategy: Based on the statistical distribution of target aspect ratios in the new feature map, the anchor box templates are reordered, retaining the top 3 sets of templates close to the main target size and deleting the 2 sets with larger deviations, thus optimizing the anchor box configuration. After completing the above parameter updates, the YOLO model runs again, outputting new bounding box prediction results. For a set of bounding boxes with an overlap greater than 0.7, the system uses a weighted summation method to generate the final target recognition data: the confidence score of each group of overlapping boxes is used as a weighting factor, and their center coordinates, width, and height are weighted and averaged. If the overlapping boxes are of the same category, they are merged into a single bounding box; if the categories are different but the confidence score difference exceeds 0.2, they are retained as two separate targets. All the fused bounding boxes output are the final target recognition results for the current image frame, possessing high confidence, spatial consistency, and category reliability, providing stable input for target tracking and semantic reasoning.

[0097] Attention weight amplification threshold (set to 0.8): Based on statistical analysis of attention maps in the training and validation sets, it was found that regions with a value higher than 0.8 overlapped with the true target locations in over 85% of cases. Therefore, 0.8 was set as the lower limit for attention region enhancement, ensuring that parameter amplification is applied only to the most representative key regions, avoiding overgeneralization and false detections. Convolution kernel parameter amplification coefficient (set to 1.2): In controlled experiments, the coefficient was gradually increased from 1.0 to 1.5 to evaluate its marginal contribution to mAP improvement. It was ultimately determined that 1.2 resulted in the most stable performance gain without causing fluctuations in detection results. 1.2 improved the small target detection rate by approximately 6% in various test environments. mAP accuracy threshold (set to 0.75): Based on the minimum requirements for target detection accuracy in industrial applications and the average level of existing models on the COCO and VOC validation sets, the mAP threshold was set to 0.75. This means that the accuracy of the adjusted model should not be lower than 75% of the initial model, ensuring necessary accuracy before residual optimization. The IoU overlap threshold in non-maximum suppression (NMS) is set to 0.5. The rationale is that boxes with an IoU exceeding 0.5 typically represent multiple redundant detections of the same target. Based on the original YOLO literature and standard implementations, 0.5 is set as a reasonable lower limit for boundary overlap, used to remove duplicate boxes. The residual fusion weight is set to 0.5. This is because when NMS-suppressed bounding boxes participate in fusion, their reliability is relatively low compared to the main bounding box. Using 0.5 as a neutral coefficient can control error propagation while preserving structural information, avoiding misleading information. The attention overlap threshold is set to 0.6. Analysis of the overlap between real target bounding boxes and attention hotspots shows that when the overlap is greater than 60%, most targets can be effectively identified; below this value, detection errors are more likely. Therefore, it is used as the baseline for effective target region matching. The sequence consistency judgment frame number (3 frames) and position offset threshold (5 pixels) are set based on the fact that selecting 3 frames as the time window allows observation of motion trends without increasing computational burden. A center offset of more than 5 pixels between consecutive frames is considered a decrease in stability. This value is determined based on the target's average moving speed and image resolution evaluation results. The consistency ratio threshold (set to 70%) is based on the fact that in real video scenes, more than 70% of targets can maintain a stable position within 3 frames. When it falls below this ratio, it indicates that the target is significantly disturbed or the model extraction is unstable, thus requiring further optimization of the feature extraction process. The confidence threshold adjustment range (from 0.5 to 0.45) is based on the fact that the detector's original threshold is 0.5. Experiments show that a slight reduction can improve the recognition rate of low-confidence targets, while introducing a small increase in the false detection rate; therefore, a reduction of 0.05 is chosen as the minimum effective adjustment amount.The small target category score improvement coefficient (set to 1.1) is based on the fact that the recall rate of small target categories in the sample set is usually low. Adjusting the category weights increases their influence on the total score, and 1.1 has been verified as the most balanced initial optimization coefficient. The anchor frame aspect ratio dynamic adjustment mechanism is based on the following: the system aggregates and statistically analyzes all target dimensions according to the first set of environmental features extracted in S1. For example, if the aspect ratio of most targets is concentrated between 1.6 and 1.9, the system prioritizes retaining anchor frames with aspect ratios less than 0.3 from this range, and deletes or downweights those with a difference exceeding 0.5, to ensure a high degree of match between the anchor frame template and scene characteristics.

[0098] S7 includes generating initial target detection results through a fourth attention weight distribution; processing first boundary adjustment data using a feedforward network layer of a Transformer model to verify the response to environmental changes and obtain first stability index data; if the first stability index data meets a preset threshold, locking runtime control parameters and generating first optimized control data; for scenarios where the illumination intensity is lower than a preset threshold, extracting bounding box features from the first optimized control data and refining coordinate calculations using a convolutional neural network to obtain second boundary adjustment data; updating the attention allocation of the Transformer model using the second boundary adjustment data to generate a second attention weight distribution; if the sequence consistency of the second attention weight distribution is lower than a preset threshold, adjusting the feature extraction weights using a convolutional neural network to generate first feature extraction data; and optimizing the weighted calculation of the bounding box for target detection based on the first feature extraction data to obtain the final target detection data.

[0099] First, the fourth attention weight distribution and the candidate bounding box list output by the YOLO detector are obtained to generate the initial object detection result. Specifically, for the current frame image, all candidate bounding boxes are extracted from the YOLO detector, each containing a class prediction, location coordinates, width, height, and confidence score. The system matches these candidate boxes with the attention values ​​of their corresponding spatial locations in the fourth attention weight distribution, that is, maps the position of the center point of each candidate box on the attention map to the corresponding attention pixel and reads the attention value. If the attention value is greater than or equal to a set threshold of 0.8, the candidate box is marked as an "attention box". The initial object detection result is the set obtained by merging the boxes with the highest confidence among all the marked "attention boxes" (e.g., the top 10 sorted by confidence) with the unmarked boxes with extremely high confidence (e.g., confidence greater than 0.9).

[0100] The confidence score is determined based on the bounding box prediction mechanism of the YOLO detector. This score measures the confidence level that a specific class of object exists in each candidate bounding box. Specifically, during the detection process, the YOLO model outputs two key parameters for each candidate box: an objectivity score, representing the probability that any object exists in the box; and a classification probability, representing the probability that the object in the box belongs to a specific class. The confidence score is the product of these two values, ranging from 0 to 1. The objectivity score is output through a sigmoid function and optimized by the location regression branch in the network during the training phase using a binary cross-entropy loss function; the classification probability is output by the classification branch and trained using a multi-class cross-entropy loss function matched with the ground truth labels. During the inference phase, the system multiplies the objectivity score of each candidate box by its highest class probability to obtain the final confidence score. To ensure detection accuracy, the system sets the confidence score screening threshold to 0.7. This value is obtained based on statistical analysis of the training and validation sets. While ensuring a recall rate of no less than 90%, it can effectively reduce the false positive rate to below 10%, thereby improving the reliability and accuracy of the detection results.

[0101] Next, the feedforward network layer of the Transformer model is used to process the "first boundary adjustment data" and the historical environmental change parameter sequence to verify the response to environmental changes and obtain the first stability index data. The specific steps are as follows: The system summarizes the position coordinate change rate (i.e., the ratio of horizontal and vertical offset of the center point between frames and the ratio of size change) and category change frequency (i.e., the proportion of times the category prediction is different from the current category in these frames) of all "boxes of interest" in the "first boundary adjustment data" of the current frame and several past frames (e.g., 5 frames) to form a feature vector; at the same time, the historical environmental parameter sequence such as ambient light intensity, target density, and light change rate is taken as part of the input; this merged feature is input into the feedforward network layer of the Transformer model. The structure of this layer is two linear transformation layers with ReLU activation in the middle and 256 hidden nodes; the output is a scalar stability index data, ranging from 0 to 1, which characterizes the stability of the response of the recent frame detection results to environmental disturbances.

[0102] A stability threshold of 0.85 is set. This value is based on statistics from multiple scenarios with different lighting and dynamic backgrounds (a total of 300 frame sequences). In these scenarios, when the index is greater than or equal to 0.85, the average inter-frame drift deviation of the target detection output in terms of category, position, and size is lower than the set tolerance (position deviation less than 5 pixels, size change ratio less than 10%), and the false detection and false negative frequency is less than 5%. If the first stability index data is greater than or equal to 0.85, the system locks the adjustment parameters of the current frame (including attention weight allocation scheme, anchor box aspect ratio, and confidence threshold setting) as runtime adjustment parameters and generates "first optimized adjustment data".

[0103] When the ambient light intensity is below a set threshold of 80 (the average grayscale value of the sampled brightness in the image is less than 80, and the brightness range is 0–255), the system considers it to be in a low-light scene. Specifically, the system extracts all bounding box features from the first optimized adjustment data, including the center coordinates, width, height, and confidence score of each bounding box. The system takes an image patch (e.g., ±16 pixels from the center of the bounding box) from the original image and inputs this patch into a small convolutional neural network. This network consists of two convolutional layers, each with a 3×3 kernel, a stride of 1, and zero edge padding. The first layer outputs 64 channels, and the second layer outputs 32 channels. After each convolution, batch normalization and ReLU activation are performed. The network outputs predicted offsets dx and dy (center coordinate adjustment) and width and height fine-tuning factors. These adjustments are applied to the original bounding boxes to generate "second boundary adjustment data."

[0104] Subsequently, the system updates the attention allocation strategy in the Transformer model based on the coordinate and size changes in the second boundary adjustment data. Specifically, for each bounding box in the second boundary adjustment data, its coordinate adjustment amount and size adjustment ratio are mapped to an attention compensation factor. For example, if the coordinate offset is greater than 3 pixels or the width / height change ratio is greater than 10%, the attention weight of the corresponding spatial position of the box is multiplied by 1.1 in the attention map; otherwise, the original weight is maintained. This method generates a new second attention weight distribution, which is different from the original fourth attention weight distribution.

[0105] Next, the system determines whether the sequence consistency of the new second attention weight distribution is below a set threshold. Specifically, it selects the top 10 bounding boxes from the attention weight distributions in the current frame and the two previous frames, and compares the spatial offset distance of the center points of these bounding boxes across the three frames. If any box's center offset exceeds 5 pixels across the three frames, or the overall overlap (IoU) is below 70%, then the consistency is considered insufficient. This sequence consistency threshold is set to 70%, based on statistics from previous evaluation sets showing that when sequence consistency is below 70%, the detection box output fluctuates significantly, and localization is unstable. If the consistency is below 70%, the system calls a convolutional neural network to adjust the feature extraction weights. Specifically, the system inputs the original image of the current frame and the concatenated second attention weight map channels into a two-layer CNN network. Each layer has a 3×3 kernel, a stride of 1, and outputs 64 and 128 channels respectively. After each convolution, batch normalization and ReLU activation are performed. The network outputs adjusted feature maps, which are used to replace the original feature extraction part of the YOLO detector, becoming the "first feature extraction data."

[0106] Finally, the system extracts data based on the first feature and enters the weighted bounding box fusion module. Specifically, for all candidate bounding boxes, their coordinates and dimensions are combined with factors such as confidence, attention weight, and class score to calculate the fusion weight of each box; for multiple boxes with an overlap (IoU) greater than 0.7, if the categories are the same, their coordinate center and width and height are calculated by weighted summation, with the weight ratio based on the fusion weight of each box; boxes with different categories are retained separately; finally, a set of non-overlapping bounding boxes is generated, which is the final object detection data output.

[0107] Stability threshold 0.85: Statistical analysis of class prediction and bounding box position / size drift and false positive rate from a 300-frame sequence. When the index ≥ 0.85, the average change rate of box position and size is less than 5 pixels and 10%, respectively, with a false positive / false negative rate of less than 5%. Attention matching threshold 0.8: In the statistical analysis of the overlap between the attention map and the bounding box center, when the region with an attention value ≥ 0.8 has an average overlap rate with the real target exceeding 85%, it indicates that this threshold can be used to determine the box of interest. Illumination intensity threshold 80 (grayscale value): Calculating the average light intensity by sampling several brightness values ​​from the image. When the average is below 80, the image is significantly dark, and coordinate refinement is necessary; this value is based on statistics from multiple nighttime and low-light images. Low-light correction CNN network structure: Two layers of 3×3 convolutional kernels, 64→32 channels. In experiments, this is the minimum network size in low-light scenes that can significantly improve the box localization accuracy. Coordinate offset threshold of 3 pixels and width / height change ratio of 10%: These are the trigger conditions for bounding box fine-tuning, determined based on the median of the error distribution in the samples and the statistical effect of the fourth attention weight adjustment. Sequence consistency frame count of 3 frames, position offset threshold of 5 pixels, and overlap of 70%: These parameters are used as consistency evaluation criteria because they represent the limits of sharp decline in target stability and accuracy observed in continuous frame testing. Number of channels in the feature extraction CNN: 64 and 128: These are choices that balance model capacity and computational resource consumption, effectively improving feature representation capabilities without being too resource-intensive in experiments. Overlap IoU threshold of 0.7 is used for the merging condition of bounding boxes: determined based on commonly used standards in detection tasks and the statistical relationship between IoU and false detection rate between fused candidate boxes in the samples.

[0108] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for visual large model-based complex scene target detection and semantic reasoning, characterized in that, include: S1. Acquire the current scene image data, and extract the preliminary bounding box coordinates and category confidence scores based on image grid division using the YOLO detector to obtain the first environmental feature set; S2. Calculate the scene complexity score based on the first set of environmental features. If the score is higher than a preset threshold, mark it as a high-complexity environment and determine the second set of environmental features. S3. Using the second set of environmental features, the Transformer model is used to generate an adaptive weight initial matrix for the attention mechanism. If a high-complexity environment exists, the multi-head attention calculation value of the key target recognition area is amplified to obtain the first attention weight distribution. The intermediate feature map set output by the YOLO detector at at least two different resolutions is a multi-scale feature layer, and the feature layer with a pixel step size less than a preset threshold is a small-scale feature layer. S4. Obtain the first attention weight distribution, perform a weighted summation operation on the multi-scale feature layer, increase the update priority of the anchor box size set corresponding to the small-scale feature layer, and dynamically adjust the width and height of the anchor box according to the target size statistical results to obtain the second attention weight distribution. S5. Based on the second attention weight distribution, fuse multi-scale feature maps, integrate feature fusion through layer-by-layer convolution operations, and determine the third attention weight distribution; S6. Update the internal parameters of the YOLO detector through the third attention weight distribution. If the detection accuracy index is lower than the real-time requirement, iteratively adjust the weight allocation to obtain the fourth attention weight distribution. S7. Based on the fourth attention weight distribution, output the final detection result. Verify the stability index quantification under the simulation of environmental change response through the feedforward network layer of the Transformer model. If the real-time requirements are met, lock the runtime adjustment parameters to obtain the optimized target detection output. S1 includes: Acquire scene image data and generate first image data through preprocessing; If the resolution of the first image data is lower than a preset threshold, the resolution is adjusted using bilinear interpolation to obtain the second image data. The second image data is divided into grids using a YOLO detector, and the bounding box coordinates and class confidence scores are extracted to generate the first feature set. Based on the bounding box coordinates in the first feature set, the object density is calculated to obtain the first density data; Pixel brightness values ​​are extracted from the second image data, and light intensity is calculated to obtain the first light data; The K-means clustering algorithm is used to perform cluster analysis on the first density data and the first light data to generate the first environmental feature set; The data in the first environmental feature set are processed by a weighted summation method to calculate the scene complexity score and obtain the first complexity data. S3 includes: Spatiotemporal dynamic features are extracted from the second set of environmental features, and first feature embedding data is generated using a convolutional neural network. Based on the first feature embedding data, the spatial correlation of the target region is calculated to obtain the first correlation distribution data; If the target region exists in the first correlation distribution data, the multi-head attention weights are adjusted through the attention mechanism to generate the second attention weight data; The Transformer model is used to optimize the second attention weight data to obtain the first optimized attention matrix; By using the first optimized attention matrix, key target enhancement features are extracted from the environmental feature set to generate the first enhanced feature data; If the first enhanced feature data meets the preset complexity threshold, then update the environment complexity label and generate the second complexity label data; Based on the second complexity label data, the weight allocation of the target recognition region is adjusted to obtain the first target recognition optimization data; S4 includes: To obtain multi-scale features, a multi-scale feature layer is extracted from the input image, and a convolutional neural network is used to generate the first feature embedding data. Based on the first feature embedding data, the spatial correlation of the multi-scale feature layer is calculated to obtain the first correlation distribution data; If a relevant region exists in the first correlation distribution data, the multi-head attention weights are adjusted through the attention mechanism to generate the first attention weight data; The Transformer model is used to optimize the first attention weight data to obtain the first optimized attention matrix; Based on the first optimized attention matrix, the anchor box size set corresponding to the small-scale feature layer is prioritized and sorted to obtain the first sorting result data, wherein the anchor box size set is determined according to the target size statistics in the first environmental feature set in S1. Based on the first sorting result data, dynamically adjust the aspect ratio of the anchor frame to generate the first adjusted anchor frame data; If the first adjusted anchor frame data meets the preset target size threshold, the weight distribution is updated to obtain the second attention weight distribution.

2. The method according to claim 1, wherein: S2 includes: Principal component weights are extracted from the first dimensionality-reduced feature data, and the environmental change response coefficients are calculated to obtain the first response data. This first dimensionality-reduced feature data is extracted from the multi-scale feature map of the YOLO detector using the principal component analysis method. The first response data is grouped using cluster analysis to generate the first group data. If clustering exists in the first group of data, dynamic object data is extracted from the scene image data to generate the first dynamic feature set; The support vector machine algorithm is used to classify the first dynamic feature set to obtain the first classification data; Based on the first classification data, edge features are extracted from the background texture data to generate the first edge data; The first edge data and the first response data are integrated using a weighted fusion method to obtain the second complexity data; If the second complexity data is higher than the preset threshold, then update the first complexity label and generate the second complexity label.

3. The method according to claim 1, wherein: S5 includes: Multi-scale feature maps are obtained from the second attention weight distribution, and features are fused through layer-by-layer convolution operations. The first fused feature data is obtained by inputting a position embedding encoding sequence. If the spatial consistency of the first fused feature data is lower than a preset threshold, the second fused feature data is generated by adjusting the weights of the convolution kernel. Based on the second fusion feature data, the position offset of the sequence input is calculated to obtain the first position offset data; The first position offset data is optimized using an attention mechanism to generate a third attention weight distribution; Based on the third attention weight distribution, a weighted summation operation is performed on the multi-scale feature maps to obtain the third fused feature data. If the sequence consistency of the third fusion feature data is lower than a preset threshold, the sequence input weights are adjusted through the Transformer model to generate optimized sequence data. Based on the optimized sequence data, the third attention weight distribution is updated to determine the final feature weight distribution.

4. The method according to claim 1, wherein, S6 includes: The convolution kernel parameters of the YOLO detector are adjusted by the third attention weight distribution to generate the first parameter adjustment data; If the detection accuracy of the first parameter adjustment data is lower than the preset threshold, the non-maximum suppression process is optimized through residual connection to obtain the first optimized suppression data. Based on the first optimized suppression data, the bounding box weights for target recognition are calculated, and the first boundary weight data is generated.

5. The method according to claim 4, wherein: S6 further includes: The fourth attention weight distribution is obtained by sequentially processing the first boundary weight data through an attention mechanism. If the sequence consistency of the fourth attention weight distribution is lower than a preset threshold, the feature extraction weights are adjusted through a convolutional neural network to generate second feature extraction data. Based on the data extracted from the second feature, the dynamic optimization parameters of the YOLO detector are updated to obtain the second parameter adjustment data; By adjusting the data using the second parameter, the bounding boxes of the target recognition are weighted and summed to determine the final target recognition data.

6. The method according to claim 1, wherein, S7 includes: The initial target detection results are generated by the fourth attention weight distribution. The first boundary adjustment data is processed by the feedforward network layer of the Transformer model to verify the response to environmental changes and obtain the first stability index data. If the first stability index data meets the preset threshold, then the runtime control parameters are locked, and the first optimized control data is generated. For scenarios where the light intensity is below a preset threshold, bounding box features are extracted from the first optimized control data, and a convolutional neural network is used to refine the coordinate calculations to obtain the second boundary adjustment data.

7. The method according to claim 6, wherein: The S7 also includes: By adjusting the data through the second boundary, the attention allocation of the Transformer model is updated, generating a second attention weight distribution; If the sequence consistency of the second attention weight distribution is lower than a preset threshold, the feature extraction weights are adjusted through a convolutional neural network to generate the first feature extraction data. Based on the data extracted from the first feature, the weighted calculation of the bounding box for object detection is optimized to obtain the final object detection data.

Images