Image data real-time analysis and intelligent decision method and system empowered by large model

Abstract

The application provides a large model enabled image data real-time analysis and intelligent decision method and system, relating to the field of image processing, comprising obtaining an original image and performing preliminary super-resolution reconstruction, extracting texture and semantic features using a visual large model. Based on the semantic features, a target area is identified and a semantic level is divided, according to which a reconstruction priority is assigned and a reconstruction intensity parameter is determined in combination with spatial distribution. Further, a large model decoder is used to analyze semantic correlation to adjust the reconstruction parameter, and finally an adaptive super-resolution reconstruction is performed on the target area of the original image to generate an optimized image. The application realizes intelligent and adaptive enhancement of the key area of the image, and improves the reconstruction efficiency and effect.

Description

Technical Field

[0001] This invention relates to image processing technology, and more particularly to a method and system for real-time image data analysis and intelligent decision-making empowered by a large model. Background Technology

[0002] In the field of image data processing and analysis, real-time image enhancement and understanding, and driving subsequent decisions based on the understanding results, are core requirements in computer vision applications. Existing technologies typically employ a separate pipeline architecture to achieve this process. A common approach is to first use traditional super-resolution algorithms or lightweight neural networks to globally enhance the resolution of the original low-quality image to improve the overall visual effect. Then, the processed image is input into an independently trained object detection or image recognition model to frame and identify key objects. After obtaining the object's category and location information, the system formulates subsequent processing strategies based on pre-set simple rules (e.g., identifying object regions of a specific importance category). Finally, another dedicated image processing module may be invoked for secondary optimization of the specified region. Throughout the entire process, image enhancement, feature extraction, semantic understanding, and decision generation are performed sequentially by different, specialized models or algorithms.

[0003] However, the aforementioned conventional technical approaches have significant limitations. On the one hand, the separate model architecture leads to low processing efficiency and information loss. Initial super-resolution reconstruction often focuses on global uniform processing, failing to consider the differences in importance of different semantic regions in the image. This may result in insufficient improvement for key targets, while causing unnecessary waste of computational resources for background regions. On the other hand, the subsequent semantic analysis module performs recognition based on the initially reconstructed image, and its recognition accuracy is limited by the effect of the previous reconstruction. Furthermore, the decision-making mechanism based on simple preset rules lacks flexibility and cannot capture the deep semantic relationships and contextual information between multiple targets in complex scenes. This results in the final local adaptive reconstruction strategy being neither accurate nor intelligent, making it difficult to achieve a high degree of coordination between perception and decision-making in resource-constrained real-time scenarios. Summary of the Invention

[0004] This invention provides a method and system for real-time image data analysis and intelligent decision-making empowered by large models, which can solve the problems in the prior art.

[0005] A first aspect of this invention provides a method for real-time image data analysis and intelligent decision-making empowered by large models, comprising:

[0006] The original image to be processed is obtained, and a preliminary super-resolution reconstruction process is performed on the original image to be processed to generate a preliminary reconstructed image. The preliminary reconstructed image is input into the encoder of the large visual model, and the texture features and semantic features are output.

[0007] Target recognition based on semantic features yields a set of target regions, where each target region contains a semantic category identifier and spatial location coordinates.

[0008] Semantic hierarchy is performed on the semantic category identifiers in the target region set. Reconstruction priority is assigned to each target region according to the semantic hierarchy. Reconstruction intensity parameters are determined based on the spatial distribution characteristics of reconstruction priority and spatial location coordinates.

[0009] Texture features and semantic features are input into the decoder of the large visual model to obtain semantic correlation information between target regions. Based on the semantic correlation information, the reconstruction intensity parameters are adjusted to obtain the adjusted reconstruction parameter set.

[0010] Based on the adjusted set of reconstruction parameters and spatial coordinates, adaptive super-resolution reconstruction is performed on the corresponding target region of the original image to obtain the optimized image.

[0011] The original image to be processed undergoes preliminary super-resolution reconstruction to generate a preliminary reconstructed image. This preliminary reconstructed image is then input into the encoder of the large visual model, which outputs texture and semantic features, including:

[0012] The gradient magnitude is calculated to obtain a gradient distribution map of the original image to be processed. The kernel size is determined based on the statistical distribution of the gradient magnitude in the gradient distribution map. The upsampling factor is determined based on the degree of clustering of the gradient directions in the gradient distribution map.

[0013] A multi-scale convolutional network is constructed based on the kernel size, and convolution operations are performed on the original image to be processed to obtain a multi-scale feature map. Pixel rearrangement is performed on the multi-scale feature map according to the upsampling factor to complete the initial super-resolution reconstruction process and generate an initial reconstructed image.

[0014] The initially reconstructed image is input into the encoder of the large visual model. The encoder divides the initially reconstructed image into multiple image blocks. Frequency domain transformation is performed on multiple image blocks to extract frequency domain components, which are then input into the frequency domain coding path of the encoder to obtain frequency domain coding features. Spatial convolution is performed on multiple image blocks to extract spatial components, which are then input into the spatial coding path of the encoder to obtain spatial coding features.

[0015] Frequency domain coding features and spatial coding features are fused to obtain fused coding features. The fused coding features are then separated to obtain texture features and semantic features, which are used as the encoder output of the large visual model.

[0016] The target region set obtained by target recognition based on semantic features includes:

[0017] Multi-scale semantic features are obtained by performing multi-scale convolution on semantic features, and the spatial attention distribution of multi-scale semantic features is calculated and attention weights are generated.

[0018] Attention weights are used to weight multi-scale semantic features to generate weighted semantic features. A sliding window operation is performed on the weighted semantic features to generate candidate target regions and calculate confidence scores. Valid candidate target regions are obtained by filtering candidate target regions based on their confidence scores.

[0019] Extract regional features from valid candidate target regions and classify them to obtain semantic category identifiers, and calculate the boundary coordinates of valid candidate target regions to obtain spatial location coordinates;

[0020] Based on the spatial location coordinates, local texture features of the corresponding region are extracted from the texture features, and the matching degree between the local texture features and the semantic category identifier is calculated to obtain the texture consistency score;

[0021] A comprehensive score is obtained by weighted summation of texture consistency score and confidence score. Valid candidate target regions are selected based on the comprehensive score to obtain the final target region. The semantic category identifier and spatial location coordinates of the final target region are combined to form the target region set.

[0022] Based on spatial location coordinates, local texture features of the corresponding region are extracted from the texture features. The texture consistency score is obtained by calculating the matching degree between the local texture features and the semantic category identifier, including:

[0023] The horizontal and vertical coordinate ranges of the target region are obtained by parsing the spatial location coordinates. Based on the horizontal and vertical coordinate ranges, the local texture features of the corresponding region are cropped and extracted from the texture features.

[0024] Local texture features are averaged by channel pooling to obtain texture feature vectors. Semantic category identifiers are vector-encoded to obtain semantic vectors. The cosine similarity between texture feature vectors and semantic vectors is calculated to obtain semantic matching scores.

[0025] The frequency domain features are obtained by performing Fourier transform on the local texture features. The amplitude spectrum is extracted from the frequency domain features to obtain the frequency energy distribution. The standard frequency energy distribution is retrieved from the preset texture library according to the semantic category identifier. The mean square error between the frequency energy distribution and the standard frequency energy distribution is calculated to obtain the frequency deviation score.

[0026] The gradient magnitude of local texture features is calculated to obtain the edge intensity distribution. The edge intensity reference range is determined according to the semantic category identifier. The percentage of pixels whose edge intensity distribution falls within the edge intensity reference range is calculated to obtain the edge consistency score.

[0027] The texture consistency score is obtained by weighted summation of semantic matching score, frequency deviation score and edge consistency score.

[0028] The semantic category identifiers in the target region set are semantically hierarchically divided. A reconstruction priority is assigned to each target region based on the semantic hierarchy. The reconstruction intensity parameters are determined based on the spatial distribution characteristics of the reconstruction priority and spatial location coordinates, including:

[0029] The semantic category identifiers in the target region set are vector-encoded to obtain semantic encoding vectors. The similarity between semantic encoding vectors is calculated and clustered to obtain multiple semantic clusters.

[0030] The variance of the intra-cluster semantic encoding vector of each semantic cluster is calculated as a semantic consistency index. A semantic level is assigned to each semantic cluster based on the semantic consistency index and the number of semantic category identifiers contained in the semantic cluster.

[0031] An initial priority value is determined for each target region based on the semantic hierarchy. The number of target regions at each semantic hierarchy is counted to obtain the semantic hierarchy quantity distribution. The initial priority value is normalized and adjusted based on the semantic hierarchy quantity distribution to obtain the reconstruction priority.

[0032] Cluster analysis is performed on the spatial coordinates of all target regions in the target region set to obtain multiple spatial clusters. The spatial cluster to which each target region belongs is determined, and the reconstruction priority distribution of all target regions within the spatial cluster is statistically analyzed as a spatial distribution characteristic.

[0033] The spatial adjustment factor is obtained by calculating the difference between the reconstruction priority and spatial distribution characteristics of the target area. The reconstruction priority is then corrected based on the spatial adjustment factor, and the corrected reconstruction priority is used as the reconstruction intensity parameter.

[0034] Texture and semantic features are input into the decoder of the large visual model to obtain semantic correlation information between target regions. Based on this semantic correlation information, the reconstruction intensity parameters are adjusted to obtain the adjusted set of reconstruction parameters, which includes:

[0035] The texture features and semantic features are aligned in terms of feature dimensions, and the aligned texture features and semantic features are then concatenated to obtain a fused feature vector.

[0036] The fused feature vectors are input into the decoder of the large visual model and processed through multiple layers to obtain the decoded feature representation. Multi-head attention is then performed on the decoded feature representation to obtain the attention weight matrix. The attention weight matrix is ​​then reorganized according to the target region index to obtain the correlation strength matrix between target regions.

[0037] Extract the pairing relationships of target regions with intensity values ​​exceeding a preset threshold from the association strength matrix, and use the corresponding association strength values ​​as semantic association information between target regions;

[0038] Obtain the reconstruction intensity parameters corresponding to all target regions involved in the target region pairing relationship in the semantic association information. Adjust the reconstruction intensity parameters according to the association intensity value in the target region pairing relationship, output the adjusted reconstruction intensity parameters, and summarize the adjusted reconstruction intensity parameters of all target regions to form the adjusted reconstruction parameter set.

[0039] Based on the adjusted set of reconstruction parameters and spatial coordinates, adaptive super-resolution reconstruction is performed on the corresponding target region of the original image to obtain the optimized image, including:

[0040] Based on the adjusted set of reconstruction parameters, the upsampling factor and convolution kernel size are determined as adaptive super-resolution reconstruction parameters for each target region;

[0041] Image patches corresponding to the target region are extracted from the original image to be processed based on spatial location coordinates, and feature extraction of the image patches is performed based on adaptive super-resolution reconstruction parameters to obtain the target region features;

[0042] Upsampled features are obtained by upsampling the target region features based on adaptive super-resolution reconstruction parameters. Overlapping boundary regions of adjacent target regions are identified based on spatial location coordinates. The upsampled features of the overlapping boundary regions are then weighted and averaged to obtain the fused features.

[0043] The fused features are convolved and dimensionality reduced to obtain the reconstructed features of the image channel dimension. The reconstructed features are converted into pixel values ​​to obtain the reconstructed target region image. The reconstructed target region image is filled into the corresponding position of the original image to be processed according to the spatial location coordinates to obtain the optimized image.

[0044] A second aspect of this invention provides a real-time image data analysis and intelligent decision-making system powered by large models, comprising:

[0045] The image preprocessing unit is used to acquire the original image to be processed, perform preliminary super-resolution reconstruction processing on the original image to be processed to generate a preliminary reconstructed image, input the preliminary reconstructed image into the encoder of the visual large model, and output texture features and semantic features.

[0046] The feature extraction unit is used to perform target recognition based on semantic features to obtain a set of target regions, wherein each target region in the set of target regions contains a semantic category identifier and spatial location coordinates;

[0047] The regional analysis unit is used to perform semantic hierarchical division of semantic category identifiers in the target region set, assign reconstruction priority to each target region according to the semantic hierarchy, and determine reconstruction intensity parameters based on the spatial distribution characteristics of reconstruction priority and spatial location coordinates.

[0048] The parameter adjustment unit is used to input texture features and semantic features into the decoder of the large visual model to obtain semantic correlation information between target regions, and adjust the reconstruction intensity parameters according to the semantic correlation information to obtain the adjusted reconstruction parameter set.

[0049] The reconstruction execution unit is used to perform adaptive super-resolution reconstruction processing on the corresponding target region of the original image to be processed, based on the adjusted reconstruction parameter set and spatial location coordinates, to obtain the optimized image.

[0050] A third aspect of the present invention provides an electronic device, comprising:

[0051] processor;

[0052] Memory used to store processor-executable instructions;

[0053] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0054] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0055] In this embodiment, a large visual model is used to perform both semantic and textural analysis on the image, transforming super-resolution reconstruction from uniform enhancement to adaptive enhancement based on semantic differences. This prioritizes improving the detail clarity and structural integrity of key target regions while suppressing over-sharpening and noise amplification in non-critical areas. Dynamically adjusting the reconstruction intensity based on semantic hierarchy and regional correlation helps enhance structural consistency and overall visual coherence between targets, improving recognition accuracy and decision reliability in complex scenes. Simultaneously, it achieves more efficient resource allocation and real-time processing capabilities under computationally limited conditions. Attached Figure Description

[0056] Figure 1 A flowchart illustrating the real-time image data analysis and intelligent decision-making method for enabling large-scale models in embodiments of the present invention;

[0057] Figure 2 This is a flowchart illustrating the adaptive super-resolution reconstruction logic of an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0060] Figure 1 A flowchart illustrating the real-time image data analysis and intelligent decision-making method for empowering large models in embodiments of the present invention is shown below. Figure 1 As shown, the method includes:

[0061] The original image to be processed is obtained, and a preliminary super-resolution reconstruction process is performed on the original image to be processed to generate a preliminary reconstructed image. The preliminary reconstructed image is input into the encoder of the large visual model, and the texture features and semantic features are output.

[0062] Target recognition based on semantic features yields a set of target regions, where each target region contains a semantic category identifier and spatial location coordinates.

[0063] Semantic hierarchy is performed on the semantic category identifiers in the target region set. Reconstruction priority is assigned to each target region according to the semantic hierarchy. Reconstruction intensity parameters are determined based on the spatial distribution characteristics of reconstruction priority and spatial location coordinates.

[0064] Texture features and semantic features are input into the decoder of the large visual model to obtain semantic correlation information between target regions. Based on the semantic correlation information, the reconstruction intensity parameters are adjusted to obtain the adjusted reconstruction parameter set.

[0065] Based on the adjusted set of reconstruction parameters and spatial coordinates, adaptive super-resolution reconstruction is performed on the corresponding target region of the original image to obtain the optimized image.

[0066] The original image to be processed undergoes preliminary super-resolution reconstruction to generate a preliminary reconstructed image. This preliminary reconstructed image is then input into the encoder of the large visual model, which outputs texture and semantic features, including:

[0067] The gradient magnitude is calculated to obtain a gradient distribution map of the original image to be processed. The kernel size is determined based on the statistical distribution of the gradient magnitude in the gradient distribution map. The upsampling factor is determined based on the degree of clustering of the gradient directions in the gradient distribution map.

[0068] A multi-scale convolutional network is constructed based on the kernel size, and convolution operations are performed on the original image to be processed to obtain a multi-scale feature map. Pixel rearrangement is performed on the multi-scale feature map according to the upsampling factor to complete the initial super-resolution reconstruction process and generate an initial reconstructed image.

[0069] The initially reconstructed image is input into the encoder of the large visual model. The encoder divides the initially reconstructed image into multiple image blocks. Frequency domain transformation is performed on multiple image blocks to extract frequency domain components, which are then input into the frequency domain coding path of the encoder to obtain frequency domain coding features. Spatial convolution is performed on multiple image blocks to extract spatial components, which are then input into the spatial coding path of the encoder to obtain spatial coding features.

[0070] Frequency domain coding features and spatial coding features are fused to obtain fused coding features. The fused coding features are then separated to obtain texture features and semantic features, which are used as the encoder output of the large visual model.

[0071] When performing preliminary super-resolution reconstruction on the original image, the gray-level change rate of each pixel in the horizontal and vertical directions is first calculated. Then, the Sobel operator or Prewitt operator is used to perform convolution operations on the original image to obtain the horizontal gradient component G. x With the vertical gradient component G y Calculate the gradient magnitude The gradient magnitudes of all pixels are used to construct a gradient distribution map.

[0072] The gradient distribution map is analyzed to determine the numerical range of gradient magnitudes. The mean and standard deviation of the gradient magnitudes are calculated. When the mean is greater than a preset threshold, it indicates that the image contains rich edge information; in this case, a larger convolutional kernel size, such as 7×7 or 9×9, is chosen to capture a wider range of spatial context. When the mean is smaller, a 3×3 or 5×5 convolutional kernel is chosen to preserve details. The gradient direction angle θ = arctan(G) is calculated. y / G x The distribution histogram of angles in each direction is statistically analyzed. If the gradient ratio in a certain direction exceeds 40%, it is considered that the gradient direction is highly clustered. In this case, the upsampling factor is set to 2 or 3 times. If the gradient direction is dispersed, the upsampling factor is set to 4 times to enhance the reconstruction effect.

[0073] A multi-scale convolutional network is constructed based on the determined kernel size, comprising shallow, mid, and deep convolutional layers. Shallow convolutional layers use smaller kernels to extract local texture features, mid-scale convolutional layers use medium-sized kernels to extract medium-range structural features, and deep convolutional layers use larger kernels to extract global semantic features. The original image to be processed is input into each of the three scale convolutional layers for convolutional operations. After each convolution, batch normalization and ReLU activation are performed to obtain shallow, mid, and deep feature maps. The three scale feature maps are concatenated along the channel dimension to form a multi-scale feature map. Pixel rearrangement is performed on the multi-scale feature map according to the upsampling factor. Specifically, the number of channels in the feature map is increased by the square of the original number of channels through the convolutional layers. Then, the channel-dimensional features are rearranged into the spatial dimension to achieve spatial resolution improvement, completing the initial super-resolution reconstruction and generating a preliminary reconstructed image.

[0074] The initially reconstructed image is input into the encoder of the large-scale visual model. The encoder divides the initially reconstructed image into multiple image blocks of 16×16 or 32×32 pixels according to a fixed size. A two-dimensional discrete cosine transform or Fourier transform is performed on each image block to convert the spatial domain signal into a frequency domain signal, extracting low-frequency, mid-frequency, and high-frequency components as frequency domain components. The low-frequency components reflect the overall brightness and color distribution of the image block, the mid-frequency components reflect edge and contour information, and the high-frequency components reflect detail texture and noise. The extracted frequency domain components are input into the frequency domain coding path of the encoder. This path consists of multiple fully connected layers and an attention mechanism, performing nonlinear mapping and feature enhancement on the frequency domain components to obtain frequency domain coded features.

[0075] Simultaneously, spatial convolution operations are performed on multiple image patches using 3×3 convolution kernels to extract the spatial structural relationships and pixel neighborhood correlations within the image patches as spatial components. The spatial convolutional layer contains multiple convolution kernels, each extracting spatial features at different directions and scales. These spatial components are input into the spatial encoding path of the encoder, and spatial encoded features are obtained through multiple layers of convolution, pooling, and residual connections. The spatial encoded features preserve the geometric shape and spatial layout information of the image patches.

[0076] Frequency domain encoded features and spatial encoded features are weighted and summed or concatenated along the feature dimension, with the weights dynamically adjusted according to feature importance, to obtain fused encoded features. These fused encoded features contain both frequency domain and spatial information. Principal component analysis (PCA) or independent component analysis (ICA) are applied to the fused encoded features for feature separation. High-frequency components and features with strong edge responses are separated into texture features, while low-frequency components and features with strong regional continuity are separated into semantic features. These are then used as outputs of the large-scale visual encoder for subsequent processing.

[0077] The target region set obtained by target recognition based on semantic features includes:

[0078] Multi-scale semantic features are obtained by performing multi-scale convolution on semantic features, and the spatial attention distribution of multi-scale semantic features is calculated and attention weights are generated.

[0079] Attention weights are used to weight multi-scale semantic features to generate weighted semantic features. A sliding window operation is performed on the weighted semantic features to generate candidate target regions and calculate confidence scores. Valid candidate target regions are obtained by filtering candidate target regions based on their confidence scores.

[0080] Extract regional features from valid candidate target regions and classify them to obtain semantic category identifiers, and calculate the boundary coordinates of valid candidate target regions to obtain spatial location coordinates;

[0081] Based on the spatial location coordinates, local texture features of the corresponding region are extracted from the texture features, and the matching degree between the local texture features and the semantic category identifier is calculated to obtain the texture consistency score;

[0082] A comprehensive score is obtained by weighted summation of texture consistency score and confidence score. Valid candidate target regions are selected based on the comprehensive score to obtain the final target region. The semantic category identifier and spatial location coordinates of the final target region are combined to form the target region set.

[0083] Three convolutional operations with different kernel sizes (3×3, 5×5, and 7×7) are performed on the semantic features, with a stride of 1 for each kernel and zero padding to maintain the feature map size. The three kernels capture fine-grained local texture, medium-range structural information, and large-range contextual semantic information, respectively, resulting in three semantic feature maps at different scales. For each scale's semantic feature map, the global average pooling value and global max pooling value are calculated along the channel dimension. The two pooling results are concatenated and passed through a two-layer fully connected network to generate a spatial attention weight map, with the weight map values ​​normalized to between 0 and 1. The attention weight maps corresponding to the three scales are then multiplied element-wise with their respective semantic feature maps, and the three weighted feature maps are concatenated along the channel dimension to obtain the weighted semantic features.

[0084] A sliding window is set on the weighted semantic features. The window size is determined based on the typical size range of the target region, and the window sliding step is set to one-quarter of the window size to ensure sufficient coverage. Feature vectors are extracted for each window location, and these feature vectors are input into a pre-trained binary classifier to determine whether the region contains the target object. The classifier outputs a confidence score representing the probability of containing the target. A confidence threshold of 0.6 is set, and window regions with confidence scores greater than the threshold are selected as valid candidate target regions.

[0085] For each valid candidate target region, a feature block at the corresponding location is cropped from the weighted semantic features. This feature block is then processed by a region feature extraction network to generate a fixed-dimensional region feature vector. The region feature vector is input into a multi-classifier, which outputs the probability distribution of each semantic category. The category with the highest probability is selected as the semantic category identifier for that region. Simultaneously, the coordinates of the upper-left and lower-right corners of the valid candidate target region in the original image coordinate system are recorded, and these four coordinate values ​​are combined to form the spatial location coordinates.

[0086] The target region's location in the texture feature map is determined based on its spatial coordinates. Considering the scaling relationship between the feature map and the original image, the spatial coordinates are mapped proportionally to the feature map coordinate system. The corresponding region is cropped from the texture feature map to obtain a local texture feature block. The statistical properties of this feature block, including the texture orientation gradient histogram and the gray-level co-occurrence matrix, are calculated. The local texture features are then compared with the standard texture template corresponding to the semantic category identifier of the region. A cosine similarity metric is used, and the similarity score is normalized to between 0 and 1 as the texture consistency score. The texture consistency score reflects the degree of matching between the texture characteristics of the target region and its semantic category; a high score indicates that the region indeed belongs to the identified semantic category.

[0087] For each valid candidate target region, its texture consistency score and confidence score are weighted and summed using weighting coefficients of 0.4 and 0.6 respectively to calculate a comprehensive score. A comprehensive score threshold of 0.65 is set, and valid candidate target regions with comprehensive scores higher than the threshold are retained as final target regions. Non-maximum suppression is performed on final target regions with spatial overlap, and the intersection-union ratio (IUR) between regions is calculated. When the IUR is greater than 0.5, the region with the higher comprehensive score is retained, and the other region is deleted. The semantic category identifiers and spatial coordinates of the filtered final target regions are combined one by one to form a target region set containing information on several target regions. Each element in the set contains complete semantic and spatial information for subsequent processing.

[0088] Based on spatial location coordinates, local texture features of the corresponding region are extracted from the texture features. The texture consistency score is obtained by calculating the matching degree between the local texture features and the semantic category identifier, including:

[0089] The horizontal and vertical coordinate ranges of the target region are obtained by parsing the spatial location coordinates. Based on the horizontal and vertical coordinate ranges, the local texture features of the corresponding region are cropped and extracted from the texture features.

[0090] Local texture features are averaged by channel pooling to obtain texture feature vectors. Semantic category identifiers are vector-encoded to obtain semantic vectors. The cosine similarity between texture feature vectors and semantic vectors is calculated to obtain semantic matching scores.

[0091] The frequency domain features are obtained by performing Fourier transform on the local texture features. The amplitude spectrum is extracted from the frequency domain features to obtain the frequency energy distribution. The standard frequency energy distribution is retrieved from the preset texture library according to the semantic category identifier. The mean square error between the frequency energy distribution and the standard frequency energy distribution is calculated to obtain the frequency deviation score.

[0092] The gradient magnitude of local texture features is calculated to obtain the edge intensity distribution. The edge intensity reference range is determined according to the semantic category identifier. The percentage of pixels whose edge intensity distribution falls within the edge intensity reference range is calculated to obtain the edge consistency score.

[0093] The texture consistency score is obtained by weighted summation of semantic matching score, frequency deviation score and edge consistency score.

[0094] Before performing super-resolution reconstruction on each target region, it is necessary to evaluate the consistency between local texture and semantic labeling. First, the spatial coordinates of the target region are parsed. Assume the spatial coordinates of a identified target region are represented as (x... min y min x max y max ), where x min With x max These represent the starting and ending values ​​of the x-axis, y. min With y max These represent the starting and ending values ​​of the ordinate. Based on the rectangular area defined by these four values, a spatial cropping operation is performed on the texture feature tensor output by the large visual model encoder to extract the local texture feature sub-tensor corresponding to the target region. This local texture feature preserves the texture details within the region.

[0095] To evaluate the matching degree between texture and semantics, the extracted local texture features are first subjected to channel-dimensional average pooling, compressing the multi-channel features into a one-dimensional feature vector as the texture feature vector. Simultaneously, the semantic category identifier of the target region is encoded using one-hot encoding or embedding layer encoding to obtain a fixed-dimensional semantic vector. The cosine similarity between the texture feature vector and the semantic vector is calculated; this similarity value ranges from -1 to 1, with a value closer to positive one indicating a stronger correlation between the texture feature and the semantic category. This similarity is then mapped to the zero-to-one interval to obtain the semantic matching score.

[0096] This study analyzes the energy distribution characteristics of texture features from a frequency domain perspective. A two-dimensional discrete Fourier transform is performed on the local texture features to obtain their complex frequency domain representation. The amplitude spectrum of these complex features is extracted, reflecting the energy magnitude of different frequency components and forming a frequency energy distribution. Based on the semantic category identifier of the target region, a standard frequency energy distribution template for the corresponding category is retrieved from a pre-built texture library. The mean square error between the current frequency energy distribution and the standard frequency energy distribution is calculated; a larger error value indicates a more severe deviation of the texture features from the standard pattern. The mean square error is normalized and inverted to obtain a frequency deviation score, where a higher score indicates that the frequency characteristics are more consistent with expectations.

[0097] The texture library is a standard frequency feature database built offline based on the training dataset. Specifically, for each semantic category, a set of sample images for that category is collected, the sample images are cropped and subjected to a two-dimensional discrete Fourier transform, the frequency energy distribution of the amplitude spectrum of each sample is statistically analyzed, the frequency energy distribution is normalized and then the mean or median is calculated to obtain the standard frequency energy distribution template corresponding to that semantic category, and it is stored in the database with a mapping relationship between the category identifier and the corresponding template parameter vector.

[0098] Verify the structural consistency of textures from an edge perspective. Calculate gradients for local texture features in both the horizontal and vertical directions. Take the square root of the sum of the squares of the horizontal and vertical gradients to obtain the gradient magnitude, which reflects the edge strength at each pixel location. Statistically analyze the gradient magnitude distribution of all pixels within the entire target area. Query a preset edge strength reference range based on the semantic category identifier; for example, the reference range for buildings is 50 to 150, and for vegetation it is 10 to 50. Count the number of pixels in the current edge strength distribution that fall within the reference range, calculate the proportion of this number to the total number of pixels in the target area, and obtain the edge consistency score.

[0099] Based on the evaluation results of the three dimensions above, a weighted fusion of the semantic matching score, frequency deviation score, and edge consistency score is performed. The weights for the semantic matching score, frequency deviation score, and edge consistency score are set to 0.4, 0.3, and 0.3 respectively. The three scores are then linearly weighted and summed according to their respective weights to obtain the texture consistency score of the target region. This score comprehensively reflects the degree of matching between local texture features and semantic category labels, providing a quantitative basis for subsequent adjustments to reconstruction intensity parameters. A higher texture consistency score indicates a high degree of consistency between the texture features and semantic labels in the region, allowing for a suitable reduction in reconstruction intensity to avoid overprocessing; a lower score indicates the presence of texture anomalies, requiring increased reconstruction intensity for repair.

[0100] The semantic category identifiers in the target region set are semantically hierarchically divided. A reconstruction priority is assigned to each target region based on the semantic hierarchy. The reconstruction intensity parameters are determined based on the spatial distribution characteristics of the reconstruction priority and spatial location coordinates, including:

[0101] The semantic category identifiers in the target region set are vector-encoded to obtain semantic encoding vectors. The similarity between semantic encoding vectors is calculated and clustered to obtain multiple semantic clusters.

[0102] The variance of the intra-cluster semantic encoding vector of each semantic cluster is calculated as a semantic consistency index. A semantic level is assigned to each semantic cluster based on the semantic consistency index and the number of semantic category identifiers contained in the semantic cluster.

[0103] An initial priority value is determined for each target region based on the semantic hierarchy. The number of target regions at each semantic hierarchy is counted to obtain the semantic hierarchy quantity distribution. The initial priority value is normalized and adjusted based on the semantic hierarchy quantity distribution to obtain the reconstruction priority.

[0104] Cluster analysis is performed on the spatial coordinates of all target regions in the target region set to obtain multiple spatial clusters. The spatial cluster to which each target region belongs is determined, and the reconstruction priority distribution of all target regions within the spatial cluster is statistically analyzed as a spatial distribution characteristic.

[0105] The spatial adjustment factor is obtained by calculating the difference between the reconstruction priority and spatial distribution characteristics of the target area. The reconstruction priority is then corrected based on the spatial adjustment factor, and the corrected reconstruction priority is used as the reconstruction intensity parameter.

[0106] After obtaining the target region set, deep analysis of the semantic category identifiers is required to determine the reconstruction strength parameters. A pre-trained semantic encoder is used to convert each semantic category identifier into a 512-dimensional semantic encoding vector. This encoder is trained on a large-scale visual semantic dataset and can capture semantic associations between categories. The cosine similarity between any two semantic encoding vectors is calculated to construct a similarity matrix. Density clustering is used to cluster the semantic encoding vectors, setting a similarity threshold of 0.75 and a minimum cluster size of 3, resulting in several semantic clusters, each containing semantically similar category identifiers.

[0107] For each semantic cluster, the variance of all semantic encoding vectors within the cluster is calculated across all dimensions, and the average of the variances across all dimensions is taken as the semantic consistency index. A smaller semantic consistency index indicates that the semantic categories within the cluster are more similar. Simultaneously, the number of different semantic category identifiers contained in each semantic cluster is counted. The semantic consistency index is then normalized and weighted by the reciprocal of the number of categories, with weighting coefficients of 0.6 and 0.4 respectively. Based on the weighting result, semantic clusters are assigned semantic levels from high to low, with level values ​​ranging from 1 to 5.

[0108] An initial priority value is directly assigned based on the semantic level of the semantic cluster to which the target region belongs; level 1 corresponds to priority 10, and level 5 corresponds to priority 2. The number of target regions contained in each semantic level is counted, and a histogram of the semantic level quantity distribution is constructed. If the proportion of target regions in a certain semantic level exceeds 0.4 of the total, the initial priority value of that level is multiplied by a decay coefficient of 0.85. All adjusted initial priority values ​​are normalized to their maximum and minimum values, mapping them to the interval between 0 and 1 to obtain the reconstruction priority.

[0109] The center point coordinates of all target regions in the target region set are extracted. K-means clustering is then used for spatial clustering, with the number of clusters K dynamically determined based on the total number of target regions, set as the square root of the total number of target regions rounded up. Each target region is assigned to the nearest spatial cluster based on its center point coordinates. For each spatial cluster, the reconstruction priority values ​​of all target regions within the cluster are statistically analyzed, and the mean and standard deviation of the priorities are calculated. The pair consisting of the mean and standard deviation is used as the spatial distribution characteristic of that spatial cluster.

[0110] For any target region i, obtain its original reconstruction priority P. i The spatial cluster k to which the target region belongs is determined. The reconstruction priority values ​​of all target regions within the spatial cluster k are statistically analyzed, and the mean priority value μ within the cluster is calculated. k and priority standard deviation σ k .

[0111] Subsequently, the normalized difference in target region reconstruction priority relative to intra-cluster distribution is calculated: ;

[0112] Where ε is a preset minimum positive number, used to avoid division by zero when the standard deviation is zero.

[0113] Standardized difference D i The spatial adjustment factor S is obtained by performing a nonlinear mapping using the Sigmoid function. i : ;

[0114] The spatial adjustment factor S i The value ranges from 0 to 1 and is used to reflect the degree to which the priority of the target region deviates from the average level within the cluster.

[0115] Based on the relationship between the reconstruction priority of the target region and the mean within the cluster, the original reconstruction priority is modified in segments:

[0116] When P i ≥μ k At that time, P i =Pi ×( When P i <μ k At that time, P i =P i ×( ), where α and β are preset adjustment coefficients, which are 0.3 and 0.2 respectively in this embodiment, but are not limited to this and can be adaptively adjusted according to specific application scenarios.

[0117] Ultimately, the corrected reconstruction priority Pi′ is used as the reconstruction intensity parameter for the target region, and is used to adjust the allocation of computational resources and parameter configuration of the reconstruction algorithm in subsequent adaptive super-resolution reconstruction processing.

[0118] Through the aforementioned spatial adaptive correction mechanism, target areas with significant differences in priority within the same spatial clustering region can obtain a more reasonable allocation of reconstruction resources, thereby improving the spatial consistency and visual coordination of the overall reconstruction results.

[0119] Texture and semantic features are input into the decoder of the large visual model to obtain semantic correlation information between target regions. Based on this semantic correlation information, the reconstruction intensity parameters are adjusted to obtain the adjusted set of reconstruction parameters, which includes:

[0120] The texture features and semantic features are aligned in terms of feature dimensions, and the aligned texture features and semantic features are then concatenated to obtain a fused feature vector.

[0121] The fused feature vectors are input into the decoder of the large visual model and processed through multiple layers to obtain the decoded feature representation. Multi-head attention is then performed on the decoded feature representation to obtain the attention weight matrix. The attention weight matrix is ​​then reorganized according to the target region index to obtain the correlation strength matrix between target regions.

[0122] Extract the pairing relationships of target regions with intensity values ​​exceeding a preset threshold from the association strength matrix, and use the corresponding association strength values ​​as semantic association information between target regions;

[0123] Obtain the reconstruction intensity parameters corresponding to all target regions involved in the target region pairing relationship in the semantic association information. Adjust the reconstruction intensity parameters according to the association intensity value in the target region pairing relationship, output the adjusted reconstruction intensity parameters, and summarize the adjusted reconstruction intensity parameters of all target regions to form the adjusted reconstruction parameter set.

[0124] After obtaining texture and semantic features, their feature vectors typically differ in dimensionality because these two types of features originate from different branches of the visual large-scale encoder. Texture features might have a 512-dimensional dimension, while semantic features might have a 768-dimensional dimension. To achieve effective feature fusion, feature dimension alignment is required first. This alignment is achieved through a linear projection layer, projecting texture features into the same 768-dimensional space as the semantic features, preserving local neighborhood relationships within the feature space during the projection process. The aligned texture and semantic features are then concatenated along the channel dimension to form a 1536-dimensional fused feature vector. This fused feature vector simultaneously contains both texture details of the image and semantic understanding information of the object.

[0125] The fused feature vectors are fed into the decoder of the large-scale visual model for multi-layer decoding. The decoder typically contains 6 to 12 Transformer decoding layers, each including a self-attention mechanism and a feedforward neural network. The decoding process extracts higher-order semantic relationships from the fused features layer by layer, outputting a decoded feature representation of dimension N×D, where N represents the number of target regions and D represents the feature dimension. A multi-head attention mechanism is applied to the decoded feature representation, with 8 attention heads, each independently calculating the correlation between target regions. The multi-head attention calculation outputs an N×N attention weight matrix, where the element in the i-th row and j-th column represents the degree of attention given by the i-th target region to the j-th target region.

[0126] The row and column indices of the attention weight matrix correspond to the indexes of the target regions identified in the initially reconstructed image. To establish the association between target regions, the attention weight matrix is ​​re-dimensioned according to the target region indices. The re-dimension operation performs average pooling on the weight values ​​of multiple attention heads corresponding to the same pair of target regions in the matrix, resulting in the association strength matrix between target regions. The association strength matrix is ​​a symmetric matrix, with element values ​​ranging from 0 to 1; larger values ​​indicate a stronger semantic association between the two target regions.

[0127] Elements with intensity values ​​exceeding a preset threshold are selected from the association strength matrix. This threshold is typically set to 0.6. The matrix position (i, j) where the intensity value exceeds the threshold indicates a significant semantic association between the i-th and j-th target regions, forming a target region pairing. The association strength value corresponding to this pairing is also recorded. All pairings that meet the conditions and their association strength values ​​together constitute the semantic association information between target regions. This semantic association information reflects which targets in the image have semantically cooperative relationships; for example, "person" and "chair" often appear together and have an interactive relationship.

[0128] The system retrieves all target regions involved in pairing relationships from semantic association information and queries the corresponding reconstruction intensity parameters for these regions. These parameters include values ​​such as upsampling factor and filter kernel size. For paired target region pairs, a weighted adjustment is performed based on their association strength values. The adjustment strategy is as follows: if the association strength values ​​of two target regions are high, the reconstruction intensity parameters are shifted towards the parameter of the higher-priority target, with the adjustment magnitude proportional to the association strength value. The adjustment calculation uses a weighted average method, with weight coefficients obtained by normalizing the association strength values. The adjusted reconstruction intensity parameters ensure visual consistency among semantically associated target regions. All adjusted reconstruction intensity parameters for all target regions are arranged in target region index order and summarized to form an adjusted reconstruction parameter set for subsequent adaptive super-resolution reconstruction processing.

[0129] like Figure 2 As shown, Figure 2 This is the adaptive super-resolution reconstruction logic flow of an embodiment of the present invention.

[0130] Based on the adjusted set of reconstruction parameters and spatial coordinates, adaptive super-resolution reconstruction is performed on the corresponding target region of the original image to obtain the optimized image, including:

[0131] Based on the adjusted set of reconstruction parameters, the upsampling factor and convolution kernel size are determined as adaptive super-resolution reconstruction parameters for each target region;

[0132] Image patches corresponding to the target region are extracted from the original image to be processed based on spatial location coordinates, and feature extraction of the image patches is performed based on adaptive super-resolution reconstruction parameters to obtain the target region features;

[0133] Upsampled features are obtained by upsampling the target region features based on adaptive super-resolution reconstruction parameters. Overlapping boundary regions of adjacent target regions are identified based on spatial location coordinates. The upsampled features of the overlapping boundary regions are then weighted and averaged to obtain the fused features.

[0134] The fused features are convolved and dimensionality reduced to obtain the reconstructed features of the image channel dimension. The reconstructed features are converted into pixel values ​​to obtain the reconstructed target region image. The reconstructed target region image is filled into the corresponding position of the original image to be processed according to the spatial location coordinates to obtain the optimized image.

[0135] After obtaining the adjusted set of reconstruction parameters, a personalized super-resolution reconstruction process is executed for each target region. First, each region in the target region set is traversed, and the corresponding parameter values ​​are extracted from the adjusted set of reconstruction parameters. These parameter values ​​include numerical indices reflecting the reconstruction intensity. The upsampling factor is determined based on the mapping relationship between this numerical indices and a preset scaling factor. For example, a 4x upsampling factor is selected when the reconstruction intensity parameter is in the range [0.7, 1.0], a 2x upsampling factor is selected when it is in the range [0.4, 0.7), and the original resolution is used when it is below 0.4. Simultaneously, the convolution kernel size is selected based on the semantic category characteristics of the target region. For regions containing fine textures such as faces or text, a small 3×3 convolution kernel is used to preserve details, while for geometrically structured regions such as buildings or vehicles, a 5×5 or 7×7 convolution kernel is used to enhance the overall contour.

[0136] Based on the bounding box information of the target region recorded in the spatial coordinates, corresponding image patches are cropped from the pixel matrix of the original image to be processed. For each image patch, a multi-layer convolutional neural network is constructed for feature extraction. The first layer uses a fixed kernel size to extract primary texture features, and subsequent layers extract deeper semantic features step by step through residual connections. During the feature extraction process, the number of channels in the feature map is dynamically adjusted according to the upsampling ratio. For example, 4x upsampling corresponds to 256 channels, and 2x upsampling corresponds to 128 channels, ensuring that subsequent upsampling operations have sufficient feature representation capabilities. The extracted target region features retain the semantic information and texture distribution characteristics of the original image patches.

[0137] When performing upsampling on the target region features, subpixel convolution or transposed convolution techniques are used to generate a high-resolution feature map based on a predetermined upsampling factor. Specifically, the feature channels are reorganized according to the square of the upsampling factor; for example, for 2x upsampling, four consecutive channels are rearranged into a 2×2 spatial grid, achieving a physical expansion of the feature map size. After upsampling, upsampled features matching the target resolution are obtained. To avoid obvious stitching boundaries between adjacent target regions in the image, overlapping boundary regions need to be identified. By calculating the intersection of the spatial coordinates of the target regions, pairs of overlapping adjacent regions are identified, and upsampled features corresponding to the overlapping pixel range are extracted. A weighted average fusion strategy is used for the features of the overlapping boundary regions. The weight coefficient is inversely proportional to the distance from the pixel to the center of its respective target region; the closer to the center, the greater the weight, ensuring a smooth transition. The fused features retain the reconstruction quality of each target region while eliminating stitching artifacts.

[0138] In the post-processing stage of the fused features, dimensionality reduction is first performed using a 1×1 convolutional layer to map the multi-channel features back to the RGB three-channel representation of the image, obtaining the reconstructed features in the image channel dimension. The numerical range of this reconstructed feature is normalized, and the feature values ​​are mapped to the pixel value range [0, 255] through linear transformation or the Sigmoid activation function, completing the transformation from feature space to pixel space and generating the reconstructed target region image. Finally, based on the original bounding box positions recorded by spatial coordinates, the reconstructed target region image is precisely filled back into the corresponding pixel positions of the original image to be processed. For background areas not covered by any target regions, the pixel values ​​of the initially reconstructed image remain unchanged. After completing the filling operation by traversing all target regions, an optimized image with improved overall resolution and significantly enhanced clarity in key areas is obtained. This image achieves differentiated reconstruction effects while maintaining global visual coherence.

[0139] A second aspect of this invention provides a large-model-enabled real-time image data analysis and intelligent decision-making system, the system comprising:

[0140] The image preprocessing unit is used to acquire the original image to be processed, perform preliminary super-resolution reconstruction processing on the original image to be processed to generate a preliminary reconstructed image, input the preliminary reconstructed image into the encoder of the visual large model, and output texture features and semantic features.

[0141] The feature extraction unit is used to perform target recognition based on semantic features to obtain a set of target regions, wherein each target region in the set of target regions contains a semantic category identifier and spatial location coordinates;

[0142] The regional analysis unit is used to perform semantic hierarchical division of semantic category identifiers in the target region set, assign reconstruction priority to each target region according to the semantic hierarchy, and determine reconstruction intensity parameters based on the spatial distribution characteristics of reconstruction priority and spatial location coordinates.

[0143] The parameter adjustment unit is used to input texture features and semantic features into the decoder of the large visual model to obtain semantic correlation information between target regions, and adjust the reconstruction intensity parameters according to the semantic correlation information to obtain the adjusted reconstruction parameter set.

[0144] The reconstruction execution unit is used to perform adaptive super-resolution reconstruction processing on the corresponding target region of the original image to be processed, based on the adjusted reconstruction parameter set and spatial location coordinates, to obtain the optimized image.

[0145] A third aspect of the present invention provides an electronic device, comprising:

[0146] processor;

[0147] Memory used to store processor-executable instructions;

[0148] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0149] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0150] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for real-time image data analysis and intelligent decision-making empowered by large models, characterized in that: include: The original image to be processed is obtained, and a preliminary super-resolution reconstruction process is performed on the original image to be processed to generate a preliminary reconstructed image. The preliminary reconstructed image is input into the encoder of the large visual model, and the texture features and semantic features are output. Target recognition based on semantic features yields a set of target regions, where each target region contains a semantic category identifier and spatial location coordinates. Semantic hierarchy is performed on the semantic category identifiers in the target region set. Reconstruction priority is assigned to each target region according to the semantic hierarchy. Reconstruction intensity parameters are determined based on the spatial distribution characteristics of reconstruction priority and spatial location coordinates. Texture features and semantic features are input into the decoder of the large visual model to obtain semantic correlation information between target regions. Based on the semantic correlation information, the reconstruction intensity parameters are adjusted to obtain the adjusted reconstruction parameter set. Based on the adjusted set of reconstruction parameters and spatial coordinates, adaptive super-resolution reconstruction is performed on the corresponding target region of the original image to obtain the optimized image.

2. The method according to claim 1, characterized in that, The original image to be processed undergoes preliminary super-resolution reconstruction to generate a preliminary reconstructed image. This preliminary reconstructed image is then input into the encoder of the large visual model, which outputs texture and semantic features, including: The gradient magnitude is calculated to obtain a gradient distribution map of the original image to be processed. The kernel size is determined based on the statistical distribution of the gradient magnitude in the gradient distribution map. The upsampling factor is determined based on the degree of clustering of the gradient directions in the gradient distribution map. A multi-scale convolutional network is constructed based on the kernel size, and convolution operations are performed on the original image to be processed to obtain a multi-scale feature map. Pixel rearrangement is performed on the multi-scale feature map according to the upsampling factor to complete the initial super-resolution reconstruction process and generate an initial reconstructed image. The initially reconstructed image is input into the encoder of the large visual model. The encoder divides the initially reconstructed image into multiple image blocks. Frequency domain transformation is performed on multiple image blocks to extract frequency domain components, which are then input into the frequency domain coding path of the encoder to obtain frequency domain coding features. Spatial convolution is performed on multiple image blocks to extract spatial components, which are then input into the spatial coding path of the encoder to obtain spatial coding features. Frequency domain coding features and spatial coding features are fused to obtain fused coding features. The fused coding features are then separated to obtain texture features and semantic features, which are used as the encoder output of the large visual model.

3. The method according to claim 1, characterized in that, The target region set obtained by target recognition based on semantic features includes: Multi-scale semantic features are obtained by performing multi-scale convolution on semantic features, and the spatial attention distribution of multi-scale semantic features is calculated and attention weights are generated. Attention weights are used to weight multi-scale semantic features to generate weighted semantic features. A sliding window operation is performed on the weighted semantic features to generate candidate target regions and calculate confidence scores. Valid candidate target regions are obtained by filtering candidate target regions based on their confidence scores. Extract regional features from valid candidate target regions and classify them to obtain semantic category identifiers, and calculate the boundary coordinates of valid candidate target regions to obtain spatial location coordinates; Based on the spatial location coordinates, local texture features of the corresponding region are extracted from the texture features, and the matching degree between the local texture features and the semantic category identifier is calculated to obtain the texture consistency score; A comprehensive score is obtained by weighted summation of texture consistency score and confidence score. Valid candidate target regions are selected based on the comprehensive score to obtain the final target region. The semantic category identifier and spatial location coordinates of the final target region are combined to form the target region set.

4. The method according to claim 3, characterized in that, Based on spatial location coordinates, local texture features of the corresponding region are extracted from the texture features. The texture consistency score is obtained by calculating the matching degree between the local texture features and the semantic category identifier, including: The horizontal and vertical coordinate ranges of the target region are obtained by parsing the spatial location coordinates. Based on the horizontal and vertical coordinate ranges, the local texture features of the corresponding region are cropped and extracted from the texture features. Local texture features are averaged by channel pooling to obtain texture feature vectors. Semantic category identifiers are vector-encoded to obtain semantic vectors. The cosine similarity between texture feature vectors and semantic vectors is calculated to obtain semantic matching scores. The frequency domain features are obtained by performing Fourier transform on the local texture features. The amplitude spectrum is extracted from the frequency domain features to obtain the frequency energy distribution. The standard frequency energy distribution is retrieved from the preset texture library according to the semantic category identifier. The mean square error between the frequency energy distribution and the standard frequency energy distribution is calculated to obtain the frequency deviation score. The gradient magnitude of local texture features is calculated to obtain the edge intensity distribution. The edge intensity reference range is determined according to the semantic category identifier. The percentage of pixels whose edge intensity distribution falls within the edge intensity reference range is calculated to obtain the edge consistency score. The texture consistency score is obtained by weighted summation of semantic matching score, frequency deviation score and edge consistency score.

5. The method according to claim 1, characterized in that, The semantic category identifiers in the target region set are semantically hierarchically divided. A reconstruction priority is assigned to each target region based on the semantic hierarchy. The reconstruction intensity parameters are determined based on the spatial distribution characteristics of the reconstruction priority and spatial location coordinates, including: The semantic category identifiers in the target region set are vector-encoded to obtain semantic encoding vectors. The similarity between semantic encoding vectors is calculated and clustered to obtain multiple semantic clusters. The variance of the intra-cluster semantic encoding vector of each semantic cluster is calculated as a semantic consistency index. A semantic level is assigned to each semantic cluster based on the semantic consistency index and the number of semantic category identifiers contained in the semantic cluster. An initial priority value is determined for each target region based on the semantic hierarchy. The number of target regions at each semantic hierarchy is counted to obtain the semantic hierarchy quantity distribution. The initial priority value is normalized and adjusted based on the semantic hierarchy quantity distribution to obtain the reconstruction priority. Cluster analysis is performed on the spatial coordinates of all target regions in the target region set to obtain multiple spatial clusters. The spatial cluster to which each target region belongs is determined, and the reconstruction priority distribution of all target regions within the spatial cluster is statistically analyzed as a spatial distribution characteristic. The spatial adjustment factor is obtained by calculating the difference between the reconstruction priority and spatial distribution characteristics of the target area. The reconstruction priority is then corrected based on the spatial adjustment factor, and the corrected reconstruction priority is used as the reconstruction intensity parameter.

6. The method according to claim 1, characterized in that, Texture and semantic features are input into the decoder of the large visual model to obtain semantic correlation information between target regions. Based on this semantic correlation information, the reconstruction intensity parameters are adjusted to obtain the adjusted set of reconstruction parameters, which includes: The texture features and semantic features are aligned in terms of feature dimensions, and the aligned texture features and semantic features are then concatenated to obtain a fused feature vector. The fused feature vectors are input into the decoder of the large visual model and processed through multiple layers to obtain the decoded feature representation. Multi-head attention is then performed on the decoded feature representation to obtain the attention weight matrix. The attention weight matrix is ​​then reorganized according to the target region index to obtain the correlation strength matrix between target regions. Extract the pairing relationships of target regions with intensity values ​​exceeding a preset threshold from the association strength matrix, and use the corresponding association strength values ​​as semantic association information between target regions; Obtain the reconstruction intensity parameters corresponding to all target regions involved in the target region pairing relationship in the semantic association information. Adjust the reconstruction intensity parameters according to the association intensity value in the target region pairing relationship, output the adjusted reconstruction intensity parameters, and summarize the adjusted reconstruction intensity parameters of all target regions to form the adjusted reconstruction parameter set.

7. The method according to claim 1, characterized in that, Based on the adjusted set of reconstruction parameters and spatial coordinates, adaptive super-resolution reconstruction is performed on the corresponding target region of the original image to obtain the optimized image, including: Based on the adjusted set of reconstruction parameters, the upsampling factor and convolution kernel size are determined as adaptive super-resolution reconstruction parameters for each target region; Image patches corresponding to the target region are extracted from the original image to be processed based on spatial location coordinates, and feature extraction of the image patches is performed based on adaptive super-resolution reconstruction parameters to obtain the target region features; Upsampled features are obtained by upsampling the target region features based on adaptive super-resolution reconstruction parameters. Overlapping boundary regions of adjacent target regions are identified based on spatial location coordinates. The upsampled features of the overlapping boundary regions are then weighted and averaged to obtain the fused features. The fused features are convolved and dimensionality reduced to obtain the reconstructed features of the image channel dimension. The reconstructed features are converted into pixel values ​​to obtain the reconstructed target region image. The reconstructed target region image is filled into the corresponding position of the original image to be processed according to the spatial location coordinates to obtain the optimized image.

8. A large-model-enabled real-time image data analysis and intelligent decision-making system, used to implement the method of any one of claims 1-7, characterized in that, include: The image preprocessing unit is used to acquire the original image to be processed, perform preliminary super-resolution reconstruction processing on the original image to be processed to generate a preliminary reconstructed image, input the preliminary reconstructed image into the encoder of the visual large model, and output texture features and semantic features. The feature extraction unit is used to identify targets based on semantic features to obtain a set of target regions, wherein each target region in the set of target regions contains a semantic category identifier and spatial location coordinates; The regional analysis unit is used to perform semantic hierarchical division of semantic category identifiers in the target region set, assign reconstruction priority to each target region according to the semantic hierarchy, and determine reconstruction intensity parameters based on the spatial distribution characteristics of reconstruction priority and spatial location coordinates. The parameter adjustment unit is used to input texture features and semantic features into the decoder of the large visual model to obtain semantic correlation information between target regions, and adjust the reconstruction intensity parameters according to the semantic correlation information to obtain the adjusted reconstruction parameter set. The reconstruction execution unit is used to perform adaptive super-resolution reconstruction processing on the corresponding target region of the original image to be processed, based on the adjusted reconstruction parameter set and spatial location coordinates, to obtain the optimized image.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images