Image data distribution drift strength metric method, system, and apparatus

Abstract

The application provides an image data distribution drift strength measurement method, system and device, the method comprising: using a preset attention-based deep semantic feature extraction network module, performing semantic feature extraction and attention enhancement processing on source domain images used for training an image semantic coding and decoding model and target domain images predicted by the image semantic coding and decoding model, to obtain enhanced feature data of the source domain images and the target domain images; performing semantic distance calculation of a preset dimension on the enhanced feature data to obtain original data distribution drift data; and performing ranking normalization and segmented nonlinear stretching on the original data distribution drift data to generate corresponding image data distribution drift strength data for performance evaluation of the image semantic coding and decoding model. The application can realize robust measurement of the distribution difference between the source domain and the target domain, and provide a reliable basis for model updating and performance maintenance in an image semantic communication system.

Description

Technical Field

[0001] This application relates to the fields of image processing and semantic communication technology, and in particular to a method, system and device for measuring the intensity of image data distribution drift. Background Technology

[0002] In edge intelligence and semantic communication systems, the performance of image semantic encoding and decoding models heavily depends on the consistency between the distribution of training data and real-time data. However, in dynamic scenarios such as connected vehicles and industrial vision, the distribution of real-time acquired image data can drift due to changes in environment, lighting, viewpoint, or task objectives, leading to a decrease in the reconstruction quality of the model in the target domain.

[0003] Existing methods for measuring data distribution drift mainly involve directly calculating the distance in the feature space. However, these methods have significant shortcomings when applied to image semantic features. First, the Euclidean distance of deep image features has an unbounded range, making it difficult to set a universal threshold to determine the severity of drift. Second, they are not sensitive to semantic changes in image content in dimensions such as spatial structure and channel design. Furthermore, the measurement results of existing methods are unstable, highly susceptible to sample noise, and cannot establish a reliable correlation with the model performance degradation rate. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method, system, and device for measuring the intensity of image data distribution drift, in order to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of this application provides a method for measuring the intensity of image data distribution drift, which includes the following steps: A pre-defined deep semantic feature extraction network module based on attention enhancement is used to perform semantic feature extraction and attention enhancement processing on the source domain image used to train the image semantic encoding and decoding model and the target domain image predicted by the image semantic encoding and decoding model, respectively, to obtain the enhanced feature data of the source domain image and the target domain image. The enhanced feature data is subjected to semantic distance calculation of a preset dimension to obtain the original data distribution drift data; The original data distribution drift data is ranked, normalized, and piecewise nonlinearly stretched to generate corresponding image data distribution drift intensity data for performance evaluation of the image semantic encoding and decoding model.

[0006] In some embodiments of this application, the attention-enhanced deep semantic feature extraction network module includes a deep semantic feature extraction network unit, a spatial attention sub-network unit, a channel attention sub-network unit, and a fusion unit; The deep semantic feature extraction network unit is used to receive the source domain image and the target domain image, and perform multi-scale semantic feature extraction on the source domain image and the target domain image to obtain the original feature data of the source domain image corresponding to the source domain image and the original feature data of the target domain image corresponding to the target domain image, and transmit the original feature data of the source domain image and the original feature data of the target domain image to the spatial attention sub-network unit and the channel attention sub-network unit respectively; The spatial attention sub-network unit is used to receive the original feature data of the source domain image and perform weight calculation on the original feature data of the source domain image to obtain the spatial weights corresponding to the original feature data of the source domain image. The channel attention sub-network unit is used to receive the original feature data of the target domain image and perform weight calculation on the original feature data of the target domain image to obtain the channel weights corresponding to the original feature data of the target domain image. The fusion unit is used to fuse the enhanced feature data of the source domain image and the target domain image respectively based on the spatial weight, the channel weight, the original feature data of the source domain image and the original feature data of the target domain image.

[0007] In some embodiments of this application, the fusion unit includes: The normalization subunit is used to normalize the spatial weights and the channel weights to obtain normalized spatial weights and normalized channel weights. The dot product fusion subunit is used to perform dot product fusion on the normalized spatial weights and the original feature data of the source domain image, and the normalized channel weights and the original feature data of the target domain image, respectively, to obtain the enhanced feature data of the source domain image and the target domain image; wherein, the enhanced feature data includes enhanced feature data of the source domain image and enhanced feature data of the target domain image.

[0008] In some embodiments of this application, the preset dimensions include a global distribution dimension, a spatial structure dimension, a channel statistics dimension, and an attention weighting dimension; Correspondingly, the step of calculating the semantic distance of the enhanced feature data in a preset dimension to obtain the original data distribution drift data includes: The enhanced feature data is subjected to global average pooling to obtain a domain-level mean feature vector; and L2 distance is calculated on the domain-level mean feature vector to obtain the global feature distribution distance corresponding to the global distribution dimension. The enhanced feature data is averaged according to the spatial structure dimension to obtain a two-dimensional spatial response map; the response value difference of the two-dimensional spatial response map is calculated and averaged to obtain the spatial structure distance corresponding to the spatial structure dimension. The average activation intensity difference of each channel in the enhanced feature data is calculated and averaged to obtain the channel statistical distance corresponding to the channel statistical dimension; The feature difference map of the enhanced feature data is weighted based on the spatial weight and the channel weight to obtain a weighted feature difference map; and the average of the weighted feature difference maps is taken to obtain the attention weighting distance corresponding to the attention weighting dimension; wherein, the feature difference map of the enhanced feature data is the element-wise difference between the source domain enhanced feature data and the target domain enhanced feature data; Based on preset weighting coefficients, the global feature distribution distance, the spatial structure distance, the channel statistical distance, and the attention weighted distance are weighted and fused to obtain the original data distribution drift data.

[0009] In some embodiments of this application, the step of performing ranking normalization and piecewise nonlinear stretching on the original data distribution drift data to generate corresponding image data distribution drift intensity data includes: The original data distribution drift data is added to a preset historical distance buffer pool and the mean and standard deviation are calculated to obtain the mean and standard deviation corresponding to the original data distribution drift data. The original data distribution drift data is used as the current sample distance. The standard deviation of the current sample distance is calculated based on the mean. The calculated standard deviation is then mapped based on a preset Sigmoid function to obtain the normalized composite distance corresponding to the current sample distance. The normalized composite distance is subjected to piecewise nonlinear stretching to generate the corresponding image data distribution drift intensity data.

[0010] In some embodiments of this application, the piecewise nonlinear stretching includes: if the normalized composite distance is less than a preset low threshold, then a power function with a preset low stretching coefficient greater than 1 is used for gentle polarization stretching to obtain the image data distribution drift intensity data; if the normalized composite distance is greater than a preset high threshold, then a power function with a preset high stretching coefficient less than 1 is used for stretching to obtain the image data distribution drift intensity data; if the preset low threshold is less than or equal to the normalized composite distance and less than or equal to the preset high threshold, then linear output is maintained to obtain the image data distribution drift intensity data.

[0011] Another aspect of this application provides an image data distribution drift intensity measurement system, the system comprising: A deep semantic feature extraction network module based on attention enhancement is used to perform semantic feature extraction and attention enhancement processing on the source domain image used to train the image semantic encoding and decoding model and the target domain image predicted by the image semantic encoding and decoding model, respectively, to obtain the enhanced feature data of the source domain image and the target domain image. The multidimensional semantic distance measurement module is used to calculate the semantic distance of the enhanced feature data in a preset dimension to obtain the original data distribution drift data; An enhanced data distribution drift measurement module is used to perform ranking normalization and piecewise nonlinear stretching on the original data distribution drift data to generate corresponding image data distribution drift intensity data for performance evaluation of the image semantic encoding and decoding model.

[0012] A third aspect of this application provides an electronic device including a processor and a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the image data distribution drift intensity measurement method.

[0013] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the image data distribution drift intensity measurement method.

[0014] The fifth aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, implements the image data distribution drift intensity measurement method.

[0015] This application presents an image data distribution drift intensity measurement method, comprising the following steps: employing a pre-defined deep semantic feature extraction network module based on attention enhancement, performing semantic feature extraction and attention enhancement processing on the source domain image used to train the image semantic encoding and decoding model and the target domain image predicted by the image semantic encoding and decoding model, respectively, to obtain enhanced feature data for the source domain image and the target domain image; calculating the semantic distance of the enhanced feature data in a pre-defined dimension to obtain the original data distribution drift data; and performing ranking normalization and piecewise nonlinear stretching on the original data distribution drift data to generate corresponding image data distribution drift intensity data for performance evaluation of the image semantic encoding and decoding model. This method can more effectively capture key semantic regions and information-rich feature channels in images, improving the discriminative power of feature representation; avoid the one-sidedness of a single indicator, improving the stability and reliability of the measurement results; and solve the problem of numerical scale differences caused by different datasets and feature extractors, enhancing cross-scene comparability and universality; it can more flexibly adapt to complex and changing edge environments, maintaining measurement stability during slight drifts and responding promptly during severe drifts, improving the accuracy and reliability of drift intensity quantification.

[0016] Additional advantages, objectives, and features of this application will be set forth in part in the description which follows, and will in part become apparent to those skilled in the art upon review of the following description, or may be learned by practice of the application. The objectives and other advantages of this application can be realized and obtained by means of the structures specifically pointed out in the specification and drawings.

[0017] Those skilled in the art will understand that the purposes and advantages that can be achieved with this application are not limited to those specifically described above, and that the above and other purposes that this application can achieve will be more clearly understood from the following detailed description. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, do not constitute a limitation thereof. The components in the drawings are not drawn to scale but are merely for illustrating the principles of this application. For ease of illustration and description of certain parts of this application, corresponding portions in the drawings may be enlarged, i.e., may appear larger relative to other components in an exemplary device actually manufactured according to this application. In the drawings: Figure 1 This is a schematic diagram of the first process of an image data distribution drift intensity measurement method in one embodiment of this application.

[0019] Figure 2 This is a schematic diagram of a second process for measuring the intensity of image data distribution drift in one embodiment of this application.

[0020] Figure 3 This is a schematic diagram of the third process of the image data distribution drift intensity measurement method in one embodiment of this application.

[0021] Figure 4 This is a schematic diagram of a system framework for a method for measuring the intensity of image data distribution drift, as illustrated in a specific example of this application.

[0022] Figure 5 This is a schematic diagram of a deep semantic feature extraction network module based on attention enhancement, which is a specific example of an image data distribution drift intensity measurement method in this application.

[0023] Figure 6 This is a flowchart illustrating a multidimensional semantic distance metric module for an image data distribution drift intensity measurement method, as illustrated in a specific example of this application.

[0024] Figure 7 This is a flowchart illustrating an enhanced data distribution drift measurement module for an image data distribution drift intensity measurement method, as described in a specific example of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain this application, but are not intended to limit it.

[0026] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the structures and / or processing steps closely related to the solution according to this application are shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0027] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0028] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0029] In the following description, embodiments of the present application will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0030] It's important to note that traditional methods for measuring data distribution drift mainly fall into three categories: statistical hypothesis testing, distance metrics, and simple machine learning detection. Statistical hypothesis testing methods, such as KL divergence (Kullback-Leibler divergence, a method for measuring the difference between two probability distributions) and JS divergence (Jensen-Shannon divergence, a smoothed version of KL divergence, lying between the two distributions and being symmetrical), rely on accurate probability density estimation, resulting in large estimation errors on high-dimensional features and sensitivity to intervals with zero probability. Distance metrics, such as L2 distance (Euclidean distance, calculating the straight-line distance between two points), cosine similarity (a similarity measure based on the angle between vectors), and MMD (maximum mean difference, used to determine whether two distributions are identical), have unbounded numerical ranges or high computational complexity, making it difficult to set universal thresholds and insufficiently characterizing the geometric structure of high-dimensional features. Simple machine learning detection methods, such as adversarial verification, require training an auxiliary discriminator, incurring high computational costs and being prone to overfitting on specific datasets. These traditional methods share challenges such as low semantic sensitivity, inability to capture changes in image content across spatial structure and channel response, poor noise robustness, lack of consideration for the relative position of historical sample distribution, and inability to establish a stable quantitative correlation with model performance degradation rate. These challenges make it difficult to meet the practical needs of semantic communication systems for quantitative evaluation of drift intensity. Semantic communication-based transmission extracts the semantic information of the image to be sent from the source, rather than transmitting the bitstream of the message. Under the same bandwidth conditions, it can transmit more meaningful information, effectively compressing data redundancy, improving the effectiveness of information transmission, reducing network transmission pressure, and lowering the processing latency of intelligent tasks. Compared with traditional networks, semantic communication can better meet the needs in situations with poor communication channel quality and complex data distribution. Based on data distribution drift measurement using deep feature distance, the system first inputs images from the source domain (training data) and the target domain (real-time data) into a pre-trained feature extraction network. The network extracts shallow texture features and deep semantic features of the image layer by layer through multiple convolutional and pooling layers, outputting a fixed-dimensional feature vector. Subsequently, the system calculates the distance between the extracted source and target domain feature sets. Commonly used distance metrics include L2 Euclidean distance, cosine similarity, or Manhattan distance, calculating the distance between the centroids of the two feature sets or the average distance between all sample pairs. The raw data after distance calculation can be directly output as drift intensity values, or output after simple linear weighted fusion. The entire feature extraction and distance calculation process mainly relies on traditional convolutional neural network structures and basic vector distance calculation modules, lacking differentiated processing of the spatial structure and channel importance of semantic features. In edge intelligence and semantic communication systems, the performance of image semantic encoding and decoding models depends on the consistency between the distribution of training data and real-time data.However, in dynamic scenarios such as connected vehicles and industrial vision, real-time acquired image data can experience distribution drift due to changes in environment, lighting, viewpoint, or task objectives, leading to a decrease in the reconstruction quality of the model in the target domain. Image transmission in a semantic communication-based image transmission system comprises six steps: image input, semantic feature extraction, source-channel joint coding, channel transmission, source-channel joint decoding, and semantic feature reconstruction. Semantic feature extraction maps the input image to low-dimensional latent features. Source-channel joint coding consists of two steps: semantic coding and channel coding. The purpose of semantic coding is to extract deep semantic information from the image, remove spatial redundancy, and improve transmission efficiency. Channel coding requires adding check bits to the encoded features to achieve error detection and correction, thereby increasing the reliability of feature transmission in noisy channels. The receiving end recovers the reconstructed image from the received semantic features through source-channel joint decoding. Therefore, existing methods using direct L2 distance or simple weighted fusion are significantly insufficient when facing complex and variable edge environments in semantic communication systems. The unbounded range of L2 distance values ​​in deep feature spaces makes it difficult to set a universal threshold to judge the severity of drift. Distance values ​​generated by different datasets or feature extractors vary significantly in scale, lacking comparability across different scenarios. Existing methods are insensitive to semantic changes in image content across dimensions such as spatial structure and channel response, resulting in large differences in model performance degradation rates even with similar drift intensities. Existing methods do not consider the relative position of samples in historical distributions and lack the ability to suppress sample-specific noise; outliers in a single sample can significantly affect the measurement results. Existing methods use linear distance calculations, which cannot characterize the robustness of neural networks to slight drifts or their collapse characteristics under severe drifts. Furthermore, this simple weighted fusion method results in a scattered numerical distribution in the feature space, and fixed threshold judgments may not accurately reflect the true semantic distribution differences in some cases. Edge intelligence and semantic communication are emerging, and traditional bit-based communication methods cannot meet the demand for efficient transmission. Semantic communication focuses on the semantic understanding and transmission of image content, improving transmission efficiency and user experience, but it faces the challenge of data distribution drift in dynamic environments, requiring accurate drift measurement methods to support model maintenance. Based on this, the inventors of this application first conceived of an image data distribution drift intensity measurement method that, by leveraging the semantic feature differences between the source and target domains through attention-enhanced feature extraction, multi-dimensional distance calculation, ranking normalization, and piecewise nonlinear stretching, can generate a stable and robust drift intensity scalar. This allows for the quantification of the degree of data distribution drift, improving the accuracy and reliability of model performance evaluation in semantic communication systems. Compared to traditional fixed threshold or simple distance calculation methods, this application can more flexibly adapt to complex and changing edge environments, maintaining metric stability during slight drifts and responding promptly during severe drifts, thus improving the accuracy and reliability of drift intensity quantification.In other words, the method provided in this application can robustly and accurately quantify the intensity of distribution differences between the source and target domains from the semantic level of images, providing a stable metric input for the update decision of the image semantic model.

[0031] The following examples will provide a detailed description.

[0032] This application provides a method for measuring the intensity of image data distribution drift. See also... Figure 1 The method includes the following steps: Step 100: Using a preset deep semantic feature extraction network module based on attention enhancement, semantic feature extraction and attention enhancement processing are performed on the source domain image used to train the image semantic encoding and decoding model and the target domain image predicted by the image semantic encoding and decoding model, respectively, to obtain the enhanced feature data of the source domain image and the target domain image. In step 100, the source domain image and the target domain image are input into the deep semantic feature extraction network module based on attention enhancement. Multi-scale semantic features are extracted by the deep network encoder and then weighted and enhanced by the attention module to generate the enhanced feature data of the source domain image and the target domain image respectively.

[0033] Step 200: Calculate the semantic distance of the enhanced feature data in a preset dimension to obtain the original data distribution drift data; In step 200, the original data distribution drift data can be a preliminary scalar value that quantifies the difference in feature distribution between the source and target domains. The semantic distance of the enhanced feature data can be calculated from four complementary perspectives: global distribution, spatial structure, channel statistics, and attention weighting. These are then weighted and fused to obtain the original data distribution drift data.

[0034] Step 300: Perform ranking normalization and piecewise nonlinear stretching on the original data distribution drift data to generate corresponding image data distribution drift intensity data for performance evaluation of the image semantic encoding and decoding model.

[0035] In step 300, the image data distribution drift intensity data can be a stable scalar obtained after normalization and nonlinear adjustment, and its value range is mapped to... An interval is used to quantify the semantic distribution offset between the source and target domains; it is a normalized scalar value ranging from 0 to 1. The original data distribution drift data is normalized based on drift ranking and subjected to piecewise nonlinear stretching to generate normalized image data distribution drift intensity data with nonlinear robustness, used to evaluate the degree of model performance degradation. The obtained image data distribution drift intensity data is then used to construct a correlation model for the performance degradation rate of the semantic encoding / decoding model.

[0036] In one or more embodiments of this application, the performance of the image semantic encoding and decoding model is evaluated using image data distribution drift intensity data, and the model is maintained (fine-tuned, retrained with replacement training data, etc.) based on the evaluation results, so as to improve the accuracy and reliability of image semantic recognition or prediction (such as face recognition) using the model.

[0037] As described above, the image data distribution drift intensity measurement method provided in this application, through attention-enhanced deep semantic feature extraction, can more effectively capture key semantic regions and information-rich feature channels in images, improve the discriminativeness of feature expression, and thus support more accurate distribution difference quantification. Furthermore, by employing a multi-dimensional complementary semantic distance measurement strategy, it can characterize the distribution differences between the source and target domains from four perspectives: global distribution, spatial structure, channel statistics, and attention weighting, avoiding the one-sidedness of a single indicator and improving the stability and reliability of the measurement results. Moreover, by using a drift ranking-based normalization method to map the original unbounded distance to a bounded interval, it can solve the problem of numerical scale differences generated by different datasets and feature extractors, enhancing cross-scene comparability and universality. In addition, it employs a segment nonlinear stretching mechanism, which can simulate the robustness of semantic encoding / decoding models to slight drifts and the collapse characteristics of severe drifts, making the measurement results consistent with the real model performance response patterns, and improving the semantic sensitivity and practicality of the measurement results.

[0038] In order to further capture key semantic regions and information-rich feature channels in images, improve the discriminativeness of feature representation, and thus support more accurate distribution difference quantification, in the image data distribution drift intensity measurement method provided in this application embodiment, the deep semantic feature extraction network module based on attention enhancement includes a deep semantic feature extraction network unit, a spatial attention sub-network unit, a channel attention sub-network unit, and a fusion unit. The deep semantic feature extraction network unit is used to receive the source domain image and the target domain image, and perform multi-scale semantic feature extraction on the source domain image and the target domain image to obtain the original feature data of the source domain image corresponding to the source domain image and the original feature data of the target domain image corresponding to the target domain image, and transmit the original feature data of the source domain image and the original feature data of the target domain image to the spatial attention sub-network unit and the channel attention sub-network unit respectively; The spatial attention sub-network unit is used to receive the original feature data of the source domain image and perform weight calculation on the original feature data of the source domain image to obtain the spatial weights corresponding to the original feature data of the source domain image. The channel attention sub-network unit is used to receive the original feature data of the target domain image and perform weight calculation on the original feature data of the target domain image to obtain the channel weights corresponding to the original feature data of the target domain image. The fusion unit is used to fuse the enhanced feature data of the source domain image and the target domain image respectively based on the spatial weight, the channel weight, the original feature data of the source domain image and the original feature data of the target domain image.

[0039] In one or more embodiments of this application, the original feature data of the source domain image can be an unweighted deep semantic feature map from the source domain image; the original feature data of the target domain image can be an unweighted deep semantic feature map extracted from the target domain image; the spatial weight can be a spatial attention weight map generated by a spatial attention sub-network, used to represent key semantic regions in the image; the channel weight can be a channel attention weight vector generated by a channel attention sub-network, used to filter information-rich feature channels. The deep semantic feature extraction network unit extracts deep semantic features from the input image to obtain the original feature data of the source domain image and the original feature data of the target domain image; the inter-attention sub-network unit and the channel attention sub-network unit adaptively weight the key semantic regions and information-rich feature channels respectively through an attention mechanism, and then fuse them through the fusion unit to generate the enhanced feature data of the source domain image and the target domain image with strong discriminative power. The deep semantic feature extraction network module based on attention enhancement is deployed at the receiving end to process the source domain image and the target domain image.

[0040] To further capture key semantic regions and information-rich feature channels in images, improve the discriminative power of feature representation, and thus support more accurate distribution difference quantification, in an image data distribution drift intensity measurement method provided in this application embodiment, the fusion unit includes: The normalization subunit is used to normalize the spatial weights and the channel weights to obtain normalized spatial weights and normalized channel weights. The dot product fusion subunit is used to perform dot product fusion on the normalized spatial weights and the original feature data of the source domain image, and the normalized channel weights and the original feature data of the target domain image, respectively, to obtain the enhanced feature data of the source domain image and the target domain image; wherein, the enhanced feature data includes enhanced feature data of the source domain image and enhanced feature data of the target domain image.

[0041] In one or more embodiments of this application, the deep semantic feature extraction network module with attention enhancement performs semantic feature extraction and attention enhancement on source and target domain images. The extracted reference features are the original feature data of the source image and the original feature data of the target image, respectively. The fusion unit in this module adaptively weights the original feature data of the source and target images using spatial attention (generated by the network) and channel attention (generated by the network), respectively, and then normalizes them using a sigmoid activation function to generate the enhanced feature data for each of the source and target images.

[0042] To further avoid the one-sidedness of a single indicator and improve the stability and reliability of the measurement results, an image data distribution drift intensity measurement method is provided in this application embodiment, see [link to relevant documentation]. Figure 2 The preset dimensions include global distribution dimension, spatial structure dimension, channel statistics dimension, and attention weighting dimension; Correspondingly, step 200 includes: Step 210: Perform global average pooling on the enhanced feature data to obtain a domain-level mean feature vector; and perform L2 distance calculation on the domain-level mean feature vector to obtain the global feature distribution distance corresponding to the global distribution dimension; In step 210, the domain-level mean feature vector can be a C-dimensional feature vector generated after global flat pooling, used to characterize the global distribution information of the entire image domain; the global feature distribution distance can be the L2 distance between the domain-level mean feature vectors of the source domain and the target domain.

[0043] Step 220: Perform spatial structure dimension averaging on the enhanced feature data to obtain a two-dimensional spatial response map; calculate the response value difference of the two-dimensional spatial response map and take the average to obtain the spatial structure distance corresponding to the spatial structure dimension; In step 220, the two-dimensional spatial response map can be a spatial response map of size H×W generated by averaging the enhanced feature data in the channel dimension, used to characterize the overall response intensity of each spatial location in the image; the spatial structure distance can be the response difference value between the enhanced feature data of the source domain and the target domain in the spatial dimension, used to quantify the degree of change in the spatial layout of objects such as position and shape in the image.

[0044] Step 230: Calculate the average activation intensity difference of each channel in the enhanced feature data and take the average to obtain the channel statistical distance corresponding to the channel statistical dimension; In step 230, the channel statistical distance can be the average activation intensity difference value of the enhanced feature data of the source domain and the target domain in each channel, which is used to quantify the degree of distribution difference of the image in channel features such as texture and color.

[0045] Step 240: Based on the spatial weight and the channel weight, perform weighted processing on the feature difference map of the enhanced feature data to obtain a weighted feature difference map; and take the average of the weighted feature difference maps to obtain the attention weighting distance corresponding to the attention weighting dimension; wherein, the feature difference map of the enhanced feature data is the element-wise difference between the source domain enhanced feature data and the target domain enhanced feature data; The attention-weighted distance mentioned in step 240 can be the feature difference value between the source domain and the target domain after attention weighting, which is used to quantify the distribution offset of key semantic regions in the image.

[0046] Step 250: Based on preset weight coefficients, the global feature distribution distance, the spatial structure distance, the channel statistical distance, and the attention weighted distance are weighted and fused to obtain the original data distribution drift data.

[0047] In step 250, the original data distribution drift data can affect the reconstruction accuracy of the semantic encoding and decoding model. It can be represented as: in, , , , These represent the global feature distribution distance, spatial structure distance, channel statistical distance, and attention-weighted distance, respectively. , , , These are the weighting coefficients. + + + =1.

[0048] In one or more embodiments of this application, if the global feature distribution distance is calculated... Then global average pooling is performed on the enhanced features. Then, the L2 distance is calculated to obtain the global feature distribution distance; if the spatial structure distance is calculated... Then, after averaging in the spatial dimension, the differences are calculated pixel by pixel and averaged to obtain the spatial structure distance; if calculating the channel statistical distance... Then, each channel is treated as an independent distribution, the average activation difference is calculated, and the average is taken to obtain the statistical distance of the channel; if attention-weighted distance is calculated... Then use spatial weights. and channel weight The average value is weighted over the difference map, and the Frobenius norm (a norm that measures the size of a matrix or the overall size) is calculated and averaged to obtain the attention-weighted distance; finally, the four distance indicators are combined. , , , With weighting coefficients , , , The weighted fusion yields the original data distribution drift data. Output later.

[0049] To further address the issue of numerical scale discrepancies arising from different datasets and feature extractors, and to enhance comparability and universality across different scenarios, this application provides an image data distribution drift intensity measurement method, see [link to relevant documentation]. Figure 3 Step 300 includes: Step 310: Add the original data distribution drift data to a preset historical distance buffer pool and calculate the mean and standard deviation to obtain the mean and standard deviation corresponding to the original data distribution drift data; In step 310, the historical distance buffer pool can be a dynamic collection storing historical raw data distribution drift data, used to provide a statistical benchmark for the current raw data distribution drift data. Raw data distribution drift data is added to the historical distance buffer pool, and the mean and standard deviation of each distance in the historical data are calculated in real time to update the statistical parameters.

[0050] Step 320: Using the original data distribution drift data as the current sample distance, calculate the standard deviation of the current sample distance based on the mean, and map the calculated standard deviation based on the preset Sigmoid function to obtain the normalized composite distance corresponding to the current sample distance; In step 320, the preset Sigmoid function can be expressed as: a nonlinear function that maps the standard deviation of the current sample distance to the [0,1] interval, and its output is the normalized distance value corresponding to each dimension of distance, which is used for subsequent weighted fusion of multi-dimensional distances to obtain the normalized comprehensive distance value. If the standard deviation is to be calculated, the standard deviation of the current sample distance from the mean is calculated and mapped to the zero-one interval through the Sigmoid function with a scaling factor to obtain the normalized distance, and the normalized comprehensive distance value is obtained after weighted fusion.

[0051] Step 330: Perform piecewise nonlinear stretching on the normalized composite distance to generate the corresponding image data distribution drift intensity data.

[0052] To further simulate the robustness of semantic encoding / decoding models to slight drifts and the collapse characteristics of severe drifts, and to make the measurement results consistent with the actual model performance response patterns, thereby improving the semantic sensitivity and practicality of the measurement results, an image data distribution drift intensity measurement method provided in this application embodiment includes a piecewise nonlinear stretching method: if the normalized synthesis distance is less than a preset low threshold, then a power function with a preset low stretching coefficient greater than 1 is used for gentle polarization stretching to obtain the image data distribution drift intensity data; if the normalized synthesis distance is greater than a preset high threshold, then a power function with a preset high stretching coefficient less than 1 is used for stretching to obtain the image data distribution drift intensity data; if the preset low threshold is less than or equal to the normalized synthesis distance and less than or equal to the preset high threshold, then linear output is maintained to obtain the image data distribution drift intensity data.

[0053] In one or more embodiments of this application, the preset low threshold can be a critical normalized composite distance value used to distinguish between slight drift and moderate drift, and can be 0.2; the preset high threshold can be a critical normalized composite distance value used to distinguish between moderate drift and severe drift, and can be 0.5; mild polarization stretching refers to nonlinearly adjusting the normalized composite distance through a power function, using a power exponent greater than 1 in the low drift range to suppress small fluctuations, and using a power exponent less than 1 in the high drift range to amplify severe drift features, thereby enhancing the sensitivity of the measurement results to semantic drift.

[0054] This application embodiment also provides an image data distribution drift intensity measurement system, the system comprising: The deep semantic feature extraction network module 10 based on attention enhancement is used to perform semantic feature extraction and attention enhancement processing on the source domain image used to train the image semantic encoding and decoding model and the target domain image predicted by the image semantic encoding and decoding model, respectively, to obtain the enhanced feature data of the source domain image and the target domain image. The multidimensional semantic distance measurement module 20 is used to calculate the semantic distance of the enhanced feature data in a preset dimension to obtain the original data distribution drift data; The enhanced data distribution drift measurement module 30 is used to perform ranking normalization and piecewise nonlinear stretching on the original data distribution drift data to generate corresponding image data distribution drift intensity data for performance evaluation of the image semantic encoding and decoding model.

[0055] In one or more embodiments of this application, the attention-enhanced deep semantic feature extraction network module 10, the multidimensional semantic distance metric module 20, and the enhanced data distribution drift metric module 30 are all deployed at the receiving end.

[0056] In a specific example of the image data distribution drift intensity measurement method provided in this application, the system framework of the method is as follows: Figure 4 As shown, the method includes the following steps: The system inputs source and target domain images into an attention-enhanced deep semantic feature extraction network, extracts multi-scale semantic features through a deep network encoder, and generates enhanced features through weighted enhancement by an attention module. The system will input enhanced features into the multidimensional semantic distance measurement module, calculate semantic distance from four complementary perspectives: global distribution, spatial structure, channel statistics, and attention weighting, and then weighted and fused to obtain the original drift metric. The system inputs the original drift metric into the enhanced data distribution drift metric module, performs normalization based on drift ranking and mild polarization adjustment, and generates a normalized enhanced drift intensity scalar with nonlinear robustness perception capability, which is used to evaluate the degree of model performance degradation.

[0057] In this embodiment, the specific implementation of the image data distribution drift intensity measurement method mainly relies on three core modules: an attention-enhanced deep semantic feature extraction network, a multidimensional semantic distance measurement module, and an enhanced data distribution drift measurement module.

[0058] The deep semantic feature extraction network module based on attention enhancement extracts deep semantic features from the input image and adaptively weights key semantic regions and information-rich feature channels through an attention mechanism to generate highly discriminative enhanced feature representations. This module is deployed at the receiver and processes two types of input: the source domain image is a reference image pre-stored during the training phase, and the target domain image is the real-time received test image. This module processes the source domain image... and target domain image Semantic feature extraction and attention enhancement are performed, and the extracted reference features are as follows: and The two types of features are then processed by a spatial attention network to generate spatial weights. Channel attention and channel weights generated by the network Adaptive weighting is applied, followed by the Sigmoid activation function. Enhanced features are generated after normalization. and Output. The specific algorithm for this module is shown in Table 1, and the flowchart is as follows. Figure 5 As shown.

[0059] Table 1. Detailed algorithm description of the attention-enhanced image deep semantic feature extraction network module.

[0060] The multidimensional semantic distance metric module quantifies the distributional differences between source and target domain augmented features from four complementary aspects: global distribution, spatial structure, channel design, and attention weighting, generating a raw drift metric. Deployed at the receiver, this module calculates the semantic distance between the target and source domain images in real time. This module also evaluates the source domain augmented features. and target domain enhancement features To perform multidimensional semantic distance calculation, if we calculate the global feature distribution distance... Then global average pooling is performed on the enhanced features. Then calculate the L2 distance, if calculating the spatial structure distance. Then, after averaging in the spatial dimension, the differences are calculated pixel by pixel and averaged again. If channel statistical distance is calculated... Then, treating each channel as an independent distribution, the average activation difference is calculated and averaged. If attention-weighted distance is calculated... Then use spatial weights and channel weight The average value is weighted onto the difference plot, the Frobenius norm is calculated, and the average is taken. Finally, the four distance indicators are... , , , With weighting coefficients , , , Weighted fusion yields the original drift metric. The output is then displayed. The specific algorithm for this module is shown in Table 2, and the flowchart is as follows: Figure 6 As shown.

[0061] Table 2 describes the specific algorithm of the multidimensional semantic distance metric module for images.

[0062] The enhanced data distribution drift metric module performs ranking normalization and mild polarization adjustment on the original drift metric to generate a normalized, non-linearly robust, and enhanced drift intensity scalar. Deployed at the receiver, this module outputs the drift intensity scalar to assess the performance degradation of the current semantic encoding / decoding model, providing a quantitative basis for model updates. This module optimizes the original drift metric. To perform ranking normalization and mild polarization adjustment, firstly... Add to historical distance buffer pool Calculate the mean and standard deviation If we calculate the standard deviation Then calculate the current distance. Deviation from the mean The standard deviation and scaled by the Sigmoid function. Mapping to the zero-one interval yields the normalized distance, which is then weighted and fused to obtain the comprehensive distance value. .like Below the low threshold Then use a low elongation coefficient Power functions greater than one undergo gentle polarization stretching; if the value exceeds a high threshold... Then with a high elongation coefficient Power functions less than one undergo gentle polarization stretching; if they lie between two thresholds, they maintain linear output, ultimately generating an enhanced drift intensity scalar. The output is then processed. The specific algorithm for this module is shown in Table 3, and the flowchart is as follows: Figure 7 As shown in the figure.

[0063] Table 3. Detailed algorithm description of the enhanced data distribution drift measurement module for images.

[0064] This application also provides an electronic device, which may include a processor, a memory, a receiver, and a transmitter. The processor is used to execute the image data distribution drift intensity measurement method mentioned in the above embodiments. The processor and the memory can be connected via a bus or other means, taking a bus connection as an example. The receiver can be connected to the processor and the memory via wired or wireless means.

[0065] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0066] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the image data distribution drift intensity measurement method described in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the image data distribution drift intensity measurement method described in the above method embodiments.

[0067] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0068] The one or more modules are stored in the memory, and when executed by the processor, the image data distribution drift intensity measurement method described in the embodiment is executed.

[0069] In some embodiments of this application, the user equipment may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected via a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0070] As one implementation method, the functions of the receiver and transmitter in this application can be implemented by transceiver circuits or dedicated transceiver chips, and the processor can be implemented by dedicated processing chips, processing circuits or general-purpose chips.

[0071] As another implementation approach, the server provided in this application embodiment can be implemented using a general-purpose computer. That is, the program code implementing the processor, receiver, and transmitter functions is stored in memory, and the general-purpose processor implements the processor, receiver, and transmitter functions by executing the code in memory.

[0072] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned image data distribution drift intensity measurement method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0073] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned image data distribution drift intensity measurement method.

[0074] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave.

[0075] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0076] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0077] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to the embodiments of this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for measuring the intensity of image data distribution drift, characterized in that, The method includes: A pre-defined deep semantic feature extraction network module based on attention enhancement is used to perform semantic feature extraction and attention enhancement processing on the source domain image used to train the image semantic encoding and decoding model and the target domain image predicted by the image semantic encoding and decoding model, respectively, to obtain the enhanced feature data of the source domain image and the target domain image. The enhanced feature data is subjected to semantic distance calculation of a preset dimension to obtain the original data distribution drift data; The original data distribution drift data is ranked, normalized, and piecewise nonlinearly stretched to generate corresponding image data distribution drift intensity data for performance evaluation of the image semantic encoding and decoding model.

2. The method according to claim 1, characterized in that, The attention-enhanced deep semantic feature extraction network module includes a deep semantic feature extraction network unit, a spatial attention sub-network unit, a channel attention sub-network unit, and a fusion unit; The deep semantic feature extraction network unit is used to receive the source domain image and the target domain image, and perform multi-scale semantic feature extraction on the source domain image and the target domain image to obtain the original feature data of the source domain image corresponding to the source domain image and the original feature data of the target domain image corresponding to the target domain image, and transmit the original feature data of the source domain image and the original feature data of the target domain image to the spatial attention sub-network unit and the channel attention sub-network unit respectively; The spatial attention sub-network unit is used to receive the original feature data of the source domain image and perform weight calculation on the original feature data of the source domain image to obtain the spatial weights corresponding to the original feature data of the source domain image. The channel attention sub-network unit is used to receive the original feature data of the target domain image and perform weight calculation on the original feature data of the target domain image to obtain the channel weights corresponding to the original feature data of the target domain image. The fusion unit is used to fuse the enhanced feature data of the source domain image and the target domain image respectively based on the spatial weight, the channel weight, the original feature data of the source domain image and the original feature data of the target domain image.

3. The method according to claim 2, characterized in that, The fusion unit includes: The normalization subunit is used to normalize the spatial weights and the channel weights to obtain normalized spatial weights and normalized channel weights. The dot product fusion subunit is used to perform dot product fusion on the normalized spatial weights and the original feature data of the source domain image, and the normalized channel weights and the original feature data of the target domain image, respectively, to obtain the enhanced feature data of the source domain image and the target domain image; wherein, the enhanced feature data includes enhanced feature data of the source domain image and enhanced feature data of the target domain image.

4. The method according to claim 3, characterized in that, The preset dimensions include global distribution dimension, spatial structure dimension, channel statistics dimension, and attention weighting dimension; Correspondingly, the step of calculating the semantic distance of the enhanced feature data in a preset dimension to obtain the original data distribution drift data includes: The enhanced feature data is subjected to global average pooling to obtain a domain-level mean feature vector; and L2 distance is calculated on the domain-level mean feature vector to obtain the global feature distribution distance corresponding to the global distribution dimension. The enhanced feature data is averaged according to the spatial structure dimension to obtain a two-dimensional spatial response map; the response value difference of the two-dimensional spatial response map is calculated and averaged to obtain the spatial structure distance corresponding to the spatial structure dimension. The average activation intensity difference of each channel in the enhanced feature data is calculated and averaged to obtain the channel statistical distance corresponding to the channel statistical dimension; The feature difference map of the enhanced feature data is weighted based on the spatial weight and the channel weight to obtain a weighted feature difference map; and the average of the weighted feature difference maps is taken to obtain the attention weighting distance corresponding to the attention weighting dimension; wherein, the feature difference map of the enhanced feature data is the element-wise difference between the source domain enhanced feature data and the target domain enhanced feature data; Based on preset weighting coefficients, the global feature distribution distance, the spatial structure distance, the channel statistical distance, and the attention weighted distance are weighted and fused to obtain the original data distribution drift data.

5. The method according to claim 1, characterized in that, The step of ranking and normalizing the original data distribution drift data and performing piecewise nonlinear stretching to generate corresponding image data distribution drift intensity data includes: The original data distribution drift data is added to a preset historical distance buffer pool and the mean and standard deviation are calculated to obtain the mean and standard deviation corresponding to the original data distribution drift data. The original data distribution drift data is used as the current sample distance. The standard deviation of the current sample distance is calculated based on the mean. The calculated standard deviation is then mapped based on a preset Sigmoid function to obtain the normalized composite distance corresponding to the current sample distance. The normalized composite distance is subjected to piecewise nonlinear stretching to generate the corresponding image data distribution drift intensity data.

6. The method according to claim 5, characterized in that, The piecewise nonlinear stretching includes: if the normalized composite distance is less than a preset low threshold, then a power function with a preset low stretching coefficient greater than 1 is used for gentle polarization stretching to obtain the image data distribution drift intensity data; if the normalized composite distance is greater than a preset high threshold, then a power function with a preset high stretching coefficient less than 1 is used for stretching to obtain the image data distribution drift intensity data; if the preset low threshold is less than or equal to the normalized composite distance and less than or equal to the preset high threshold, then linear output is maintained to obtain the image data distribution drift intensity data.

7. A system for measuring the intensity of image data distribution drift, characterized in that, The system includes: A deep semantic feature extraction network module based on attention enhancement is used to perform semantic feature extraction and attention enhancement processing on the source domain image used to train the image semantic encoding and decoding model and the target domain image predicted by the image semantic encoding and decoding model, respectively, to obtain the enhanced feature data of the source domain image and the target domain image. The multidimensional semantic distance measurement module is used to calculate the semantic distance of the enhanced feature data in a preset dimension to obtain the original data distribution drift data; An enhanced data distribution drift measurement module is used to perform ranking normalization and piecewise nonlinear stretching on the original data distribution drift data to generate corresponding image data distribution drift intensity data for performance evaluation of the image semantic encoding and decoding model.

8. An electronic device, characterized in that, It includes a processor and a memory; when the processor executes the running program stored in the memory, it implements the image data distribution drift intensity measurement method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the image data distribution drift intensity measurement method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the image data distribution drift intensity measurement method according to any one of claims 1 to 6.

Images