Eye movement fixation duration weighted natural image roi division method and system

Abstract

This invention relates to the field of image processing technology, specifically disclosing a method and system for natural image ROI segmentation based on eye-tracking gaze duration weighting. Addressing the discrepancy between the segmentation results of traditional methods and the actual attention regions, this method extracts effective gaze points and their gaze durations through eye-tracking data preprocessing. A gaze duration-weighted heatmap is constructed using density peak clustering and a gaze duration weighting model, effectively enhancing the feature representation of high-attention regions. By fusing the eye-tracking heatmap with multimodal semantic features such as color, texture, and edge, the semantic consistency of ROI segmentation is significantly improved. Furthermore, by introducing image complexity adaptive adjustment of the segmentation threshold, flexible adaptation to the segmentation needs of images with varying complexity is achieved. Experimental results demonstrate that the performance of this method and system is significantly superior to traditional methods, thus providing solid theoretical and practical support for intelligent ROI segmentation of high-resolution natural images oriented towards visual attention.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for natural image ROI segmentation based on eye-tracking fixation duration weighting. Background Technology

[0002] In the fields of computer vision, image processing, and eye-tracking research, a "natural image" typically refers to a real-world scene image directly captured by optical devices (such as cameras or webcams), rather than a computer-generated composite image, pure graphics, charts, text documents, or heavily abstracted visual content. In digital image applications, the Region of Interest (ROI) carries the core semantic information of the image, and its accurate segmentation directly determines the efficiency and performance of computer vision tasks. In e-commerce product retrieval, efficient ROI segmentation can focus on the subject, suppress noise, and improve retrieval accuracy; in autonomous driving perception systems, quickly locating key ROIs such as pedestrians is a core prerequisite for ensuring real-time decision-making and safety.

[0003] Existing methods for ROI segmentation in natural images can be mainly classified into the following categories:

[0004] 1. Low-level visual feature-based methods: These methods extract low-level features of an image, such as color, texture, and edges, and use clustering or segmentation algorithms to determine the Region of Interest (ROI). For example, SIFT feature points are used for clustering, or the K-means algorithm is used for color space segmentation. However, these methods are highly dependent on the objective properties of the image, making it difficult to match human subjective visual preferences, and are easily affected by factors such as changes in lighting and cluttered backgrounds, leading to unstable segmentation results.

[0005] 2. Saliency Detection-Based Methods: Represented by the classic ITTI model, these methods simulate human visual attention mechanisms, fusing multi-scale features to generate saliency maps, and then segment ROIs. However, these methods lack adaptability to individual differences in visual attention when facing low-contrast and complex scenes, and their fixed-threshold segmentation strategy is also difficult to cope with the diversity of image content, resulting in limited segmentation accuracy.

[0006] 3. Deep learning-based methods: These methods utilize convolutional neural networks (CNNs) to automatically learn high-level semantic features of images, achieving end-to-end ROI partitioning. However, deep models typically rely on large-scale, high-quality labeled data for training, resulting in insufficient generalization ability in small-sample natural image scenarios, and high training costs.

[0007] To more accurately capture human visual attention, some studies have attempted to incorporate eye-tracking data. Eye-tracking data (including fixation points, fixation duration, and saccade paths) is a direct physiological mapping of visual attention, accurately depicting the observer's level of attention to different regions of an image. Existing research combining eye-tracking data, such as clustering methods based on fixation point spatial density, often focuses only on the spatial distribution of fixation points, neglecting the crucial temporal information of "fixation duration," which reflects the weight of attention intensity. Furthermore, existing methods generally employ fixed-threshold segmentation strategies, failing to dynamically adjust according to the semantic complexity of the image itself, easily leading to over-segmentation of ROIs or semantic loss.

[0008] In summary, existing ROI segmentation techniques have the following shortcomings: traditional methods lack consideration of human subjective visual attention; eye-tracking-assisted methods fail to fully utilize the attention intensity information contained in fixation duration; and segmentation strategies lack adaptability, making it difficult to achieve high-precision and highly semantically consistent ROI segmentation in complex and diverse natural images. Summary of the Invention

[0009] This invention provides a method and system for ROI segmentation of natural images based on eye-tracking fixation duration weighting. The technical problem it solves is how to achieve high-precision and highly semantically consistent ROI segmentation in complex and diverse natural images.

[0010] To address the above technical problems, this invention provides a method for natural image ROI segmentation weighted by eye-tracking fixation duration, comprising the following steps:

[0011] S1. Clean and denoise the raw eye movement data to obtain effective fixation point data, and extract the coordinates and fixation duration of the effective fixation points from the effective fixation point data;

[0012] S2. Generate a gaze duration-weighted heatmap reflecting the distribution of human visual attention based on the coordinates and gaze duration of the extracted effective gaze points;

[0013] S3. Extract the color, texture, and semantic edge features of the natural image corresponding to the original eye-tracking data, and perform weighted fusion with the gaze duration weighted heatmap to obtain a fused feature map;

[0014] S4. The fused feature map is segmented using an adaptive thresholding algorithm based on image complexity to obtain a binary image; the binary image is then subjected to ROI post-processing to obtain the ROI partitioning result.

[0015] Further, in step S1, the extraction of the coordinates and fixation duration of the effective fixation points from the effective fixation point data specifically includes:

[0016] Based on the velocity threshold method, when the eye movement velocity is lower than the preset velocity and the dwell time exceeds the preset time, it is determined to be a valid fixation point, and the coordinates of the fixation point are extracted.

[0017] The fixation duration for each effective fixation point is calculated, which is the time interval between two adjacent saccades, to obtain the fixation duration sequence.

[0018] Furthermore, step S2 specifically includes:

[0019] Spatial clustering of effective gaze points is performed to identify regions of attention concentration, resulting in multiple clusters.

[0020] Calculate the average gaze duration for each cluster;

[0021] The average fixation duration of each cluster is used as the attention weight for that cluster to calculate the attention value for each effective fixation point;

[0022] Normalize the attention value of each effective fixation point to The interval is used to obtain a gaze duration-weighted heatmap.

[0023] Furthermore, regarding effective fixation points Its attention value is calculated as follows:

[0024] ,

[0025] in, For the first Clusters The center coordinates, For Gaussian kernel function, For kernel width; For the first Clusters Average fixation duration This represents the number of clusters.

[0026] Further, in step S3, the color, texture, and semantic edge features of the natural image corresponding to the original eye-tracking data are extracted, specifically as follows:

[0027] The natural image is converted to the HSV space, the brightness and saturation channels are fused, a color feature map is generated and normalized;

[0028] Local binary mode is used to extract texture features from natural images, resulting in texture feature maps that are then normalized.

[0029] The semantic edges of natural images are extracted based on the Canny algorithm, and morphological closing operations are used to fill the edge gaps, retaining weak edge information, resulting in a semantic edge map that is then normalized.

[0030] Further, in step S4, the fused feature map is segmented using an adaptive thresholding algorithm based on image complexity to obtain a binary image, specifically as follows:

[0031] The image complexity of the fused feature map is calculated. The image complexity is defined as the weighted sum of edge density and entropy value of HSV color space. Edge density is equal to the number of edge pixels in the semantic edge map divided by the total number of pixels.

[0032] The initial segmentation threshold is obtained based on the maximum inter-class variance algorithm. Dynamically adjust the threshold based on image complexity ;

[0033] Based on the calculated adaptive threshold The fused feature map is segmented to obtain a binary image.

[0034] Furthermore, threshold , To adjust the coefficient, Let be the image complexity.

[0035] Further, in step S4, the ROI post-processing is as follows: performing morphological opening and closing operations on the binary image after threshold segmentation, extracting connected regions as the final ROI, and outputting the ROI boundary coordinates and mask image.

[0036] Furthermore, the method also includes:

[0037] S5. Using the same process as steps S1 and S2, process the eye movement data in the natural image-eye movement dataset to generate the corresponding gaze duration weighted heatmap, and construct an image heatmap dataset including natural images and their gaze duration weighted heatmaps.

[0038] S6. A lightweight encoder-decoder structure is selected as the basic model. The basic model is trained using the image heatmap dataset to obtain the prediction network. The gaze duration weighted heatmap is constructed as the supervision ground truth.

[0039] S7. For new, unseen natural images, directly input the new natural images into the prediction network and output a saliency map consistent with the gaze duration weighted heatmap format; based on the saliency map and the new natural images, execute steps S3 and S4 to obtain the ROI partitioning results.

[0040] The present invention also provides a natural image ROI segmentation system weighted by eye-tracking fixation duration, the key of which is: it includes an eye-tracking data processing module, a weighted heatmap construction module, a multimodal feature fusion module, an ROI segmentation module, a dataset construction module, a network training module, and an ROI inference module, which are respectively used to execute steps S1, S2, S3, S4, S5, S6, and S7 in the natural image ROI segmentation method weighted by eye-tracking fixation duration.

[0041] This invention provides a method and system for natural image ROI segmentation based on eye-tracking gaze duration weighting. It extracts effective gaze points and their gaze durations through eye-tracking data preprocessing, and constructs a gaze duration-weighted heatmap using density peak clustering and a gaze duration weighting model, effectively enhancing the feature representation of high-attention regions. By fusing the eye-tracking heatmap with multimodal semantic features such as color, texture, and edge, it significantly improves the semantic consistency of ROI segmentation. Furthermore, by introducing an image complexity-adaptive segmentation threshold, it achieves flexible adaptation to the segmentation needs of images with varying complexity. Experimental results demonstrate that the performance of this method and system is significantly superior to traditional methods, thus providing solid theoretical and practical support for intelligent ROI segmentation of high-resolution natural images oriented towards visual attention.

[0042] To address the dependence of the original system on eye-tracking hardware and measured eye-tracking data, this invention further proposes an efficient integrated optimization scheme. This scheme relies on a dataset and uses a gaze duration-weighted attention heatmap generated by the aforementioned method as a high-precision supervised ground truth to train a depth saliency prediction network that requires no eye-tracking input. This enables the network to autonomously output a high-accuracy saliency map based solely on a new natural image during the testing phase, and to perform intelligent ROI segmentation accordingly, all without requiring any eye-tracking equipment. By replacing online eye-tracking acquisition with offline learning during the training phase, this scheme effectively overcomes hardware limitations, significantly broadens the applicable scenarios of the technology, and greatly enhances the practical deployment value and promotion potential of the research results. Attached Figure Description

[0043] Figure 1 This is a flowchart of the natural image ROI segmentation method weighted by eye-tracking fixation duration provided in an embodiment of the present invention;

[0044] Figure 2 This is a visualization result of the ROI partitioning process provided in an embodiment of the present invention. Figure 2 In the middle, (a), (b), (c), (d), and (e) are the original image, the baseline density map, the gaze duration weighted heatmap, the fused feature map, and the ROI segmentation results, respectively. Detailed Implementation

[0045] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. The embodiments are given for illustrative purposes only and should not be construed as limiting the present invention. The accompanying drawings are for reference and illustration only and do not constitute a limitation on the scope of patent protection of the present invention, because many changes can be made to the present invention without departing from the spirit and scope of the present invention.

[0046] This invention first provides a method for natural image ROI segmentation based on eye-tracking fixation duration weighting, such as... Figure 1 As shown in the flowchart, the method includes:

[0047] S1. Eye movement data preprocessing and feature extraction: Clean and denoise the raw eye movement data to obtain effective fixation point data, and extract the coordinates and fixation duration of the effective fixation points in the effective fixation point data;

[0048] S2. Construction of gaze duration weighted heatmap: Based on density peak clustering (DPC) and gaze duration weighted model, a gaze duration weighted heatmap reflecting the distribution of human visual attention is generated based on the coordinates and gaze duration of the extracted effective gaze points.

[0049] S3. Multimodal feature fusion: Extract color, texture, and semantic edge features of the natural image corresponding to the original eye-tracking data, and fuse them with the gaze duration weighted heatmap to obtain a fused feature map;

[0050] S4. Adaptive Threshold Segmentation and ROI Generation: The fused feature map is segmented using an adaptive threshold algorithm based on image complexity to obtain a binary image; ROI post-processing is performed on the binary image to obtain the ROI partitioning result.

[0051] (1) Step S1: Eye-tracking data preprocessing and feature extraction

[0052] First, raw eye-tracking data and its corresponding natural images are collected. The raw eye-tracking data includes the coordinates (x, y) of the eye movement trajectory points when the subject views the natural image, sampling timestamps, blink markers, and other information. The preprocessing and feature extraction process includes:

[0053] S11. Cleaning and noise reduction: The 3σ criterion is used to screen and remove spurious noise points (such as abnormal coordinates caused by blinking) in the eye movement trajectory, and retain the effective fixation point data.

[0054] S12. Fixation Point Recognition: Based on the velocity threshold method, when the eye movement velocity is lower than a preset velocity (e.g., 50° / s) and the fixation time exceeds a preset time (e.g., 100 ms), it is determined to be a valid fixation point, and the coordinates of the fixation point are extracted. ( , (Number of effective fixations);

[0055] S13. Fixation Duration Statistics: Calculate the fixation duration for each effective fixation point. That is, the time interval between two consecutive saccades, which gives the fixation duration sequence. .

[0056] Step S1 accurately extracts effective fixation points and their durations from the original eye movement trajectory using a velocity threshold method and a dwell time criterion. This process effectively eliminates invalid data such as blinks and noise, ensuring the accuracy of subsequent attention intensity quantification and laying a reliable data foundation for high-precision ROI segmentation.

[0057] (2) Step S2: Construction of gaze duration weighted heatmap

[0058] This step aims to generate a heatmap reflecting the distribution of human visual attention, specifically including:

[0059] S21. Spatial Clustering of Fixations: The Density Peak Clustering (DPC) algorithm is used to spatially cluster effective fixations, identifying regions of attention concentration, and obtaining... There are several clusters. Cluster radius. Adaptive settings based on image resolution (assuming image width is...) Height is , ), (This represents the minimum value function). The minimum number of cluster points is set to 3 to ensure the stability of the clustering results.

[0060] S22, Fixation Duration Weighted Model: For each cluster ( ), calculate the average fixation duration of all fixations within the cluster:

[0061] ,

[0062] in, For the first Clusters The number of fixations in the middle, For the first Clusters The Middle One point of focus.

[0063] Then, the average fixation duration of each cluster is used as the attention weight for that cluster to calculate each effective fixation point. Attention value:

[0064] ,

[0065] in, For the first Clusters The center coordinates, For Gaussian kernel function, The kernel width is used to smooth the attention distribution.

[0066] S23. Normalize the attention value of each valid fixation point to... The interval is used to obtain a gaze duration-weighted heatmap.

[0067] Step S2 employs a density peak clustering and gaze duration weighted model to generate a heatmap that simultaneously reflects the spatial distribution of gaze and the intensity of attention. Compared to traditional methods that rely solely on point density, this step significantly improves the heatmap's sensitivity to high-attention regions, enabling subsequent ROI segmentation to maintain high recall and accuracy even in low-contrast and complex scenes.

[0068] (3) Step S3: Multimodal feature fusion

[0069] To improve the semantic consistency of ROI segmentation, the gaze duration-weighted heatmap and objective semantic features of the image are integrated. Step S3 specifically includes:

[0070] S31. Color Feature Extraction: Convert the natural image to HSV space, fuse the luminance (val) and saturation (sat) channels (weight ratio 0.7:0.3), generate a color feature map and normalize it to highlight the color differences of the semantic subject of the image;

[0071] S32. Texture feature extraction: Local Binary Pattern (LBP) is used to extract texture features from natural images, obtain texture feature maps and normalize them. The radius is set to 3 and the number of neighboring pixels to 8 to adapt to the detailed texture of high-resolution images.

[0072] S33. Edge feature extraction: Extract semantic edges of natural images based on the Canny algorithm, combine morphological closing operation (kernel size 3×3) to fill edge gaps, retain weak edge information, obtain semantic edge map and normalize it;

[0073] S34. Multimodal Feature Fusion: The corresponding gaze duration-weighted heatmap is weighted and fused with the normalized color feature map, texture feature map, and semantic edge map to obtain a fused feature map. The weight coefficients are as follows: , , and .

[0074] Step S3 involves weighted fusion of the gaze duration-weighted heatmap with multiple image semantic features, including color, texture, and edges. This fusion strategy effectively compensates for the shortcomings of pure attention information in semantic edge representation, significantly enhances the semantic consistency of ROI segmentation, and avoids the problems of blurred target contours or semantic loss caused by relying solely on saliency maps.

[0075] (4) Step S4: Adaptive threshold segmentation and ROI generation

[0076] This step employs an adaptive thresholding algorithm based on maximum inter-class variance (OTSU) and image complexity. Specifically, step S4 includes:

[0077] S41. Image Complexity Calculation: Defining Image Complexity edge density With color entropy The weighted sum, i.e.:

[0078]

[0079] in, This represents the number of edge pixels in the semantic edge map. The entropy value of the HSV color space. , These are the weighting coefficients.

[0080] S42. Adaptive Threshold Calculation: Obtaining the initial segmentation threshold based on the OTSU algorithm. Dynamically adjust the threshold based on image complexity :

[0081] ,

[0082] in, To adjust the coefficients, when the image complexity is high, Increase the size to avoid over-partitioning; when the complexity is low... Reduce the size to ensure complete extraction of the ROI.

[0083] S43, Adaptive Threshold Segmentation: Based on the calculated adaptive threshold The fused feature map is segmented to obtain a binary image;

[0084] S43. ROI Post-processing: Perform morphological opening (removing small area noise) and closing (filling in regional holes) operations on the binary image after threshold segmentation, extract connected regions as the final ROI, and output the ROI boundary coordinates and mask image.

[0085] Step S4 introduces an adaptive adjustment of the OTSU segmentation threshold based on image complexity. This mechanism allows the segmentation intensity to dynamically change with the image content, avoiding over-segmentation in simple images and preventing under-segmentation in complex images, thereby significantly improving the robustness and segmentation accuracy of ROI partitioning in different natural scenes.

[0086] Overall, the eye-tracking gaze duration-weighted ROI segmentation method for natural images provided by this invention, through the collaborative processing of steps S1 to S4, sequentially completes gaze duration extraction, gaze duration-weighted heatmap construction, multimodal feature fusion, and adaptive threshold segmentation. Ultimately, it achieves high-precision (accuracy exceeding 90%) and high semantic consistency (intersection over union ratio significantly better than traditional methods) automatic segmentation of regions of interest in complex natural images, effectively solving the technical problems of insufficient utilization of gaze duration information and lack of adaptability of segmentation thresholds in existing methods.

[0087] It should be noted that the various processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved. This embodiment does not impose any limitations on these steps.

[0088] In response to the above method, this embodiment of the invention also provides a natural image ROI segmentation system weighted by eye-tracking fixation duration, which includes an eye-tracking data processing module, a weighted heatmap construction module, a multimodal feature fusion module, and an ROI segmentation module, which are respectively used to execute steps S1, S2, S3, and S4 in the above method.

[0089] The input natural image resolution is The number of effective fixations is The number of clusters is The time complexity of the eye-tracking data preprocessing stage is... The focus clustering stage uses the DPC algorithm, with a time complexity of O(n log n). The time complexity of the heatmap construction and feature fusion stages is: The time complexity of the threshold segmentation and post-processing stages is... Because the dataset has a moderate image resolution (average 800×600), and Typically, the number of operations does not exceed 500, and the overall time complexity of the method is O(n). It can meet the needs of real-time processing.

[0090] The experiment used a publicly available eye-tracking dataset containing 120 natural scene images (covering various semantic categories such as landscapes, people, animals, and architecture). Eye movement records from 20 healthy subjects viewing the images were included. The data was stored in .fix format (a list of fixation points). Each record included the x-coordinate and y-coordinate of the fixation point, the start time, end time, and duration (in seconds). After preprocessing, the number of effective fixation points per image was stabilized at 300-500. Accompanying 120 images were saliency density benchmark maps (supporting .mat / .png formats) as a ground truth reference for ROI partitioning results, used for quantitative evaluation of algorithm performance.

[0091] Two classic ROI partitioning methods were selected for comparison:

[0092] (1) SIFT feature clustering method: extract SIFT feature points from the image, use K-means clustering (K=5) to determine the core region of ROI, and combine edge detection to achieve division;

[0093] (2) ITTI saliency detection algorithm: a classic visual saliency model that generates a saliency map by fusing multi-scale features and uses a fixed threshold (0.5×255) to segment ROI.

[0094] Three core metrics are used to evaluate ROI splitting performance:

[0095] Accuracy (PA): The proportion of pixel values ​​that perfectly match the predicted ROI mask with the reference mask (generated from the density map ground truth). PA = (predicted ROI mask / reference mask) × 100%. The higher the value, the better.

[0096] Intersection over Union (IoU): A metric for the degree of regional overlap in high-resolution images. It calculates the ratio of the area of ​​the intersection to the area of ​​the union. IoU = (intersection area / union area) × 100%, reflecting the degree of regional overlap. The higher the value, the better.

[0097] Mean Square Error (MSE): The pixel-level mean square error between the ROI mask and the reference mask; the smaller the value, the better.

[0098] The average performance comparison results on the test set are shown in Table 1. Based on the data in the table, after recalculation, the following results were obtained: the accuracy (PA) of the proposed system reached 90.89%, an improvement of 15.67% compared to the SIFT feature clustering method and 20.62% compared to the ITTI significance algorithm; the intersection-over-union (IoU) ratio reached 83.61%, an improvement of 11.45% compared to the SIFT feature clustering method and 17.62% compared to the ITTI significance algorithm; the mean squared error (MSE) was 0.0911, a decrease of 0.1567 compared to the SIFT feature clustering method and a decrease of 0.2062 compared to the ITTI significance algorithm.

[0099] Table 1 Comparison of average performance of each algorithm

[0100]

[0101] Figure 2 The visualization results of this method in the ROI segmentation process of typical images are shown. Figure 2 In the image, (a), (b), (c), (d), and (e) represent the original image, baseline density map, gaze duration-weighted heatmap, fused feature map, and ROI segmentation results, respectively. Figure 2 It can be seen that the proposed method can accurately cover the semantic main region of the image, which is highly consistent with the baseline density map, with clear edge contours and no obvious over-segmentation or under-segmentation. This further verifies the advantages of the proposed method in high-resolution image ROI segmentation.

[0102] In addition, to perform high-precision, highly semantically consistent ROI partitioning on new images without eye-tracking data, the method also includes the following steps:

[0103] S5. Construction of high-quality supervised dataset: Using the same process as steps S1 and S2, the eye movement data in the natural image-eye movement dataset is processed to generate the corresponding gaze duration weighted heatmap, and an image heatmap dataset including natural images and their gaze duration weighted heatmap is constructed.

[0104] S6. Training the Deep Saliency Network: A lightweight encoder-decoder structure (such as an improved network based on MobileNetV2) is selected as the base model. The gaze duration-weighted heatmap is used as the supervised ground truth to train an end-to-end saliency prediction network. During training, the network will automatically learn the mapping relationship between "image semantic features → human attention distribution" without the need for additional eye-tracking data.

[0105] S7. Hardware-independent ROI inference: For a new, unseen natural image, the trained saliency prediction network can directly input the new natural image and output a saliency map in the same format as the gaze duration weighted heatmap. Substitute the saliency map and the new natural image into the aforementioned steps S3 and S4 to obtain the final ROI. No eye-tracking hardware support is required throughout the process.

[0106] The extended process in steps S5-S7 effectively solves the problem of the original system's dependence on eye-tracking data. By transferring the attention patterns contained in eye-tracking data to a deep learning model, the ROI segmentation technology can be adapted to a wide range of application scenarios without eye-tracking devices (such as mobile phone image editing, intelligent monitoring target localization, embedded vision systems, etc.), significantly improving the engineering application value and promotion potential of the research results.

[0107] Corresponding to steps S5-S7, the natural image ROI segmentation system with eye-tracking fixation duration weighting provided in this embodiment of the invention includes a dataset construction module, a network training module, and an ROI inference module, which are used to execute the above steps S5, S6, and S7, respectively.

[0108] The embodiments described in this invention can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with the implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0109] Computer programs for implementing the methods and systems of the present invention may be written in any combination of one or more programming languages ​​and stored in a computer-readable storage medium. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0110] Computer-readable storage media can be tangible media that may contain or store computer programs for use by or in conjunction with an instruction execution system, apparatus, or device. Computer-readable storage media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0111] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A natural image ROI segmentation method weighted by eye-tracking fixation duration, characterized in that, Including the following steps: S1. Clean and denoise the raw eye movement data to obtain effective fixation point data, and extract the coordinates and fixation duration of the effective fixation points from the effective fixation point data; S2. Generate a gaze duration-weighted heatmap reflecting the distribution of human visual attention based on the coordinates and gaze duration of the extracted effective gaze points; S3. Extract the color, texture, and semantic edge features of the natural image corresponding to the original eye-tracking data, and perform weighted fusion with the gaze duration weighted heatmap to obtain a fused feature map; S4. The fused feature map is segmented using an adaptive thresholding algorithm based on image complexity to obtain a binary image; the binary image is then subjected to ROI post-processing to obtain the ROI partitioning result.

2. The eye-tracking fixation duration-weighted natural image ROI segmentation method according to claim 1, characterized in that, In step S1, the extraction of the coordinates and fixation duration of effective fixation points from the effective fixation point data specifically includes: Based on the velocity threshold method, when the eye movement velocity is lower than the preset velocity and the dwell time exceeds the preset time, it is determined to be a valid fixation point, and the coordinates of the fixation point are extracted. The fixation duration for each effective fixation point is calculated, which is the time interval between two adjacent saccades, to obtain the fixation duration sequence.

3. The eye-tracking fixation duration-weighted natural image ROI segmentation method according to claim 2, characterized in that, Step S2 specifically includes: Spatial clustering of effective gaze points is performed to identify regions of attention concentration, resulting in multiple clusters. Calculate the average gaze duration for each cluster; The average fixation duration of each cluster is used as the attention weight for that cluster to calculate the attention value for each effective fixation point; Normalize the attention value of each effective fixation point to The interval is used to obtain a gaze duration-weighted heatmap.

4. The eye-tracking fixation duration-weighted natural image ROI segmentation method according to claim 3, characterized in that, For effective fixation point Its attention value is calculated as follows: ， in, For the first Clusters The center coordinates, For Gaussian kernel function, For kernel width; For the first Clusters Average fixation duration This represents the number of clusters.

5. The eye-tracking fixation duration-weighted natural image ROI segmentation method according to claim 1, characterized in that, In step S3, the color, texture, and semantic edge features of the natural image corresponding to the original eye-tracking data are extracted, specifically as follows: The natural image is converted to the HSV space, the brightness and saturation channels are fused, a color feature map is generated and normalized; Local binary mode is used to extract texture features from natural images, resulting in texture feature maps that are then normalized. The semantic edges of natural images are extracted based on the Canny algorithm, and morphological closing operations are used to fill the edge gaps, retaining weak edge information, resulting in a semantic edge map that is then normalized.

6. The eye-tracking fixation duration-weighted natural image ROI segmentation method according to claim 1, characterized in that, In step S4, the fused feature map is segmented using an adaptive thresholding algorithm based on image complexity to obtain a binary image, specifically as follows: The image complexity of the fused feature map is calculated. The image complexity is defined as the weighted sum of edge density and entropy value of HSV color space. Edge density is equal to the number of edge pixels in the semantic edge map divided by the total number of pixels. The initial segmentation threshold is obtained based on the maximum inter-class variance algorithm. Dynamically adjust the threshold based on image complexity ; Based on the calculated adaptive threshold The fused feature map is segmented to obtain a binary image.

7. The eye-tracking fixation duration-weighted natural image ROI segmentation method according to claim 6, characterized in that: threshold , To adjust the coefficient, Let be the image complexity.

8. The eye-tracking fixation duration-weighted natural image ROI segmentation method according to claim 1, characterized in that, In step S4, the ROI post-processing is as follows: performing morphological opening and closing operations on the thresholded binary image, extracting connected regions as the final ROI, and outputting the ROI boundary coordinates and mask image.

9. The eye-tracking fixation duration-weighted natural image ROI segmentation method according to any one of claims 1 to 8, characterized in that, The method also includes: S5. Using the same process as steps S1 and S2, process the eye movement data in the natural image-eye movement dataset to generate the corresponding gaze duration weighted heatmap, and construct an image heatmap dataset including natural images and their gaze duration weighted heatmaps. S6. A lightweight encoder-decoder structure is selected as the basic model. The basic model is trained using the image heatmap dataset to obtain the prediction network. The gaze duration weighted heatmap is constructed as the supervision ground truth. S7. Input the new natural image into the prediction network and output a saliency map consistent with the gaze duration weighted heatmap format; based on the saliency map and the new natural image, execute steps S3 and S4 to obtain the ROI partitioning results.

10. A natural image ROI segmentation system weighted by eye-tracking fixation duration, characterized in that: The system includes an eye-tracking data processing module, a weighted heatmap construction module, a multimodal feature fusion module, an ROI partitioning module, a dataset construction module, a network training module, and an ROI inference module, which are used to execute steps S1, S2, S3, S4, S5, S6, and S7 in the natural image ROI partitioning method with eye-tracking fixation duration weighting as described in claim 9.

Images