Astronomical target retrieval system and method based on cross-modal interaction and generative enhancement

By employing cross-modal interaction and generative enhancement methods, a unified image-text feature space is constructed to enable cross-modal retrieval of natural language and images. Rare target variants are generated, and retrieval results are optimized based on user feedback. This solves the problems of semantic expression difficulties and low recall rates of rare targets in existing astronomical data retrieval systems, and realizes personalized astronomical target discovery and knowledge association.

CN121808157APending Publication Date: 2026-04-07TAIYUAN UNIVERSITY OF TECHNOLOGY +1
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-12-29
Publication Date
2026-04-07

AI Technical Summary

Technical Problem

Existing astronomical data retrieval systems lack adaptive interactive learning mechanisms, struggle to handle queries with complex semantic features, have low recall rates for rare targets, and their search results are disconnected from scientific knowledge bases, making personalized recommendations impossible.

Method used

We employ a cross-modal interaction and generative augmentation approach to achieve cross-modal retrieval of natural language and images by constructing a unified image-text feature space. We use generative models to synthesize variants to expand the candidate space and optimize the retrieval results through an iterative update mechanism based on user feedback.

Benefits of technology

It improves the efficiency and accuracy of discovering rare targets in massive astronomical data, enables personalized search recommendations, reduces the cost of manual image screening and repeated annotation, and enhances the interpretability and reproducibility of search results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121808157A_ABST
    Figure CN121808157A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of multi-modal data processing and astronomical informatics and provides an astronomical target retrieval system and method based on cross-modal interaction and generative enhancement to solve the problems that a traditional retrieval system is difficult to respond to semantic text query, lacks cross-modal matching ability and is low in rare target recall rate. Constructing a text-image shared semantic feature space and an image-image matched visual feature space by using the fine-tuned deep neural network, and supporting cross-modal retrieval of searching an image by a text and searching the image by an image; for a rare or difficult target, introducing a generative model to synthesize a morphological variant so as to expand a candidate space; in a cold start stage, realizing few-sample robust recall based on a feature space neighborhood of an interest sample; in the continuous interaction stage, an online training scoring model is fed back according to the interest of a user, and candidate results are subjected to personalized rearrangement; and finally, automatically matching a retrieval result with the astronomical knowledge base and outputting an analysis report.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of multimodal data processing and astroinformatics, and in particular to an astronomical target retrieval system and method based on cross-modal interaction and generative enhancement. Background Technology

[0002] With the implementation of various large-scale sky survey projects, the scale of astronomical observation data continues to grow, posing a significant challenge to the storage and management of massive amounts of astronomical image data. Researchers urgently need to quickly and accurately locate celestial objects similar to specific scientific targets within this vast amount of data.

[0003] Existing astronomical data retrieval systems generally fall into two categories: The first category is based on structured attribute information (such as celestial coordinates RA / Dec or physical parameters). This method relies on pre-determined physical parameters or location information for range filtering, making it difficult to handle queries with highly semantic features. For example, when researchers need to find "spiral galaxies with bar structures" or "irregular galaxies undergoing mergers," traditional coordinates or physical parameters are insufficient to effectively express such complex morphological semantics, leading to low retrieval efficiency. The second category is based on similarity matching of static image features. This method finds similar celestial objects by calculating the distance between image feature vectors. However, most existing astronomical data retrieval systems only support single-modal input and lack cross-modal mapping capabilities, failing to directly convert users' natural language descriptions into effective retrieval vectors, thus limiting the flexibility of retrieval.

[0004] Furthermore, in the field of general information retrieval and recommendation, there are already ideas such as interactive optimization methods based on user feedback, confidence region filtering, and data augmentation methods based on generative models. However, in the scenario of retrieving rare astronomical targets, due to the influence of long-tail distribution, noise artifacts, and the requirement of scientific traceability, the above ideas are difficult to directly adapt or form an end-to-end integrated closed-loop mechanism, resulting in insufficient candidate coverage, ranking stability, and knowledge association capabilities.

[0005] Existing technologies still have the following significant shortcomings in practical scientific research applications: First, they lack an adaptive interactive learning mechanism: traditional feature extraction and retrieval processes are static and do not incorporate a "human-in-the-loop" feedback mechanism. Existing astronomical data retrieval systems cannot dynamically adjust their ranking strategies based on user feedback of "interest / disinterest" during the retrieval process. This leads to unstable recall results in the cold start phase of astronomical data retrieval (i.e., when users provide only a very small number of samples or no samples); and in long-term interactions, the images recommended by astronomical retrieval cannot adaptively converge according to the specific research preferences of researchers, making personalized recommendations difficult. Second, current technologies face the problem of difficulty in discovering rare targets (difficult examples): in astronomical research, high-scientific-value targets are often rare targets in long-tail distributions (such as merging galaxies with special morphologies, low surface brightness structures, etc.). Such target samples are extremely scarce, making it difficult to form effective clusters in the feature space. Existing retrieval systems rarely utilize generative artificial intelligence technology and cannot generate "similar but different" variants to expand the candidate space while maintaining the consistency of physical features, resulting in insufficient recall for rare targets. Finally, existing search results are disconnected from scientific knowledge bases: current methods typically only output image pixels or file indexes, failing to achieve automated alignment with standard astronomical knowledge bases. After obtaining search results, researchers often cannot directly access key scientific metadata such as corresponding celestial coordinates and object types, requiring significant time for secondary matching, making it difficult to directly serve subsequent scientific annotation and physical analysis.

[0006] In conclusion, developing an astronomical image retrieval system that can understand semantic text, support the discovery of rare targets under few-sample conditions, and possess continuous learning capabilities and knowledge association functions has become an urgent need in the field of astronomical data processing. Summary of the Invention

[0007] To address the shortcomings of existing technologies, such as limited retrieval methods, difficulties in semantic expression, low recall rates for rare targets, and lack of user interaction feedback mechanisms, this invention aims to provide an astronomical target retrieval system and method based on cross-modal interaction and generative augmentation. This invention achieves cross-modal retrieval of natural language and images by constructing a unified image-text feature space; it addresses the problem of sparsity in rare target data by using generative models to synthesize variants; and it achieves personalized adaptive optimization of retrieval results through a human-in-the-loop (HITL) iterative update mechanism based on user feedback, significantly improving the efficiency and accuracy of discovering scientific targets in massive astronomical data. This invention integrates semantic understanding, generative data augmentation, and closed-loop iterative learning based on user feedback to improve the retrieval accuracy and discovery efficiency of rare targets in massive astronomical data.

[0008] To achieve the above objectives, the present invention provides the following technical solution: An astronomical target retrieval system based on cross-modal interaction and generative enhancement includes a server for storing and processing astronomical image data, and a user terminal that allows users to input search information and receive feedback. The user terminal accesses a front-end interactive interface through a browser. The front-end interactive interface collects the user input information, encapsulates it into a request, and sends it to the server. The system receives candidate results, variant previews, and ranking information returned by the server and displays them visually. The server includes a generative variant module, a cold-start screening module, and a human-in-the-loop interaction module. Specifically, the generative variant module generates variant maps based on user-annotated samples when the candidate set coverage is insufficient and / or the number of effective labeled samples is less than a preset condition N1. The system expands candidate samples, extracts features from the generated samples, and places them into a visual representation feature library. The cold start screening module, when the number of user-annotated samples is below a preset threshold T1, obtains an expanded candidate set based on similarity retrieval, filters and sorts the expanded candidate set based on statistical distance, and outputs the recommendation results for the cold start phase. The human-in-the-loop interaction module, when the number of user-annotated samples is greater than or equal to the preset threshold T1, iterates and updates the lightweight scoring model based on user annotations and ratings, and scores and reorders the candidate results. The server returns the sorted candidate results to the user terminal and receives annotation and rating information from the user terminal for subsequent iterative updates.

[0009] Furthermore, the front-end interactive interface includes a text query input component, an image upload component, a variant generation parameter setting and triggering component, an interest or non-interest annotation and rating component, and a result visualization display component. The user information collected by the front-end interactive interface includes text, images, parameters, and user annotation information for recommended astronomical images.

[0010] Furthermore, the server is set up in a server or cloud computing environment, and the backend of the server is implemented based on a web service framework. The server also includes a caching module and a task queue module. The caching module is used to cache the search results of recent or frequently queried queries and update and evict them according to a preset expiration time or a least recently used strategy.

[0011] Furthermore, it also includes a retrieval and interaction interface module, a query processing module, a data preprocessing module, a text feature extraction module, an image feature extraction module, a vector retrieval engine, and a data and knowledge base unit. The retrieval and interaction interface module receives retrieval requests from user terminals and user annotations and ratings of candidate images, and returns retrieval results to user terminals. The query processing module parses the retrieval mode and retrieval parameters, and schedules data preprocessing, feature extraction, vector retrieval, and rearrangement processes to generate a candidate set. The data preprocessing module performs standardized preprocessing on the input astronomical image data. The text feature extraction module generates text feature vectors from user-input text using a cross-modal text-image coding model. The image feature extraction module generates image feature vectors from library images, user-uploaded images, and / or generative variant images using a visual representation model. The vector retrieval engine performs similarity retrieval in the cross-modal retrieval feature library and the visual retrieval feature library to obtain a candidate result set. The data and knowledge base unit stores and maintains the image feature library, text feature library, vector index library, and / or astronomical knowledge base, and provides data access and retrieval call support to the query processing module and the vector retrieval engine.

[0012] An astronomical target retrieval method based on cross-modal interaction and generative enhancement is implemented using the aforementioned astronomical target retrieval system based on cross-modal interaction and generative enhancement. It employs a cross-modal text-image coding model and a self-supervised visual representation model to quantify the query content, obtaining two candidate sets. A retrieval performance index is calculated based on user interest feedback. When the retrieval performance index falls below a preset threshold, variant generation enhancement is automatically triggered, generating several image variants with similar morphologies under user-adjustable parameter constraints. Feature consistency screening is performed on these image variants, and those that pass the screening are used to expand the candidate set. After receiving confirmation annotations from the user terminal, the user-confirmed image variants and their corresponding candidate samples are added to an interest sample set. The method switches between a cold-start screening phase and an iterative learning phase based on user feedback, depending on the size of the interest sample set. An updated lightweight scoring model is used to personalize the candidate set, and finally, the retrieval results are output and matched with coordinate information from an astronomical knowledge base to generate a retrieval report. The specific steps include: Step 1. Astronomical image data access: fine-tuning the cross-modal text-image coding model and self-supervised visual representation model to construct image library features; the server for storing and processing astronomical image data receives massive amounts of astronomical image data from the sky survey project and constructs the image library feature database. Step 2. Cross-modal text query and image instance query: The text query is vectorized using a cross-modal language-image contrast model and the first candidate set is retrieved. The image query is vectorized using a self-supervised visual representation model and the second candidate set is retrieved. Step 3. Generative variant expansion: For the two candidate sets obtained in Step 2, the astronomical target retrieval system calculates the retrieval performance index based on the user's interest annotations and ratings. When the retrieval performance index is lower than the preset threshold, variant generation enhancement is automatically triggered, and the user is allowed to adjust or confirm the generation parameters. Step 4. Cold Start Phase Screening and Personalized Iterative Reordering of Human-in-the-Loop: The server receives and records the cumulative number of labeled samples for the same type of target by the current user; when the number of samples labeled by the user for the same type of target is less than the first threshold T1, the cold start phase screening is triggered. Neighborhood constraints are constructed based on the statistical distribution of the labeled interest sample set in the feature space, the expanded candidate set is screened and sorted, and the recommended candidate results of the cold start phase are output; when the number of samples labeled by the user for the same type of target reaches or exceeds the preset first threshold T1, a closed-loop optimization process of user feedback - lightweight scoring model update - result reordering - effect display is triggered. Step 5. Knowledge Association and Report Generation: The astronomical target retrieval system extracts the coordinate information of images of interest to the end user, automatically matches the coordinates with the astronomical knowledge base, automatically completes the references for astronomical object type, redshift, and magnitude, and generates a standardized retrieval report containing traceable image annotation information for users to export.

[0013] Furthermore, in step 2, the first candidate set is obtained as follows: Step 2.1A: The server receives the natural language query text entered by the user on the front-end interactive interface and saves the query text to the query log database. Step 2.2A calls the text encoder of the finely tuned language-image contrast pre-trained model to map the natural language text into a high-dimensional text feature vector. The server uses the high-dimensional text feature vector to perform similarity retrieval in the image feature library, generates the first candidate set, and displays it to the user for annotation. The second candidate set is obtained as follows: Step 2.1B: The server receives the local image to be queried uploaded by the user and performs image preprocessing and cropping operations. Step 2.2B calls the unsupervised self-distillation visual representation model to extract the visual feature vector of the image to be queried, calculates the cosine similarity in the visual retrieval feature library, sorts the images by cosine similarity from high to low, selects images with similarity not lower than the preset visual representation similarity threshold T0, selects the top K images with the highest similarity from these images as images with similar morphology, generates the second candidate set and displays it to the user for annotation.

[0014] Furthermore, the preset visual representation similarity threshold T0 is set to a value range of [0.7, 0.9], and the value range of K is set to [10, 100].

[0015] Furthermore, in step 3, the steps for automatically triggering variant generation enhancement and parameter adjustment are as follows: Step 3.1 Variant generation: Based on the preset visual representation similarity threshold T0, the generated variant results are screened for consistency; when the similarity between the variant and the generated seed in the visual representation space meets the preset interval condition, the variant is identified as a slightly different variant, and several variant images are generated; Step 3.2 Candidate Fusion: Extract features from the variant images selected by the user that meet the expectations, set the variants that pass the screening and are confirmed by the user as the images of interest to the user, and perform similarity retrieval in the visual representation feature library as an extended query to obtain an additional candidate set. The additional candidate set is merged with the first candidate set and the second candidate set to form an extended candidate set.

[0016] Furthermore, in step 4, the process of accumulating the number of user-annotated samples is as follows: When the cumulative number of labeled samples is less than the first threshold T1, it is the cold start phase. The server obtains the feature vectors of the currently collected labeled samples, calculates the mean vector μ and covariance matrix Σ of all labeled sample feature vectors, and constructs the feature space neighborhood using Mahalanobis distance. For the feature vector x of each candidate image in the expanded candidate set obtained in step 3, the Mahalanobis distance D of each candidate image feature vector relative to the mean vector μ and covariance matrix Σ of the interest sample set is calculated. 2 The calculation formula is: Where x represents the feature vector of the candidate image in the expanded candidate set, μ represents the mean vector of the feature vectors of the interest samples, and Σ -1 Let represent the inverse of the covariance matrix; (·) T The transpose of a vector or matrix is ​​represented, and D represents the Mahalanobis distance scalar value from the feature vector x of the candidate image to the center of the interest distribution. The Mahalanobis distance threshold is determined according to the preset confidence threshold, and the Mahalanobis ellipsoid region is constructed based on the Mahalanobis distance threshold. Candidate images whose Mahalanobis distance satisfies the Mahalanobis ellipsoid region constraint are displayed as the priority recommendation results. The personalized iterative rearrangement operation is as follows: The server automatically collects explicit user feedback and user-annotated image ratings, storing the data in an online learning sample pool. It then reads feature vectors corresponding to interest / non-interest samples from the pool as input to a lightweight scoring model. Based on user ratings, it normalizes the interest / non-interest samples to obtain sample weights. A binary classification cross-entropy loss is used, and the total loss is calculated by weighting the samples according to their weights, thus updating the lightweight scoring model. During the recommendation phase, the updated lightweight scoring model is used to re-score and rank the expanded candidate set. The server randomly selects a portion of images, scores and ranks them using the fine-tuned lightweight scoring model, and displays the results for users to intuitively judge the personalized recommendations. Once the user is satisfied with the recommendation results, the server saves the lightweight scoring model parameters for the current user, generating a directly callable personalized recommendation version suitable for the current user, which is then used for subsequent batch filtering tasks.

[0017] Compared with the prior art, the present invention has the following significant advantages: This invention introduces text-image cross-modal coding technology, enabling natural language requirements to be directly transformed into searchable representation vectors. Researchers can quickly locate targets using high-level semantic descriptions such as "spiral galaxies with bar structures" without relying on complex physical parameters, greatly lowering the retrieval threshold and improving flexibility, thus achieving cross-modal retrieval. Addressing the scarcity of high-value target samples in astronomical research, this invention introduces generative variant enhancement technology. By generating a variant expansion candidate set of "similar but different" variations in the feature space, the recall space is effectively expanded, significantly improving the ability to discover rare and marginal targets. A continuous learning mechanism based on user feedback is established. By capturing users' interests and preferences during the interaction process and fine-tuning the scoring model in real time, it can adaptively meet the research needs of different scientists, avoiding "one-size-fits-all" search results and greatly reducing the cost of manual image screening and repeated annotation. The image retrieval results are deeply linked with the astronomical knowledge base, automatically outputting scientific reports containing key information such as celestial coordinates (RA / Dec) and object type. This not only improves the interpretability of the results but also supports the verifiability of the results, enabling the retrieval tool to be seamlessly integrated into the subsequent scientific analysis and paper writing process. To address the problem of insufficient samples in the early stages of user use, this invention adopts a cold-start strategy based on Mahalanobis distance to ensure that the system can still provide stable recommendation results even without a large amount of labeled data, guaranteeing a good "ready-to-use" experience for the astronomical target retrieval system. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall architecture of the astronomical target retrieval system based on cross-modal interaction and generative enhancement of the present invention; Figure 2 The flowchart of the astronomical target retrieval method based on cross-modal interaction and generative enhancement of the present invention is shown. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0020] like Figures 1-2 As shown, this invention discloses an astronomical target retrieval system based on cross-modal interaction and generative enhancement. It includes a server for storing and processing astronomical image data, and a user terminal that allows users to input search information and receive feedback. The user terminal accesses a front-end interactive interface via a browser. The front-end interactive interface collects user input information, encapsulates it into a request, and sends it to the server. It receives candidate results, variant previews, and ranking information returned by the server and displays them visually. The front-end interactive interface includes a text query input component, an image upload component, a variant generation parameter setting and triggering component, an interest or non-interest annotation and rating component, and a result visualization component. The user information collected by the front-end interactive interface includes text, images, parameters, and user annotation information for recommended astronomical images. The server is set up in a server or cloud computing environment, and the server's backend is implemented based on a Web service framework. The server also includes a caching module and a task queue module. The caching module is used to cache recent or frequently queried search results and set a preset expiration time. The update and elimination strategy employs either the least recently used (LRU) approach or a generative variant module, a cold start filtering module, and a human-in-the-loop (HIL) interaction module. Specifically: the generative variant module generates variant images to expand the candidate sample set when the candidate set coverage is insufficient and / or the number of effective labeled samples is less than a preset condition N1. It then extracts features from the generated samples and adds them to a visual representation feature library. N1 ranges from 10% to 15%. The cold start filtering module obtains an expanded candidate set based on similarity retrieval when the cumulative number of user-labeled samples is less than a preset threshold T1. It then filters and sorts the expanded candidate set based on statistical distance and outputs the recommendation results for the cold start phase. The HIL interaction module iteratively updates the lightweight scoring model based on user annotations and ratings when the cumulative number of user-labeled samples is greater than or equal to the preset threshold T1. It also scores and re-ranks the candidate results. The server returns the sorted candidate results to the user terminal and receives the annotation and rating information returned by the user terminal for subsequent iterative updates.

[0021] This invention also includes a retrieval and interaction interface module, a query processing module, a data preprocessing module, a text feature extraction module, an image feature extraction module, a vector retrieval engine, and a data and knowledge base unit. The retrieval and interaction interface module receives retrieval requests from user terminals and user annotations and ratings of candidate images, and returns retrieval results to the user terminal. The query processing module parses the retrieval mode and retrieval parameters, and schedules data preprocessing, feature extraction, vector retrieval, and rearrangement processes to generate a candidate set. The data preprocessing module performs standardization preprocessing on the input astronomical image data, including but not limited to cropping, scaling, and normalization. The system includes functions such as normalization, denoising, intensity stretching, and background estimation; a text feature extraction module that generates text feature vectors from user-input text using a cross-modal text-image coding model; an image feature extraction module that generates image feature vectors from library images, user-uploaded images, and / or generative variant images using a visual representation model; a vector retrieval engine that performs similarity retrieval in the cross-modal retrieval feature library and the visual retrieval feature library to obtain a set of candidate results; and a data and knowledge base unit that stores and maintains image feature libraries, text feature libraries, vector index libraries, and / or astronomical knowledge bases, and provides data access and retrieval support to the query processing module and the vector retrieval engine.

[0022] This invention also discloses an astronomical target retrieval method based on cross-modal interaction and generative enhancement. Utilizing the aforementioned astronomical target retrieval system based on cross-modal interaction and generative enhancement, the method employs a cross-modal text-image coding model and a self-supervised visual representation model to quantify the query content, obtaining two candidate sets. Based on user interest feedback, a retrieval performance index is calculated. When the retrieval performance index falls below a preset threshold, variant generation enhancement is automatically triggered, generating several image variants with similar morphologies under user-adjustable parameter constraints. Feature consistency screening is performed on the image variants, and those that pass the screening are used to expand the candidate set. Upon receiving confirmation annotations from the user terminal, the user-confirmed image variants and their corresponding candidate samples are added to an interest sample set. The method switches between a cold-start screening phase and an iterative learning phase based on user feedback, depending on the size of the interest sample set. Here, Human-in-the-Loop (HITL) refers to an interactive iterative optimization mechanism where users annotate or rate candidate results during the retrieval / recommendation process, and use this feedback to update model parameters or reorder strategies. The candidate set is personalized and rearranged using an updated lightweight scoring model. The final search results are then matched with metadata from an astronomical knowledge base to generate a search report. The specific steps include: Step 1. Astronomical image data access: Fine-tuning the cross-modal text-image coding model and self-supervised visual representation model to construct image library features; The server storing and processing astronomical image data receives massive amounts of astronomical image data from the sky survey project and constructs an image library feature database. A self-supervised learning-based visual representation model (DINOV3) is used as the image encoder to extract features from the input images, and the extracted features are saved. Step 2. Cross-modal text query and image instance query: The cross-modal language-image contrast model (CLIP) is used to vectorize the text query and retrieve the first candidate set. The self-supervised visual representation model is used to vectorize the image query and retrieve the second candidate set. The first candidate set is obtained as follows: Step 2.1A: The server receives the natural language query text entered by the user on the front-end interactive interface, such as "spiral galaxy with obvious bar structure", and saves the query text to the query log database. Step 2.2A calls the text encoder of the finely tuned language-image contrast pre-trained model to map the natural language text into a high-dimensional text feature vector. The server uses the high-dimensional text feature vector to perform similarity retrieval in the image feature library, generates the first candidate set, and displays it to the user for annotation. Since the language-image contrast pre-trained model has been aligned and trained with a large number of astronomical images and texts in step one, the text vector and the image vector are in the same feature space.

[0023] The second candidate set is obtained as follows: Step 2.1B: The server receives the local image uploaded by the user and performs image preprocessing and cropping operations; the preprocessing and cropping operations are the same as in Step 1.

[0024] Step 2.2B involves extracting features from the preprocessed user-uploaded image using an unsupervised self-distillation visual representation model to obtain the corresponding visual feature vector. The unsupervised self-distillation visual representation model is then used to extract the visual feature vector of the query image. Cosine similarity is calculated in the visual retrieval feature library, and images with similarity no less than a preset visual representation similarity threshold T0 are selected. The top K images with the highest similarity from these images are chosen as morphologically similar images, generating a second candidate set, which is then displayed to the user for annotation. The preset visual representation similarity threshold T0 ranges from [0.7, 0.9], and K ranges from [10, 100].

[0025] Step 3. Generative Variant Expansion: For the two candidate sets obtained in Step 2, the astronomical target retrieval system calculates retrieval performance indicators based on the user's interest annotations and ratings. When the candidate set coverage is insufficient and / or the number of effective labeled samples is lower than the preset condition N1, variant generation enhancement is automatically triggered, and the user is allowed to adjust or confirm the generation parameters. The steps for automatically triggering variant generation enhancement and parameter adjustment are as follows: Step 3.1 Variant Generation: The generated variant results are screened for consistency based on a preset visual representation similarity threshold T0. When the similarity between the variant and the seed in the visual representation space meets the preset interval condition T0, the variant is identified as a slightly different variant, and several variant images are generated. The server calls a diffusion model fine-tuned with astronomical data. Using the current query image as a condition, while maintaining consistency with scientific principles regarding the main structure of celestial bodies (such as the galactic bulge and the general trend of spiral arms), slightly controllable changes are made to local details (such as spiral arm entanglement, brightness distribution, and local texture) by adjusting diffusion intensity, sampling steps, and text prompt parameters. This is equivalent to screening the generated results for consistency based on the visual representation similarity threshold. When the similarity between the variant and the seed in the visual representation space meets the preset interval condition, the variant is identified as a slightly different variant, and several "similar but different" variant images are generated.

[0026] Step 3.2 Candidate Fusion: Feature extraction is performed on the variant images selected by the user that meet the expectations. Variants that pass the screening and are confirmed by the user are set as images of interest to the user. The selected variants are used as extended queries to perform similarity retrieval in the visual representation feature library to obtain an additional candidate set. The additional candidate set is merged with the first and second candidate sets to form an extended candidate set. In this embodiment, the default user interest score is 3 points. When the generative variant enhancement process is triggered, variant images with similar shapes are generated based on the query image / seed image. After consistency screening of variant features, they are used for extended queries, thereby expanding the search space without changing the original image library data, improving the recall rate and candidate coverage of targets not initially covered by the search. At the same time, through feature consistency screening and user confirmation mechanisms, pseudo-variants that do not conform to scientific consistency are suppressed from entering the candidate and interest sets, reducing false detections and ranking fluctuations caused by the introduction of generation noise, and improving the stability and interpretability of search results.

[0027] Step 4. Cold Start Phase Screening and Personalized Iterative Reordering of Human-in-the-Loop: The server receives and records the cumulative number of labeled samples for the same type of target by the current user; when the number of samples labeled by the user for the same type of target is less than the first threshold T1, the cold start phase screening is triggered. Neighborhood constraints are constructed based on the statistical distribution of the labeled interest sample set in the feature space, the expanded candidate set is screened and sorted, and the recommended candidate results of the cold start phase are output; when the number of samples labeled by the user for the same type of target reaches or exceeds the preset first threshold T1, a closed-loop optimization process of user feedback - lightweight scoring model update - result reordering - effect display is triggered.

[0028] The process for accumulating the number of user-annotated samples is as follows: When the cumulative number of labeled samples is less than the first threshold T1, it is the cold start phase. The server obtains the feature vectors of the currently collected labeled samples, calculates the mean vector μ and covariance matrix Σ of all labeled sample feature vectors, and constructs the feature space neighborhood using Mahalanobis distance. In this embodiment, T1 is set to 20. For the feature vector x of each candidate image in the expanded candidate set obtained in step 3, the Mahalanobis distance D of each candidate image feature vector relative to the mean vector μ and covariance matrix Σ of the interest sample set is calculated. 2 The calculation formula is: Where x represents the feature vector of the candidate image in the expanded candidate set, μ represents the mean vector of the feature vectors of the interest samples, and Σ -1 Let represent the inverse of the covariance matrix; (·) T denoted as the transpose of a vector or matrix, and D represents the Mahalanobis distance scalar value from the feature vector x of the candidate image to the center of the interest distribution.

[0029] The Mahalanobis distance threshold is determined based on a preset confidence threshold, and a Mahalanobis ellipsoid region is constructed based on the Mahalanobis distance threshold. Candidate images whose Mahalanobis distance satisfies the constraints of the Mahalanobis ellipsoid region are displayed as priority recommendation results. Alternatively, a minimum bounding sphere can be trained using the Support Vector Data Description (SVDD) algorithm, and images falling inside the sphere can be used as recommendation results. When the number of interest samples is small, the Mahalanobis distance metric is constructed using the mean vector and covariance matrix of the interest sample features. This can form a stable neighborhood discrimination boundary under small sample conditions, improve the ability to screen "similar interest samples" in the cold start stage, and reduce mis-rejection.

[0030] The personalized iterative rearrangement operation is as follows: When the cumulative number of user-annotated samples reaches or exceeds the first threshold T1, a closed-loop optimization of "user feedback - model update - result rearrangement - effect display" is triggered. The server automatically collects explicit user feedback and the rating data of users when annotating images and stores it in the online learning sample pool. The rating data of users annotating images from 1 to 5 points is written into the online learning sample pool in the form of "image name + rating + label". The lightweight scoring model is input from an online learning sample pool, which reads feature vectors corresponding to interest / non-interest samples. This can be either image / text concatenation or image features only. Interest samples are marked as 1, and non-interest samples as 0. Sample weights are obtained by normalizing interest / non-interest samples according to user ratings. A binary classification cross-entropy loss is used, and the total loss is calculated by weighting the samples according to their weights to update the lightweight scoring model. Gradient clipping and early stopping can be used. During the recommendation phase, the updated lightweight scoring model is used to re-score and rank the expanded candidate set. The server randomly selects a portion of images, scores and ranks them using the fine-tuned lightweight scoring model, and displays the results for users to intuitively judge the personalized recommendations. Once the user is satisfied with the recommendation effect, the lightweight scoring model parameters for the current user are saved, generating a directly callable personalized recommendation version suitable for the current user, which is used for subsequent batch filtering tasks. To quantify the optimization effect, the server randomly selects a certain number of images (e.g., 10,000) from the image library and scores and ranks them using the fine-tuned lightweight scoring model. The server selects a lightweight scoring model to extract a number of images (e.g., 5 images) from each of the top 30%, top 50%, and top 5% segments of 10,000 data points and displays them side-by-side for users to intuitively judge the personalized recommendations of the lightweight scoring model. Once the user is satisfied with the recommendation, the astronomical target retrieval system saves the lightweight scoring model parameters for the current user and generates a personalized recommendation version that can be directly called upon for subsequent batch filtering tasks. By receiving user interest / non-interest annotations and ratings for candidate images, the server constructs a user interest distribution in a unified feature space and re-ranks the candidate set accordingly. This transforms the search results from "general similarity" to "personalized similarity tailored to the current task / target," enabling rapid cold start with fewer labeled samples and gradual convergence with each interaction round, improving the accuracy, ranking consistency, and interaction efficiency of subsequent searches. Simultaneously, an incremental update method is used to update only the lightweight scoring model or distribution parameters, avoiding damage to the backbone representation model, reducing computational overhead, and improving online response stability.

[0031] Step 5. Knowledge Association and Report Generation: The astronomical target retrieval system extracts the coordinate information of images of interest to the end user, automatically matches the coordinates with the astronomical knowledge base, automatically completes the references for astronomical object type, redshift, and magnitude, and generates a standardized retrieval report containing traceable image annotations for user export. By performing multi-source knowledge queries, matching alignment, and field unification on candidate targets, the original image retrieval results are transformed from a collection of visually similar images into structured entries containing multi-dimensional verifiable information such as celestial object name, type, and redshift. The system automatically generates analysis reports of "structured data + natural language description" using fixed templates, significantly reducing manual table lookups and data processing time, enhancing the traceability and reproducibility of results, and facilitating model iteration and optimization based on user feedback. The system extracts the file index or coordinate information of each image in the final retrieval results and performs a search and match in a pre-set astronomical knowledge base. If a match is successful, the corresponding celestial object coordinates (RA / Dec), celestial object type, redshift value, or magnitude information is retrieved and visualized in chart form.

[0032] exist Figure 1 The embodiment shown is implemented using a browser and server (B / S) architecture.

[0033] Server-side: The server is deployed on a high-performance computing server or cloud computing environment. The server backend can be implemented based on a web service framework (such as FastAPI), integrating data preprocessing modules, feature extraction modules, a vector retrieval engine (supporting offline batch construction and online incremental updates), a generative variant module, and a lightweight scoring model. The server includes a caching module and a task queue module. The caching module caches the retrieval results of recent or frequently accessed queries and updates and evicts them according to a preset expiration time (TTL) or least recently used (LRU) strategy to reduce redundant computation and response latency.

[0034] The user terminal (Client) is a front-end interactive interface built on HTML5 technology, accessed by the user through a browser. This interface includes: a text query input component, an image upload component, a variant generation parameter setting and triggering component, an interest / non-interest annotation and rating component, and a result visualization component. The front-end collects user input (text, images, parameters, and annotation information) and encapsulates it into a request to send to the server. Simultaneously, it receives candidate results, variant previews, and sorting information returned by the server and visualizes them. Data interaction between the front-end and the server can be achieved via an HTTP interface or through WebSocket for real-time status feedback and progress notifications.

[0035] The server receives massive amounts of astronomical image data from the sky survey project in advance and constructs a feature library. The specific preprocessing and feature extraction process is as follows: Step 1.1 Background Normalization: Convert the input color astronomical image into a single-channel grayscale image, use a background estimation algorithm to calculate and subtract the sky background, and eliminate the interference of background noise on feature extraction.

[0036] Step 1.2 Adaptive Source Detection: Gaussian smoothing is applied to the background-normalized image; a binary foreground mask is obtained through threshold segmentation, where regions with pixel values ​​higher than the threshold are recorded as foreground; a connected component labeling algorithm is used to extract connected regions on the binary mask, where a connected region is defined as a set of foreground pixels that are interconnected under 4-neighbor or 8-neighbor connectivity; each connected region corresponds to a candidate celestial target region, thereby achieving the detection of all possible celestial targets. Step 1.3 Target Selection and Cropping: Calculate the pixel area and centroid of each connected region, and remove noise regions with an area smaller than a preset threshold Amin; determine the target region in the remaining connected regions according to preset rules (e.g., maximum area / maximum intensity integral / minimum center distance). Calculate the minimum bounding box for the target region, and expand the bounding box by a margin m in the up, down, left, and right directions to obtain a safe cropping box, outputting a slice image containing the target celestial body.

[0037] The above preprocessing is used to improve the contrast of weak signal targets and the consistency of target regions, thereby improving the stability of subsequent feature extraction, similarity retrieval, and variant consistency screening. Step 1.4 Model fine-tuning: The training data uses the target slice images obtained by the preprocessing and cropping methods in Steps 1.1 to 1.3. The cross-modal language-image contrast model and the self-supervised visual representation model are respectively fine-tuned for adaptability in the astronomical domain to improve the ability of the cross-modal language-image contrast model and the self-supervised visual representation model to represent astronomical image features; in order to achieve domain adaptation for astronomical domain retrieval, this embodiment fine-tunes and trains the cross-modal text-image coding model and the self-supervised visual representation model respectively, as follows: (1) Fine-tuning of the cross-modal text-image coding model (for text-based image retrieval).

[0038] To achieve parameter-efficient domain adaptation, a lightweight Adapter module is preferably inserted into the last K Transformer layers (preferably K=6) of the image encoder and text encoder. During training, the backbone parameters of the cross-modal language-image contrast model are frozen, and only the Adapter module parameters, the projection head parameters, and the temperature / scale parameters (e.g., logit_scale) are updated. Optionally, the end-level normalization layer parameters are also updated to improve training stability. The projection head includes an image projection head and a text projection head, used to map the two ends of the embedding to a unified retrieval feature space. Beneficial effects: By updating only the Adapter, projection head, and temperature parameters, parameter-efficient astronomical domain adaptation is achieved, reducing training overhead and minimizing damage to the original model's general capabilities.

[0039] The training objective employs a contrastive learning loss: Within a batch, an image-text similarity matrix is ​​constructed and a predicted distribution is obtained using softmax. Further, based on the attribute sets corresponding to the samples, the attribute similarity between samples is calculated to construct a multi-positive sample soft-weight distribution. Weighted cross-entropy is used to jointly constrain the image-to-text and text-to-image directions. When constructing the multi-positive sample soft-weight distribution, the attribute set is extracted from the morphological attribute labels / voting results corresponding to each astronomical image. The attribute similarity is preferably calculated using a set similarity metric (e.g., Jaccard similarity). Beneficial effects: By constructing multi-positive sample soft weights based on attribute similarity, samples with similar attributes (morphology) are more stably aggregated in the retrieval space, thereby improving the ranking consistency and recall capability of multi-attribute combined queries and long-tail targets.

[0040] Training / fine-tuning of self-supervised visual representation models (for image-based image lookup and feature storage).

[0041] With the visual representation model backbone network frozen, a new retrieval projection head is added to map the backbone output representation to a unified retrieval feature space of a preset dimension (preferably 512 dimensions) and normalize it. Preferably, an efficient parameter fine-tuning mechanism (e.g., LoRA) is introduced only for the last K network layers of the backbone. During the training phase, only the projection head parameters and the LoRA parameters are updated, and the parameters of the end normalization layer can be optionally updated to improve training stability.

[0042] When the backbone output is a token sequence, the top-K token pooling is used to obtain the global representation: the tokens are sorted according to the preset importance score and the top K tokens are selected for aggregation to obtain a global vector, which is then input into the projection head to obtain the retrieval features; preferably, the importance score is the Euclidean norm (L2 norm) of the token vector, and the aggregation method is to calculate the mean of the top K token vectors; the aggregation can also be a weighted sum or a concatenation followed by a linear transformation to obtain the global vector.

[0043] The training objective adopts multi-view Figure 1 Self-supervised learning methods with consistency constraints (e.g., multi-view based on contrastive learning) Figure 1 The method employs a consistency loss mechanism (or self-distillation consistency loss) to make the retrieval features of the same image obtained through different data augmentations more similar in the unified retrieval space, and to maintain the distinguishability of retrieval features of different image samples. It achieves domain adaptation for astronomical images with fewer trainable parameters, reducing training costs and the risk of overfitting. Top-K token pooling is used to aggregate tags with high importance scores, enhancing the ability to represent local morphological differences and improving image-based retrieval recall and ranking stability in scenarios with noise perturbation, scale changes, and fine-grained morphological differences.

[0044] The output model is used for subsequent feature library construction.

[0045] After training / fine-tuning is completed, the self-supervised visual representation model is used to extract visual feature vectors from the cropped image obtained in step 1.3, and these vectors are used for feature storage and index construction in step 1.5. The cross-modal text-image coding model is used to map text queries to text feature vectors to perform text-to-image retrieval.

[0046] Step 1.5 Feature Input: Input the cropped image from Step 1.3 into the self-supervised visual representation model fine-tuned in Step 1.4 to extract high-dimensional visual feature vectors. After Euclidean norm normalization, the high-dimensional visual feature vectors are written into the image library feature database, and a vector index (such as the HNSW index) is constructed to accelerate subsequent retrieval.

[0047] Application scenarios of this invention: Scenario activation: The user opens a webpage, selects "Search images by text", and enters "a merging binary system with a tidal tail structure".

[0048] Preliminary retrieval: The server returns a batch of recommended images that match the text through a cross-modal language-image comparison model. The user selects images that meet the expected criteria and labels them with interest scores. While receiving the user's labels, the server calls a self-supervised learning-based visual representation model to recommend similar images to the labeled images. If, after two rounds of interaction, the user has only labeled 3 images, the server considers the current interest set to have too few samples (interest click-through rate is lower than the preset condition N1, which is 15% in this embodiment), and thus performs variant expansion.

[0049] Variant Expansion: The user inputs variant parameters (text prompt: a galaxy, quantity: 10, step 30, intensity 0.65, text constraint 7). Based on this, the system generates 10 virtual merged galaxy images with slightly different morphologies. Consistency screening is performed on the generated results based on a visual representation similarity threshold, removing 3 variant images with a similarity less than 0.8. The user selects 2 realistic variants from the remaining highly similar variants that meet expectations. The system then searches the entire database again, successfully retrieving another 15 real observation images with similar morphologies.

[0050] Cold start filtering: At this point, the user has selected 7 images (less than 30 images). The system uses the Mahalanobis ellipsoid algorithm to filter out interference items in the candidate set that are only similar in color but have incorrect shapes, and prioritizes displaying targets with highly consistent shapes.

[0051] Model Iteration: As users continue labeling, when 30 positive samples have accumulated, the lightweight scoring model updates itself and takes over the ranking based on the positive and negative sample images and the user's ratings of the images. The system not only recommends merged galaxies but also prioritizes subtypes with longer tidal tails based on the user's previous click preferences, achieving personalized alignment.

[0052] Results Export: Users can export a report that directly provides the coordinates and redshift distribution of these galaxies for further scientific research.

[0053] The above description is merely a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. For those skilled in the art, any improvements and modifications made without departing from the principle of the present invention (such as changing the specific feature extraction backbone network, adjusting threshold parameters, etc.) should also be considered within the scope of protection of the present invention.

Claims

1. An astronomical target retrieval system based on cross-modal interaction and generative enhancement, characterized in that, It includes a server for storing and processing astronomical image data, and a user terminal that allows users to input search information and receive feedback. The user terminal accesses a front-end interactive interface through a browser. The front-end interactive interface collects the information input by the user and encapsulates it into a request to be sent to the server. It receives candidate results, variant previews and sorting information returned by the server and displays them visually. The server includes a generative variant module, a cold start screening module, and a human-in-the-loop interaction module. Specifically: the generative variant module generates variant images based on user-annotated samples to expand the candidate sample set when the candidate set coverage is insufficient and / or the number of effective labeled samples is less than a preset condition N1, and extracts features from the generated samples and places them into a visual representation feature library; the cold start screening module obtains an expanded candidate set based on similarity retrieval when the cumulative number of user-annotated samples is less than a preset threshold T1, filters and sorts the expanded candidate set based on statistical distance, and outputs the recommendation results for the cold start phase; the human-in-the-loop interaction module iteratively updates a lightweight scoring model based on user annotation and rating information when the cumulative number of user-annotated samples is greater than or equal to the preset threshold T1, and scores and reorders the candidate results; the server returns the sorted candidate results to the user terminal and receives the annotation and rating information returned by the user terminal for subsequent iterative updates.

2. The astronomical target retrieval system based on cross-modal interaction and generative enhancement according to claim 1, characterized in that, The front-end interactive interface includes a text query input component, an image upload component, a variant generation parameter setting and triggering component, an interest or non-interest annotation and rating component, and a result visualization display component. The user information collected by the front-end interactive interface includes text, images, parameters, and user annotation information for recommended astronomical images.

3. The astronomical target retrieval system based on cross-modal interaction and generative enhancement according to claim 1, characterized in that, The server is set up in a server or cloud computing environment, and the backend of the server is implemented based on a Web service framework. The server also includes a caching module and a task queue module. The caching module is used to cache the retrieval results of recent or frequently queried queries and update and evict them according to a preset expiration time or a least recently used strategy.

4. The astronomical target retrieval system based on cross-modal interaction and generative enhancement according to claim 1, characterized in that, It also includes a retrieval and interaction interface module, a query processing module, a data preprocessing module, a text feature extraction module, an image feature extraction module, a vector retrieval engine, and a data and knowledge base unit. The retrieval and interaction interface module receives retrieval requests from user terminals and user annotations and ratings of candidate images, and returns the retrieval results to the user terminal. The query processing module parses the retrieval mode and retrieval parameters, and schedules data preprocessing, feature extraction, vector retrieval, and reordering processes to generate a candidate set. The data preprocessing module performs standardized preprocessing on the input astronomical image data. The text feature extraction module generates text feature vectors from the user input text using a cross-modal text-image coding model. The image feature extraction module generates image feature vectors from images in the image library, user-uploaded images, and / or generative variant images using a visual representation model; the vector retrieval engine performs similarity retrieval in the cross-modal retrieval feature library and the visual retrieval feature library to obtain a set of candidate results; the data and knowledge base unit stores and maintains the image feature library, text feature library, vector index library, and / or astronomical knowledge base, and provides data access and retrieval call support to the query processing module and the vector retrieval engine.

5. An astronomical target retrieval method based on cross-modal interaction and generative enhancement, implemented using the astronomical target retrieval system based on cross-modal interaction and generative enhancement as described in any one of claims 1 to 4, characterized in that, A cross-modal text-image coding model and a self-supervised visual representation model are used to quantify the query content, resulting in two candidate sets. A retrieval performance index is calculated based on user interest feedback. When the retrieval performance index falls below a preset threshold, variant generation enhancement is automatically triggered. Several image variants with similar morphologies are generated under user-adjustable parameter constraints. Feature consistency screening is performed on these image variants, and those that pass the screening are used to expand the candidate set. After receiving confirmation annotations from the user terminal, the user-confirmed image variants and their corresponding candidate samples are added to the interest sample set. The process switches between a cold-start screening phase and an iterative learning phase based on user feedback, depending on the size of the interest sample set. An updated lightweight scoring model is used to personalize and rearrange the candidate set. Finally, the retrieval results are output and matched with coordinate information from an astronomical knowledge base to generate a retrieval report. The specific steps include: Step 1. Astronomical image data access: fine-tuning the cross-modal text-image coding model and self-supervised visual representation model to construct image library features; the server for storing and processing astronomical image data receives massive amounts of astronomical image data from the sky survey project and constructs the image library feature database. Step 2. Cross-modal text query and image instance query: The text query is vectorized using a cross-modal language-image contrast model and the first candidate set is retrieved. The image query is vectorized using a self-supervised visual representation model and the second candidate set is retrieved. Step 3. Generative variant expansion: For the two candidate sets obtained in Step 2, the astronomical target retrieval system calculates the retrieval performance index based on the user's interest annotations and ratings. When the retrieval performance index is lower than the preset threshold, variant generation enhancement is automatically triggered, and the user is allowed to adjust or confirm the generation parameters. Step 4. Cold Start Phase Screening and Personalized Iterative Reordering of Human-in-the-Loop: The server receives and records the cumulative number of labeled samples for the same type of target by the current user; when the number of samples labeled by the user for the same type of target is less than the first threshold T1, the cold start phase screening is triggered. Neighborhood constraints are constructed based on the statistical distribution of the labeled interest sample set in the feature space, the expanded candidate set is screened and sorted, and the recommended candidate results of the cold start phase are output; when the number of samples labeled by the user for the same type of target reaches or exceeds the preset first threshold T1, a closed-loop optimization process of user feedback - lightweight scoring model update - result reordering - effect display is triggered. Step 5. Knowledge Association and Report Generation: The astronomical target retrieval system extracts the coordinate information of images of interest to the end user, automatically matches the coordinates with the astronomical knowledge base, automatically completes the references for astronomical object type, redshift, and magnitude, and generates a standardized retrieval report containing traceable image annotation information for users to export.

6. The astronomical target retrieval method based on cross-modal interaction and generative enhancement according to claim 5, characterized in that, In step 2, the first candidate set is obtained as follows: Step 2.1A: The server receives the natural language query text entered by the user on the front-end interactive interface and saves the query text to the query log database. Step 2.2A calls the text encoder of the finely tuned language-image contrast pre-trained model to map the natural language text into a high-dimensional text feature vector. The server uses the high-dimensional text feature vector to perform similarity retrieval in the image feature library, generates the first candidate set, and displays it to the user for annotation. The second candidate set is obtained as follows: Step 2.1B: The server receives the local image to be queried uploaded by the user and performs image preprocessing and cropping operations. Step 2.2B calls the unsupervised self-distillation visual representation model to extract the visual feature vector of the image to be queried, calculates the cosine similarity in the visual retrieval feature library, sorts the images by cosine similarity from high to low, selects images with similarity not lower than the preset visual representation similarity threshold T0, selects the top K images with the highest similarity from these images as images with similar morphology, generates the second candidate set and displays it to the user for annotation.

7. The astronomical target retrieval method based on cross-modal interaction and generative enhancement according to claim 6, characterized in that, The preset visual representation similarity threshold T0 ranges from [0.7, 0.9], and the value of K ranges from [10, 100].

8. The astronomical target retrieval method based on cross-modal interaction and generative enhancement according to claim 5, characterized in that, In step 3, the steps for automatically triggering variant generation enhancement and parameter adjustment are as follows: Step 3.1 Variant generation: Based on the preset visual representation similarity threshold T0, the generated variant results are screened for consistency; when the similarity between the variant and the generated seed in the visual representation space meets the preset interval condition, the variant is identified as a slightly different variant, and several variant images are generated; Step 3.2 Candidate Fusion: Extract features from the variant images selected by the user that meet the expectations, set the variants that pass the screening and are confirmed by the user as the images of interest to the user, and perform similarity retrieval in the visual representation feature library as an extended query to obtain an additional candidate set. The additional candidate set is merged with the first candidate set and the second candidate set to form an extended candidate set.

9. The astronomical target retrieval method based on cross-modal interaction and generative enhancement according to claim 5, characterized in that, In step 4, the process of accumulating the number of user-annotated samples is as follows: When the cumulative number of labeled samples is less than the first threshold T1, it is the cold start phase. The server obtains the feature vectors of the currently collected labeled samples, calculates the mean vector μ and covariance matrix Σ of all labeled sample feature vectors, and uses Mahalanobis distance to construct the feature space neighborhood. For each candidate image feature vector x obtained in step 3 in the expanded candidate set, calculate the Mahalanobis distance D between the feature vector of each candidate image and the mean vector μ and covariance matrix Σ of the interest sample set. 2 The calculation formula is: Where x represents the feature vector of the candidate image in the expanded candidate set, μ represents the mean vector of the feature vectors of the interest samples, and Σ -1 Let denote the inverse of the covariance matrix; (·) T It represents the transpose of a vector or matrix, and D represents the Mahalanobis distance scalar value from the feature vector x of the candidate image to the center of the interest distribution; The Mahalanobis distance threshold is determined based on a preset confidence threshold, and a Mahalanobis ellipsoid region is constructed based on the Mahalanobis distance threshold. Candidate images whose Mahalanobis distance satisfies the constraints of the Mahalanobis ellipsoid region are displayed as the preferred recommendation results. The personalized iterative rearrangement operation is as follows: The server automatically collects explicit user feedback and user-annotated image ratings, storing the data in an online learning sample pool. It then reads feature vectors corresponding to interest / non-interest samples from the pool as input to a lightweight scoring model. Based on user ratings, it normalizes the interest / non-interest samples to obtain sample weights. A binary classification cross-entropy loss is used, and the total loss is calculated by weighting the samples according to their weights, thus updating the lightweight scoring model. During the recommendation phase, the updated lightweight scoring model is used to re-score and rank the expanded candidate set. The server randomly selects a portion of images, scores and ranks them using the fine-tuned lightweight scoring model, and displays the results for users to intuitively judge the personalized recommendations. Once the user is satisfied with the recommendation results, the server saves the lightweight scoring model parameters for the current user, generating a directly callable personalized recommendation version suitable for the current user, which is then used for subsequent batch filtering tasks.