Multi-source heterogeneous data fusion method and system based on rag health care big model

Abstract

The present application belongs to the technical field of wisdom health care, and particularly relates to a multi-source heterogeneous data fusion method and system based on a RAG health care large model, which comprises the following steps: obtaining multi-source original data of a health care institution and dividing the data into data segments, and adding metadata containing abnormal labels to form a multi-source slice library; establishing a connection edge based on the same object and the condition that the time window interval is within the correlation time range, combining time closeness, semantic complementarity and cross-modal indication state to calculate edge weight and construct a cross-modal complementary graph; receiving a query text to recall seed slices, diffusing along the cross-modal complementary graph and weighting and sorting to obtain target evidence slices; and organizing the target evidence slices into three-column context blocks and injecting the blocks into a large model to generate a query answer. The present application reduces the same-modal semantic occupation problem, and enables the generated answer to have a traceable cause-and-effect chain.

Description

Technical Field

[0001] This invention relates to the field of smart healthcare technology. More specifically, this invention relates to a method and system for fusing multi-source heterogeneous data based on the RAG healthcare big data model. Background Technology

[0002] Retrieval-enhanced generative (RAG)-based elderly care services are a comprehensive service system for the elderly and patients with chronic diseases. Its execution process generates a large amount of multi-source, heterogeneous data, including physiological signs, behavioral trajectories, medication records, and nursing care work orders. To achieve intelligent elderly care monitoring, existing technologies typically employ a large-scale elderly care model architecture based on RAG, uniformly feeding various structured vital sign data and unstructured text data into a general embedding model to generate vectors and writing them into a unified vector library. During the retrieval phase, the system recalls relevant fragments based on the semantic similarity of the user's query, incorporates prompt words, and sends them to the large model to generate nursing suggestions. Due to the differences in the semantic spaces of different modalities, the aforementioned direct mixing and vectorization method leads to text-based log data having an advantage in semantic retrieval, while numerical vital sign data is easily marginalized. This makes it difficult for the retrieval results to take into account multi-dimensional elderly care information, limiting the model's ability to output comprehensive and personalized suggestions.

[0003] To address the modality imbalance issue encountered in heterogeneous data retrieval, existing technologies have introduced improved techniques that incorporate modality tags as filtering fields and train separate modality adaptation layers. This improved approach forcibly binds corresponding modality attribute identifiers to each data segment during data entry. In the retrieval and recall phase, in addition to calculating basic query semantic similarity, it further filters the recalled segments by introducing pre-defined modality weights and performing a weighted summation. This attempt, through manual intervention or specific training weight allocation logic, forces the retrieval results to contain a certain proportion of data from different modalities, aiming to achieve coarse-grained fusion of multi-source data during retrieval, thereby providing a seemingly more balanced and richer multi-dimensional contextual reference background for large models.

[0004] Improved techniques rely on semantic queries as anchors for similarity-based recall during retrieval, supplemented by fixed modal weights for allocation. However, in real-world healthcare scenarios, multi-source data generated by the same patient at similar times often exhibit strong causal complementarity rather than strong semantic similarity. This method, based on simple semantic similarity matching and static weight allocation, fails to uncover the underlying spatiotemporal inference and causal relationship between abnormal vital sign fluctuations and subsequent nursing interventions. Consequently, during retrieval ranking, semantically adjacent same-modal segments still dominate the limited priority recall slots due to their extremely high direct similarity scores, while cross-modal complementary segments with significantly different semantic features but crucial corroboration in the core nursing causal chain are buried by the underlying retrieval algorithm. This results in large models, even when receiving context pieced together by weighted rules, only passively restating the surface-level homogeneous information, losing the ability to trace back the cross-modal causal evidence chain. Ultimately, in high-value inquiries involving warning interpretation and medication recommendations, the outputs conclusions detached from the actual logic of medical care. Summary of the Invention

[0005] To address the technical problem that existing systems lose cross-modal complementary evidence due to semantic crowding during the retrieval phase, resulting in generated answers deviating from the true causal chain, this invention provides a multi-source heterogeneous data fusion method and system based on the RAG health and wellness big data model.

[0006] In a first aspect, the present invention provides a multi-source heterogeneous data fusion method based on the RAG (Rapid Aging and Healthcare) big data model, comprising: S1, acquiring multi-source raw data from healthcare institutions and segmenting it into data fragments, attaching metadata containing anomaly markers to the data fragments, converting the data fragments into semantic vectors and writing them into a vector library, and simultaneously writing the metadata into a relational index to form a multi-source slice library; S2, for the data fragments in the multi-source slice library, establishing connection edges based on the condition that they are the same object and the time window interval is within the associated time history, and combining time proximity, semantic complementarity, and cross-modal indication status. S3. Calculate edge weights to construct a cross-modal complementarity graph; S4. Receive query text and convert it into query vectors. Recall seed slices from the vector library. Starting from the seed slices, expand along the cross-modal complementarity graph to obtain a candidate evidence pool. Then, perform weighted sorting based on the semantic relevance of the query vectors and the candidate slices in the candidate evidence pool, as well as the edge weights, to obtain the target evidence slices; S5. Organize the target evidence slices into three-column context blocks according to the time axis backbone and cross-modal side branches. Inject the three-column context blocks and query text into the large model to generate query answers with evidence numbers and time anchors.

[0007] This invention constructs a cross-modal complementarity graph with spatiotemporal co-occurrence and modal complementarity as edge weights during the slice entry stage. During retrieval, target evidence slices are obtained based on graph diffusion and weighted sorting. Subsequently, a large-scale model query is performed by combining the context of the time axis backbone and cross-modal side branches. This reduces the problem of semantically adjacent fragments of the same modality crowding out the retrieval recall quota, and allows cross-modal complementary fragments with causal proof function to be retained, so that the generated answer has a traceable and complete cross-modal causal chain.

[0008] Preferably, the step of acquiring multi-source raw data from elderly care institutions and segmenting it into data fragments includes: receiving structured time-series data and unstructured text data from elderly care institutions; segmenting the structured time-series data into equal-length fragments according to a preset sliding window, and extracting numerical summaries and trend descriptions; segmenting the unstructured text data into semantically complete fragments according to event units; and summarizing the equal-length fragments and semantically complete fragments to obtain data fragments.

[0009] Preferably, the step of attaching metadata containing anomaly markers to the data segment includes: extracting the object identifier and modality category corresponding to the data segment; calculating the median and interquartile range of the object identifier in the same modality category within a preset historical time period to obtain a dynamic baseline; determining whether the value of the data segment exceeds the interquartile range multiple threshold corresponding to the dynamic baseline; if it exceeds the interquartile range multiple threshold, setting the anomaly marker to a valid state to obtain metadata containing the anomaly markers.

[0010] This invention uses the historical median and interquartile range of a specific object to calculate a dynamic baseline to determine abnormal states, avoiding a large number of false positives and false negatives across individuals caused by applying a general population threshold. This ensures that the slices on which the subsequent complementary map construction and retrieval diffusion depend have accurate and traceable structured benchmark anchors.

[0011] Preferably, the step of calculating edge weights by combining time proximity, semantic complementarity, and cross-modal indication state includes: calculating the time difference between the event center times of two data segments, and subtracting the ratio of the time difference to the associated time history from 1 to obtain the time proximity; calculating the cosine similarity of the semantic vectors of the two data segments, and subtracting the cosine similarity from 1 to obtain the semantic complementarity; determining whether the modal categories of the two data segments are the same, and if they are the same, setting the cross-modal indication state to 0, and if they are different, setting the cross-modal indication state to 1; and multiplying the time proximity, semantic complementarity, and cross-modal indication state to obtain the edge weights.

[0012] This invention calculates the product of the temporal proximity of the event center moment, the complementarity of semantic vectors, and the cross-modal indication state. This liberates the causal relationship between cross-modal events due to temporal and content complementarity from the implicit semantics of the embedding space and encodes it as explicit edge weights, providing high-confidence cross-modal navigation guidance for subsequent graph diffusion.

[0013] Preferably, the establishment of connection edges based on the condition of the same object and the time window interval being within the associated time period includes: extracting adjacent record pairs of abnormal physical signs records and nursing intervention records under the same object identifier in the relational index; calculating the timestamp difference sequence of adjacent record pairs; extracting the median of the timestamp difference sequence as the associated time period; determining whether the time window interval of the data segment is less than or equal to the associated time period, and if so, establishing connection edges based on the condition of the same object and the time window interval being within the associated time period.

[0014] Preferably, the step of obtaining the candidate evidence pool by diffusion along the cross-modal complementary graph starting from the seed slice includes: reading the anomaly marker in the metadata of the seed slice; if the anomaly marker is in a valid state, performing two-hop diffusion along the cross-modal complementary graph starting from the seed slice; if the anomaly marker is in an empty state, performing one-hop diffusion along the cross-modal complementary graph starting from the seed slice; and collecting the cross-modal neighbor slices reached by the diffusion to obtain the candidate evidence pool.

[0015] This invention dynamically drives the diffusion hop count of complementary maps by reading the abnormal markers of seed slices, without the need to introduce complex judgment calculations in real time. While ensuring sufficient backtracking causal chains to deal with abnormal situations, it can effectively control the context length under normal data conditions, achieving a good balance between retrieval depth and efficiency.

[0016] Preferably, the weighted ranking based on the semantic relevance and edge weights of the question vector and candidate slices in the candidate evidence pool includes: counting the number of modal categories of seed slices and the total number of seeds; calculating the ratio of the two when the total number of seeds is greater than 1 to obtain the modal dispersion; and directly setting the modal dispersion to 0 if the total number of seeds is 1; calculating the balance coefficient through linear mapping based on the modal dispersion and a preset balance coefficient calibration interval; calculating the semantic relevance between the question vector and the semantic vector of the candidate slice; calculating the average edge weight contribution of the candidate slice to all seed slices; and using the balance coefficient to perform a weighted summation of the semantic relevance and the average edge weight contribution to obtain a comprehensive score for ranking, thus obtaining the weighted ranking result.

[0017] This invention calculates the balance coefficient by linear mapping based on the modal dispersion of seed slices, and adaptively weights the semantic relevance and average edge weight contribution. This provides a smooth weight transition between single-modal focused queries and multi-modal comprehensive queries, avoiding extreme situations where the graph contribution is too strong or too weak in different query scenarios due to a single fixed weight allocation strategy.

[0018] Preferably, the step of organizing the target evidence slices into a three-column context block according to the time axis backbone and cross-modal side branches includes: arranging the target evidence slices in ascending order of event time to form the time axis backbone; for each backbone node on the time axis backbone, extracting the cross-modal neighbor slice with the highest edge weight from the cross-modal complementary graph as a cross-modal side branch; and combining the time, main event, and cross-modal corroboration to finally obtain the three-column context block.

[0019] Preferably, after generating the query answer with evidence number and time anchor, the method further includes: collecting adoption feedback records for the query answer; writing the query text, seed slice, target evidence slice, query answer and adoption feedback records as new unstructured slices into the multi-source slice library; and updating the association time history and balance coefficient calibration interval based on the adoption feedback records to obtain the updated multi-source slice library and association parameters.

[0020] Secondly, the present invention provides a multi-source heterogeneous data fusion system based on the RAG health and wellness big model, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned multi-source heterogeneous data fusion method based on the RAG health and wellness big model is implemented.

[0021] The technical solution of the present invention has the following beneficial technical effects:

[0022] This invention addresses the issue of semantic relevance affecting the acquisition of cross-modal neighbor slices in data fragments from a multi-source slice library. It establishes connection edges based on the condition that the data fragments are of the same object and the time window interval is within the associated time history. It also combines time proximity, semantic complementarity, and cross-modal indication state to calculate edge weights and construct a cross-modal complementary graph. This solves the problem that the semantic relevance of data of the same modality category affects the acquisition of cross-modal neighbor slices.

[0023] The system receives query text and converts it into query vectors. Seed slices are retrieved from a vector library. Starting from the seed slices, a candidate evidence pool is obtained by diffusion along a cross-modal complementary graph. The target evidence slice is then obtained by weighted sorting based on the semantic relevance and edge weights between the query vectors and the candidate slices in the pool. This method of diffusion along a cross-modal complementary graph, combined with weighted sorting based on semantic relevance and edge weights, effectively solves the problem of difficulty in obtaining neighboring slices with cross-modal relationships to the seed slice when retrieving data fragments based on semantic relevance.

[0024] The target evidence slices are organized into three-column context blocks according to the timeline main branch and cross-modal side branches. The three-column context blocks and the query text are then injected into the large model to generate query answers with evidence numbers and time anchors. By organizing the three-column context blocks according to the timeline main branch and cross-modal side branches and injecting them into the large model, the problem of the large model not combining the timeline main branch and cross-modal side branches when generating query answers is solved, thereby improving the evidence traceability of the query answers and the completeness of cross-modal causal explanation. Attached Figure Description

[0025] Figure 1 This is a flowchart of a multi-source heterogeneous data fusion method based on the RAG health and wellness big model; Figure 2 This is a comparison chart showing the trend of the same modal semantic crowding rate accumulated with the number of query rounds in the embodiments of the present invention. Detailed Implementation

[0026] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0027] This invention discloses a multi-source heterogeneous data fusion method based on the RAG health and wellness big data model, referring to... Figure 1 This includes the following steps: S1. Obtain the raw data and segment it to form a multi-source slice library.

[0028] In an optional embodiment, on the data access side of the elderly care institution, using the elderly person's ID as the aggregation key, raw data from multiple sources, including wearable devices, smart mattresses, environmental sensors, electronic nursing records, medical order systems, and family communication records, are uniformly incorporated into the collection pipeline. For structured time-series data such as vital signs, behaviors, and environments, the continuous stream is divided into segments of equal duration according to a preset sliding window; for example, the sliding window duration is 5 minutes. For unstructured text data such as nursing logs, medical orders, and communication records, the data is divided into semantically complete segments based on events, ensuring that each text segment does not truncate the complete description of the nursing event at the boundary. The segments of equal duration and semantically complete segments are then aggregated to obtain the final data segments. In addition to retaining the original content, each data segment is uniformly appended with five types of metadata fields: elderly person's ID, modality category, event start and end timestamps, occurrence location, and anomaly marker, providing physical anchors for subsequent bidirectional tracing by elderly person's ID and timeline.

[0029] Because directly concatenating structured numerical data into a general embedding model in a conversational manner dilutes the physical meaning of the numerical values ​​with a natural language template, this invention extracts numerical summaries and trend descriptions from structured slices, retaining a standardized expression that combines numerical summaries, trend descriptions, and anomaly markers in sequence. The trend description is obtained by performing sign statistics on the first-order difference of the sequence within the window, specifically outputting one of four labels: rising, falling, stable, or oscillating. Anomaly marking is triggered based on the individual's own historical baseline deviation over the same period, rather than applying a general threshold for the population. Specifically, the object identifier and modality category corresponding to the data segment are extracted. The median and interquartile range of the data of the same modality category at the same moment within a preset historical time period are calculated to obtain a dynamic baseline, which is updated daily with new data. For example, the historical time period is the last 30 days. It is then determined whether the value of the data segment exceeds the interquartile range multiple threshold corresponding to the dynamic baseline. If it does, the anomaly mark is set to valid, and metadata containing the anomaly mark is obtained. Otherwise, the anomaly mark is set to empty. For example, the interquartile range multiple threshold is 1.5, which is the standard threshold for identifying mild outliers in statistics. If sensitivity is to be increased, it can be adjusted to 1.2; if specificity is to be increased, it can be adjusted to 2. Because the anomaly determination is anchored to the individual's own historical rhythm rather than the population mean, nighttime heart rate will lead to drastically different anomaly marking conclusions for elderly people with long-term high heart rates and those with stable heart rates, thus avoiding a large number of false positives and false negatives caused by applying thresholds across individuals.

[0030] Furthermore, after the expression standardization is completed, all data fragments are uniformly converted into semantic vectors and written into the vector library, while the metadata is written into a parallel relational index. It should be noted that the specific choice of the embedding model is unrestricted; those skilled in the art can choose different pre-trained models based on the institution's computing power and language style. Similarly, the specific implementation of the relational index is unrestricted; any database solution that supports joint queries by elderly person ID and time interval can be adapted. This forms a multi-source slice library with unified spatiotemporal coordinates and bidirectional traceability by elderly person ID and time axis, providing a physical foundation for subsequent cross-modal complementary retrieval.

[0031] Thus, through a unified spatiotemporal coordinate system and individualized anomaly markers, the multi-source slice library is equipped with bidirectional traceability capabilities, laying a structured foundation for subsequent map construction and evidence retrieval.

[0032] S2. Calculate edge weights to construct a cross-modal complementary graph.

[0033] In an optional embodiment, while the slices are being added to the database, an additional cross-modal complementary graph is constructed for the data fragments in the multi-source slice database. This graph uses slices as nodes and cross-modal spatiotemporal co-occurrence intensity as edge weights, explicitly encoding the physical fact that the same elderly person co-occurred within similar time windows into a searchable structure. For any two data fragments... and A connection edge is established based on the condition that the two belong to the same elderly person ID and the time window interval is within the associated time period. If a slice pair falls within a certain range, an edge is created in the graph; otherwise, no edge is created, ensuring that edge weights are always calculated within a meaningful temporal correlation. Edge weights are calculated by combining temporal proximity, semantic complementarity, and cross-modal indication states. The edge weight reflects both the spatiotemporal proximity and modal complementarity.

[0034] To overcome the technical shortcomings of existing technologies that rely solely on semantic similarity to construct graphs, which can easily lead to the interconnection of homogeneous data, this invention constructs a relational formula for calculating edge weights in the context of health and wellness business scenarios. Conventional graph algorithms tend to assign higher edge weights to semantically similar nodes. However, in the health and wellness scenario, data generated by different modalities have extremely low semantic similarity but strong causal complementarity. Therefore, this invention introduces temporal proximity to ensure the temporal correlation of physical events, introduces the complement of cosine similarity as semantic complementarity to reward information complementarity between different modalities, and adds cross-modal indication state as a hard filtering condition, specifically: ; in, Let the edge weight be the boundary weight between the two data segments. and These represent the event center times for the two data segments; For the associated time history; and These are the semantic vectors of the two data segments, The cosine similarity between the semantic vectors of the two slices; This indicates the state across modes.

[0035] The first metric is temporal proximity, calculated by the time difference between the event center moments of two data segments and determined by the ratio of this time difference to the associated timeline. The closer the spatiotemporal proximity, the more credible the physical association, and the larger this value. The second metric is semantic complementarity, represented by the complement of the cosine similarity. That is, the lower the cosine similarity, the higher the complementarity. The reward mechanism is semantic complementarity; the cross-modal indicator state determines whether two data segments have the same modal category. If they do, the cross-modal indicator state is set to 0; otherwise, it is set to 1. Since the relationship between segments of the same modality is already fully expressed by the native vector similarity, the cross-modal indicator state is further optimized through... Hard-setting the edge weights to zero ensures that the graph focuses on carrying cross-modal complementary semantics. The edge weights are then obtained by multiplying the temporal proximity, semantic complementarity, and cross-modal indication state.

[0036] Among them, the associated time history The initial value is not set based on human experience, but is objectively calculated from the median of the time difference between records of abnormal vital signs and corresponding nursing intervention records in all nursing logs of the institution over the past 90 days. In the relational index, adjacent record pairs of records with the same object identifier are extracted; the timestamp difference sequence of adjacent record pairs is calculated; the median of the timestamp difference sequence is extracted as the association time series; it is determined whether the time window interval of the data segment is less than or equal to the association time series. If so, the final result is a connection edge established based on the same object and the time window interval is within the association time series. This value reflects the objective statistical law of the institution's nursing response rhythm and directly corresponds to the institution's size and scheduling pattern, rather than a fixed constant applicable across institutions.

[0037] Next, the actual calculation process of the edge weights will be explained using an elderly person's nighttime period as an example. For instance, assume that the time difference between the vital sign slice (heart rate rising from 72 to 105 at 02:15) and the behavioral slice (the sensor triggering three times at 02:14) is 1 minute. Taking a preset value, let's assume it's 20 minutes, and setting the similarity between the two vectors to 0.12, the edge weight calculation result is: The edge weights are higher than those between adjacent heart rate segments belonging to the same vital sign modality. Since the cross-modal indication state is zero when the modality is the same, the edge weights become zero, thus explicitly enhancing the cross-modal causal relationship in the graph. Simultaneously, if the same elderly person had a medical order to adjust their antihypertensive medication dosage three days prior, and the time difference between that slice and the vital sign slice exceeds [a certain value], [further details needed]. Then no edge is built, but if the time difference is within Within the range, due to different modalities and semantic distance, higher edge weights are obtained, thereby explicitly precipitating the causal relationship between drug adjustment events and subsequent vital sign fluctuations into the graph.

[0038] Furthermore, all edge weights are incrementally calculated and persisted when a slice is added to the database, forming a complementary graph index that runs parallel to the vector database. Whenever a new slice is written, the system only calculates the edge weights of that slice and the existing slices within the time window and appends them to the graph, without needing to recalculate the entire graph, thus ensuring the efficiency of incremental updates in high-frequency data writing scenarios.

[0039] In this way, by simultaneously encoding spatiotemporal proximity and modal complementarity into explicit edge weights, cross-modal causal relationships are liberated from the implicit representation of the embedded space, forming a structured graph index that can be directly used for retrieval of diffusion.

[0040] S3. Diffusion and weighted sorting are used to obtain the target evidence slices.

[0041] In an optional embodiment, when a caregiver or family member initiates a question, the question text is received and converted into a question vector, and the retrieval process is reconstructed into a three-stage process of semantic anchoring, graph diffusion, and evidence rearrangement, thereby breaking the homomodal semantic crowding at the process level.

[0042] The first stage is semantic anchoring: after embedding the question text, several seed slices with the highest semantic similarity are retrieved from the vector library. The seeds are only used to determine the elderly person, time focus and topic modality of the question, and are not directly used as the final context; for example, the number of seed slices is 3 to 5.

[0043] The second stage is graph diffusion: starting from each seed slice, a one-hop or two-hop diffusion is performed along the constructed cross-modal complementary graph according to edge weights, collecting the cross-modal neighbor slices reached by the diffusion, and finally obtaining the candidate evidence pool. The diffusion depth is directly driven by the anomaly marker field in the seed slice metadata: the anomaly marker in the seed slice metadata is read; if the anomaly marker is valid, that is, the interquartile range deviation of the historical baseline of the elderly person in the same period has been triggered, then a two-hop diffusion is performed along the cross-modal complementary graph starting from the seed slice to fully trace the causal chain; if the anomaly marker is empty, then a one-hop diffusion is performed along the cross-modal complementary graph starting from the seed slice to control the context length. Since the anomaly marker is accurately labeled according to the individual baseline during the database entry stage, the dynamic switching of diffusion depth does not require additional real-time judgment and calculation, and can be completed by directly reading the metadata field.

[0044] The third stage is evidence rearrangement: Based on the semantic relevance and edge weights of the question vector and candidate slices in the candidate evidence pool, a weighted ranking is performed to obtain the target evidence slice. Existing technologies typically only score the semantic similarity between the question and the candidate slice, with the corresponding formula being... The scoring relation of this invention incorporates the edge weight contribution of complementary graphs: ; in, For the scoring results, Let be the query vector. For candidate slice vectors, A collection of seed slices. The number of elements in the seed set. Slice the seeds in the set. For candidate slices, For balance coefficient, Slice the seeds With candidate slices Edge weights in complementary graphs The average edge weight contribution of the candidate slice to all seed slices, and same place Interval.

[0045] Balance coefficient Instead of using fixed empirical values, the modal dispersion is objectively determined based on the seed pool's own modal dispersion. The modal dispersion is obtained by comparing the number of modal categories in the seed slice with the total number of seeds, and calculating the ratio of these two values ​​when the total number of seeds is greater than 1. If the total number of seeds is 1, then directly... Set to 0. Based on the modal dispersion and the preset equilibrium coefficient calibration interval, the equilibrium coefficient is calculated through linear mapping. : ; in, and Take the question corresponding to the lowest preset quantile percentage of adoption rate among nursing staff from the historical questionnaire samples of this institution. The mean of the distribution and the corresponding preset quantile proportion of the highest adoption rate in the query. The mean of the distribution is objectively determined by the institution's historical adoption feedback data and is updated continuously as feedback accumulates, without relying on human experience; for example, the preset quantile ratio is 10%. The semantic relevance between the question vector and the semantic vector of the candidate slice is calculated, and the average side-weight contribution of the candidate slice to all seed slices is calculated. A weighted sum of the semantic relevance and average side-weight contribution is obtained using a balance coefficient to obtain a comprehensive score for ranking, thus yielding the target evidence slice. When all seed slices come from the same modality... Approaching 0, Approaching The contribution ratio of graph edge weights is amplified; when the seed has spanned multiple modalities, Approaching 1, Approaching Semantic similarity has regained its dominant position. This contrasts with applying a uniform, fixed approach to all queries. This adaptive calibration method provides a smooth weight transition between single-modal focused queries and multimodal comprehensive queries, avoiding extreme cases where the graph contribution is too strong or too weak in each type of query.

[0046] In this way, through the three-stage process of semantic anchoring, graph diffusion, and evidence rearrangement, cross-modal complementary slices that were originally squeezed out and submerged by same-modal semantics are systematically brought to the surface as high-confidence evidence, providing a reliable candidate set for subsequent context organization.

[0047] S4. Organize a three-column context block and generate the query answer.

[0048] In an optional embodiment, the target evidence slices are organized into a three-column context block according to the timeline backbone and cross-modal side branches. Specifically, the target evidence slices are arranged in ascending order of event time to form the timeline backbone; for each backbone node on the timeline backbone, one or two cross-modal neighbor slices with the highest edge weights are extracted from the cross-modal complementary graph as cross-modal side branch descriptions; the time, main event, and cross-modal corroboration are combined to finally obtain the three-column context block.

[0049] Next, the three-column context block and the query text are injected into the large model to generate query answers with evidence numbers and time anchors. The prompt template explicitly instructs the large model to: trace back events along the timeline when answering; label each conclusion with the slice number on which it is based; and present conflicting cross-modal evidence in parallel rather than accepting one side unilaterally. Because the correspondence between the main timeline and cross-modal branches is clearly presented when organizing the context, the large model can directly output causal reasoning with time anchors along this structure when generating answers, rather than restating conclusions after indiscriminately splicing semantically adjacent segments. The answers generated by the large model naturally carry evidence numbers and time anchors, allowing caregivers to jump back to the original slice for verification with a single click using the number, thus avoiding the answer degradation that occurs when traditional RAGs pile up similar evidence of the same modality.

[0050] Furthermore, after generating query answers with evidence numbers and time anchors, adoption feedback records for the query answers are collected. The query text, seed slice, target evidence slice, query answers, and adoption feedback records are then written into the multi-source slice library as new unstructured slices, entering the slice storage process. Based on the adoption feedback records, the associated time history and the balance coefficient calibration interval are updated on a rolling basis, ultimately resulting in the updated multi-source slice library and associated parameters. The adoption or non-adoption by nursing staff serves as an implicit monitoring signal for rolling updates. The value of and , Specifically, if the recent adoption rate consistently falls below the organization's historical average, the system triggers a re-stratification and statistical analysis of historical query samples, re-extracting the preset quantiles at both ends of the adoption rate. The mean, replacing the current value. and ; The rolling updates continuously incorporate the time lag sequence from abnormal signs to nursing intervention in the latest nursing logs, ensuring that the median estimate keeps pace with the evolution of the institution's scheduling patterns.

[0051] It should be noted that there are many other ways to achieve evidence organization and prompt injection in this field. Those skilled in the art can choose different context splicing formats and prompt engineering schemes according to the actual deployment of large model interface protocols and institutional information system conditions, without affecting the core logic of cross-modal evidence chain organization of this invention.

[0052] In this way, by organizing the target evidence slices into three-column context blocks that combine the timeline backbone with cross-modal side branches, and by using cue words that force the annotation of evidence numbers and time anchors, the query answers generated by the large model have complete cross-modal causal backtracking capabilities.

[0053] like Figure 2 As shown, the horizontal axis represents the number of query rounds, and the vertical axis represents the same-modal semantic crowding rate, ranging from 0 to 1. In the figure, the curve marked with a black dashed circle represents the existing method, i.e., the pure semantic similarity matching scheme, while the curve marked with a black solid square represents the method of this invention, i.e., the scheme of semantic anchoring, graph diffusion, and dynamic weighted evidence rearrangement. The gray dotted area between the two curves represents the difference in crowding rates between the two methods. It can be observed that the same-modal semantic crowding rate of the existing method remains at a high level even after increasing the number of query rounds, and converges slowly; the crowding rate of the method of this invention decreases to a lower range and tends to stabilize after fewer query rounds, indicating that this invention suppresses the occupation of the candidate evidence pool by same-modal fragments through graph diffusion and evidence rearrangement, allowing cross-modal evidence to continuously emerge.

[0054] This invention also discloses a multi-source heterogeneous data fusion system based on the RAG health and wellness big model, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, a multi-source heterogeneous data fusion method based on the RAG health and wellness big model according to the present invention is implemented.

[0055] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

Claims

1. A method for fusing multi-source heterogeneous data based on the RAG health and wellness large-scale model, characterized in that, include: S1. Obtain multi-source raw data from health and wellness institutions and segment it into data fragments. Attach metadata containing anomaly markers to the data fragments and convert the data fragments into semantic vectors and write them into a vector library. Simultaneously, write the metadata into a relational index to form a multi-source slice library. S2. For data fragments in the multi-source slice library, establish connection edges based on the condition that the same object and the time window interval is within the associated time history, and calculate edge weights by combining time proximity, semantic complementarity and cross-modal indication state to construct a cross-modal complementary graph. S3. Receive the query text and convert it into a query vector. Recall the seed slice in the vector library. Starting from the seed slice, spread along the cross-modal complementary graph to obtain the candidate evidence pool. Then, perform weighted sorting based on the semantic relevance and edge weight of the query vector and the candidate slices in the candidate evidence pool to obtain the target evidence slice. S4. Organize the target evidence slices into three-column context blocks according to the time axis main branch and cross-modal side branches, and inject the three-column context blocks and the query text into the large model to generate query answers with evidence numbers and time anchors.

2. The multi-source heterogeneous data fusion method based on the RAG health and wellness large model according to claim 1, characterized in that, The process of acquiring and segmenting multi-source raw data from elderly care institutions into data fragments includes: receiving structured time-series data and unstructured text data from elderly care institutions; For structured time-series data, the data is divided into segments of equal duration according to a preset sliding window, and numerical summaries and trend descriptions are extracted. For unstructured text data, it is segmented into semantically complete fragments based on event units; By combining segments of equal duration with segments of complete semantics, data segments are obtained.

3. The multi-source heterogeneous data fusion method based on the RAG health and wellness large model according to claim 1, characterized in that, The process of attaching metadata containing anomaly markers to data fragments includes: extracting the object identifier and modality category corresponding to the data fragment; The median and interquartile range of the data of the same modal category of the object identifier within a preset historical time period are calculated to obtain the dynamic baseline; Determine whether the value of the data segment exceeds the interquartile range threshold corresponding to the dynamic baseline; If the threshold exceeds the interquartile range multiple, the anomaly flag is set to valid, and metadata containing the anomaly flag is obtained.

4. The multi-source heterogeneous data fusion method based on the RAG health and wellness large model according to claim 1, characterized in that, The method of combining time proximity, semantic complementarity, and cross-modal indication state to calculate edge weights includes: calculating the time difference between the event center times of two data segments, and subtracting the ratio of the time difference to the associated time history from 1 to obtain the time proximity. Calculate the cosine similarity of the semantic vectors of two data segments, and subtract the cosine similarity from 1 to obtain the semantic complementarity. Determine whether the modal categories of two data segments are the same. If they are the same, set the cross-modal indicator status to 0; otherwise, set the cross-modal indicator status to 1. The edge weights are obtained by multiplying the temporal proximity, semantic complementarity, and cross-modal indication state.

5. The multi-source heterogeneous data fusion method based on the RAG health and wellness large model according to claim 1, characterized in that, The establishment of connection edges based on the same object and time window intervals within the associated time period includes: extracting adjacent record pairs of abnormal physical signs records and nursing intervention records under the same object identifier in the relational index; Calculate the timestamp difference sequence of adjacent record pairs; The median of the timestamp difference sequence is extracted as the associated time history; Determine whether the time window interval of the data segment is less than or equal to the associated time schedule. If so, establish a connection edge based on the condition that the data segment is the same object and the time window interval is within the associated time schedule.

6. The multi-source heterogeneous data fusion method based on the RAG health and wellness large model according to claim 1, characterized in that, The process of obtaining a candidate evidence pool by diffusing along the cross-modal complementary spectrum starting from a seed slice includes: reading anomaly markers from the metadata of the seed slice; If the anomaly is marked as a valid state, then two-hop diffusion is performed along the cross-modal complementarity map starting from the seed slice; If the anomaly is marked as empty, then a one-hop diffusion is performed along the cross-modal complementarity map starting from the seed slice; Collect the cross-modal neighbor slices that have diffused to obtain the candidate evidence pool.

7. The multi-source heterogeneous data fusion method based on the RAG health and wellness large model according to claim 1, characterized in that, The weighted sorting based on the semantic relevance and edge weight of the question vector and the candidate slices in the candidate evidence pool includes: counting the number of modal categories of the seed slice and the total number of seeds; calculating the ratio of the two when the total number of seeds is greater than 1 to obtain the modal dispersion; and setting the modal dispersion to 0 directly when the total number of seeds is 1. Based on the modal dispersion and the preset balance coefficient calibration interval, the balance coefficient is calculated through linear mapping; the semantic relevance between the question vector and the semantic vector of the candidate slice is calculated. Calculate the average edge weight contribution of the candidate slice to all seed slices; The semantic relevance and average edge weight contribution are weighted and summed using a balance coefficient to obtain a comprehensive score for ranking, resulting in a weighted ranking.

8. The multi-source heterogeneous data fusion method based on the RAG health and wellness large model according to claim 1, characterized in that, The step of organizing the target evidence slices into a three-column context block according to the time axis trunk and cross-modal side branches includes: arranging the target evidence slices in ascending order of event time to form the time axis trunk; For each main node on the time axis, the cross-modal neighbor slice with the highest edge weight is extracted from the cross-modal complementary graph as a cross-modal side branch; By combining the time frame, the main event, and cross-modal corroboration, a three-column context block is obtained.

9. The multi-source heterogeneous data fusion method based on the RAG health and wellness large model according to claim 1, characterized in that, After generating the query answer with evidence number and time anchor, the method also includes: collecting the adoption feedback record of the query answer; The query text, seed slice, target evidence slice, query answer and adoption feedback record are written as new unstructured slices into the multi-source slice library; Based on the rolling updates of the adoption feedback records and the calibration interval of the balance coefficient, the updated multi-source slice library and association parameters are obtained.

10. A multi-source heterogeneous data fusion system based on the RAG health and wellness big data model, characterized in that, include: A processor and a memory, wherein the memory stores computer program instructions that, when executed by the processor, implement the multi-source heterogeneous data fusion method based on the RAG health and wellness big model according to any one of claims 1-9.

Images