An intelligent data processing method and system supporting multi-source heterogeneous data

Abstract

This invention proposes an intelligent data processing method and system supporting multi-source heterogeneous data. The method includes: aggregating structured data from at least two business systems to construct a data snapshot unit containing task identifiers, time windows, and organizational sets; standardizing and semantically labeling fields, calculating semantic deviations based on historical task averages, generating role strength values ​​representing the influence of fields, and forming semantic fragments; matching task category templates based on semantic fragments to identify target tasks and construct task structures; constructing a causal connection graph based on causal templates, extracting the path with the highest causal strength as the causal chain, and generating suggestion confidence; finally, generating structured strategy suggestions based on the causal chain and a preset mapping template, and converting them into executable instructions for the target business system. This invention achieves closed-loop generation from multi-source data to interpretable and executable strategies, improving strategy accuracy and system adaptability.

Description

Technical Field

[0001] This invention belongs to the field of data processing, and in particular relates to an intelligent data processing method and system that supports multi-source heterogeneous data. Background Technology

[0002] As the digitalization of sectors such as government affairs, education, transportation, and energy continues to increase, the structured business data generated between systems is growing on a massive scale. Although this data mostly exists in structured forms such as tables, logs, and reports, its origins from different organizations, business systems, and task stages lead to widespread issues such as inconsistent field naming, inconsistent indicator meanings, asynchronous collection cycles, and fragmented task contexts, forming a typical "structured multi-source heterogeneous data" pattern. In actual business processing, due to the lack of a unified structure expression mechanism across organizations and time periods, data uploaded from various source systems is often only usable within their own organization, making it difficult to support collaborative analysis across systems and departments. Furthermore, even if the format is standardized at the field level, the business meaning behind each data field still varies significantly. For example, a field named "completion rate" may have different meanings and calculation methods in different schools and departments, making it difficult for upper-level business systems to use it directly. Furthermore, existing systems typically process structured data at the level of rule validation or simple statistics, lacking the ability to automatically generate semantic expressions based on field context, identify task objectives, and construct causal chains. Therefore, they cannot support the closed-loop applications common in government and enterprise scenarios, such as "event localization—cause identification—strategy generation." Existing technologies are insufficient to meet the complex needs of government and enterprise operations, especially in scenarios requiring intelligent decision-making support, risk analysis, and dynamic task assessment based on multi-source structured data. Problems such as semantic fragmentation, lack of reasoning ability, and inability to generate strategy outputs are particularly prominent in these cases. Summary of the Invention

[0003] The purpose of this invention is to design an intelligent data processing method and system that supports multi-source heterogeneous data, which can simultaneously solve problems such as data structure alignment, semantic abstraction, task identification, causal reasoning and strategy triggering, so that structured heterogeneous data can serve business intelligent processing.

[0004] To achieve the above objectives, a first aspect of the present invention provides an intelligent data processing method supporting multi-source heterogeneous data, the method comprising:

[0005] For structured data from at least two business systems, the structured data is aggregated into data snapshot units based on a preset time window, task identifier, and organization set. Each data snapshot unit contains field-value pairs within the same task, the same time window, and a specified organization.

[0006] Map the field names in the data snapshot unit to standard field names, and generate a snapshot structure in a unified format based on the field values;

[0007] Based on the snapshot structure, semantic category labeling is performed on each standard field, and the semantic deviation of the field value relative to the historical average of the task is calculated. Combining the task threshold and the task sensitivity coefficient, a role strength value representing the strength of the field's role in the current task is generated, forming a semantic fragment.

[0008] Based on the semantic category distribution and role strength value of the semantic fragments, a predefined task category template is matched to identify the target task corresponding to the current snapshot, and a subset of key semantic fragments constituting the target task is selected to form a task structure.

[0009] Based on the task structure and the preset causal template, a causal connection graph between key semantic segments is constructed, the path with the maximum causal strength is extracted as the causal chain, and the suggested confidence level is generated by combining the policy sensitivity of the field with the length of the causal chain.

[0010] Based on the causal chain, the suggestion confidence level, and the preset task-field-action mapping template, a structured strategy suggestion is generated. The strategy suggestion is then converted into an executable instruction format for the target business system through the interface registry to complete the strategy triggering.

[0011] Furthermore, during the construction of the data snapshot unit, only the original fields belonging to the field set associated with the current task are retained, and the original field names are converted into standard field names through a field name mapping table.

[0012] Furthermore, the semantic bias is calculated based on the difference between the field value in the current time window and the reference mean of the field in the historical time window under the same task.

[0013] Furthermore, the generation of the role strength value depends on the field value, task threshold, task sensitivity coefficient, and semantic deviation, which are used to distinguish whether the field represents an abnormality, a normal state, or a supporting factor in the current task.

[0014] Furthermore, the task category template is a vector structure weighted by semantic category, and task recognition is achieved by calculating the weighted cosine similarity between the snapshot semantic distribution vector and each task category template.

[0015] Furthermore, the causal connection graph is constructed based on the causal template between semantic categories preset by the task, the coupling strength of fragment roles, and the frequency of historical co-occurrence anomalies to determine the causal connection strength between fragments.

[0016] Furthermore, the extraction of the causal chain is limited to a maximum chain length, and the path with the largest sum of causal connection strength is selected as the final causal chain.

[0017] Furthermore, the priority of the strategy recommendations is determined by the recommendation confidence level and the organizational level adjustment coefficient, with different organizational levels corresponding to different response weights.

[0018] Furthermore, the action types and target fields included in the strategy recommendations are limited to the fields corresponding to the key semantic segments in the task structure and their pre-associated actions, and reference to fields or actions outside the task structure is not allowed.

[0019] A second aspect of the present invention provides an intelligent data processing system supporting multi-source heterogeneous data, the system comprising:

[0020] The data snapshot construction module is used to aggregate structured data from at least two business systems into data snapshot units based on a preset time window, task identifier, and organization set. Each data snapshot unit contains field-value pairs within the same task, the same time window, and a specified organization. The module maps the field names in the data snapshot units to standard field names and generates a snapshot structure in a unified format based on the field values.

[0021] The semantic fragment generation module is used to perform semantic category labeling on each standard field based on the snapshot structure, calculate the semantic deviation of the field value relative to the historical average of the task, and combine the task threshold and the task sensitivity coefficient to generate a role strength value that represents the strength of the field's role in the current task, thus forming a semantic fragment.

[0022] The task identification module is used to match a predefined task category template based on the semantic category distribution and role strength value of the semantic fragments, identify the target task corresponding to the current snapshot, and filter the subset of key semantic fragments that constitute the target task to form a task structure.

[0023] The causal reasoning module is used to construct a causal connection graph between key semantic segments based on the task structure and the preset causal template, extract the path with the maximum causal strength as the causal chain, and generate a suggested confidence level by combining the strategy sensitivity of the field with the length of the causal chain.

[0024] The strategy generation and triggering module is used to generate structured strategy suggestions based on the causal chain, suggestion confidence, and preset task-field-action mapping template, and to convert the strategy suggestions into an executable instruction format for the target business system through the interface registry to complete the strategy triggering.

[0025] The beneficial technical effects of the present invention are at least as follows:

[0026] To address the aforementioned issues, this invention provides an intelligent data processing method and system that supports multi-source heterogeneous data. By constructing a unified data state expression mechanism, data from multiple sources with different modalities can be structured and effectively aligned within the same spatiotemporal context, providing a stable foundation for subsequent semantic abstraction and reasoning analysis. Building upon this, the system introduces a task-oriented semantic expression mechanism, transforming heterogeneous data into interpretable, composable, and reasonable semantic fragments. This supports intelligent tasks such as behavioral chain modeling, event causal analysis, and strategy generation in complex business contexts. The entire system design emphasizes upstream and downstream consistency in the data processing process, dynamically generating processing paths and forming feedback loops based on business objectives, thus realizing a data-driven decision-making process. Furthermore, the invention's architecture balances the dual requirements of horizontal multi-organizational collaboration and vertical data semantic abstraction, giving the system excellent scalability, maintainability, and evolvability in practical deployments. It fundamentally overcomes the limitations of traditional data processing solutions, which suffer from weak heterogeneous data processing capabilities and poor intelligent decision support, making it widely applicable to complex data environments in various government and enterprise sectors. Attached Figure Description

[0027] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0028] Figure 1 This is a flowchart of an intelligent data processing method that supports multi-source heterogeneous data according to the present invention.

[0029] Figure 2 This is a framework diagram of an intelligent data processing system that supports multi-source heterogeneous data according to the present invention. Detailed Implementation

[0030] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0031] In one or more embodiments, such as Figure 1 As shown, an intelligent data processing method supporting multi-source heterogeneous data is disclosed, the method comprising the following:

[0032] S1: For structured data from at least two business systems, based on a preset time window, task identifier, and organization set, the structured data is aggregated into data snapshot units, wherein each data snapshot unit contains field-value pairs within the same task, the same time window, and a specified organization; the field names in the data snapshot units are mapped to standard field names, and a snapshot structure with a unified format is generated based on the field values;

[0033] Specifically, this step aims to construct structured snapshot units from structured data from multiple business systems, while maintaining the semantic integrity of the original fields, according to a unified time window and task organization logic, to support semantic extraction and inference processing. Unlike traditional data cleaning or integration methods, this step emphasizes contextual consistency and task-driven characteristics. That is, each snapshot unit only aggregates original data records that are related to the same task, originate from a specific organizational set, and occur within the same time interval, thus serving as the structural basis for subsequent system processing.

[0034] The input received in this step is the original data set. This set consists of multi-source structured business records, denoted as:

[0035] ;

[0036] in, Indicates the first The data collection time comes from the timestamp field of the original system. The number indicating the organization to which the data belongs is given by the organization identifier embedded in the data upload interface; Indicates the name of the original data field, which is usually the column header of the uploaded table or the field label of the log entry; This indicates the value of the field, which is either numeric or fractional, and is directly derived from the value of the corresponding field in the original business data.

[0037] Based on the task scheduling strategy, the preset time window length is... Combined with the task start time Divide the data into time intervals. Simultaneously, by utilizing the task-organization mapping relationship, the set of organizations covered by the task is selected. Combined with field-task mapping functions Filter the data fields and construct the first... Data snapshot unit Its form is as follows:

[0038] ;

[0039] in, Indicates the first The start time of each time window is derived from the task time point set in the task schedule table; The fixed time period corresponding to the task is preset according to the task type, such as 1 day for education tasks and 15 minutes for transportation tasks. For the task The specified organization scope is usually derived from the organization-task assignment table in the deployment initialization configuration; It is a rule mapping function used to determine fields. Does it belong to the task? It consists of a manually configured field dictionary and a task keyword table; This indicates the identifier number of the current task.

[0040] To ensure field consistency, the system maintains a field name mapping table. , original field name Convert to standard field name In each snapshot unit, all fields must be standardized to a uniform format, resulting in the following structure:

[0041] ;

[0042] in, Indicates the first Each standard field name is determined by a mapping function. get; This indicates that the field is in a snapshot. The representative value in the time window is used. If there are multiple records in the field within the time window, the latest collected value is taken first. If it is a statistical indicator, the weighted average within the window is taken. This indicates the total number of standard fields in the snapshot unit, depending on the number of fields required by the task.

[0043] For example, in teaching assignments, different schools might upload fields such as "completion rate," "course achievement rate," and "learning progress percentage," through... Mapping to task Then through Standardized to Finally, the most recent value of each field is summarized to form the final result. This completes the structure alignment. The final output is a collection of snapshots. Each snapshot is a set of data units that have been aligned by time window, task organization, and field semantics, and has structural consistency and task context consistency.

[0044] S2: Based on the snapshot structure, semantic category labeling is performed on each standard field, and the semantic deviation of the field value relative to the historical average of the task is calculated. Combined with the task threshold and the task sensitivity coefficient, a role strength value representing the role intensity of the field in the current task is generated to form a semantic fragment.

[0045] Specifically, this step aims to transform the standardized snapshot set output in the first step... Fields in the field — numeric pairs Transformed into a set of semantic fragments with business semantics, causal roles, and task significance. Unlike traditional methods that only extract field values, this step leverages the "time-organization-task" dimension encapsulation completed in the previous stage, imbuing all fields with organizational context and task scenario background. This allows for the construction of higher-value semantic fragment structures based on specific business scenarios. The input is a collection of snapshots. , where each snapshot ,in These are standard field names, generated from the field mapping table; The field value is derived directly from the raw data within the time window.

[0046] The processing first involves semantically labeling the fields and constructing a semantic category dictionary. Each field is mapped using manually configured mapping rules. Mapping to semantic categories For example, in an education scenario, "course completion rate" is mapped to "execution status," and "attendance rate" is mapped to "behavioral indicator." In a traffic scenario, "average number of vehicles passing through" is mapped to "traffic flow," while "road segment speed" is mapped to "congestion status." Building upon this, to enhance the contextual expressiveness of the fragments, this step introduces a "task sensitivity factor," used to measure the degree of deviation of field values ​​from the historical behavior of that task.

[0047] Therefore, we construct the following formula for calculating semantic bias:

[0048] ;

[0049] in, For fields In snapshot Semantic deviations in; The field value in the current window; Is this field in the task? The reference average in the historical window is accumulated by the system over a long period of operation. For example, in teaching tasks, the historical average of "course completion rate" can be calculated by averaging the completion rates of the most recent ten days; in traffic tasks, the historical average of "road segment speed" can be generated by averaging the speeds of the same time period over the most recent five days. This deviation value reflects the contextual meaning of the field value and is an important basis for identifying anomalies.

[0050] Next, based on the semantic category of the field and deviation value Constructing semantic roles To enable role classification to express government and enterprise task scenarios, this step designs a role discrimination mechanism with task sensitivity regularization, which gives the role classification through the following formula:

[0051] ;

[0052] in This indicates the "role strength value" of the field in the current window, used to determine its semantic role; and The manually set task sensitivity coefficient is used to adjust the weight of deviation items and threshold offset items. In the education scenario, "attendance rate" is extremely sensitive to anomalies, so a larger coefficient can be set. In the energy scenario, the change in "equipment temperature" is more important, so different coefficients can be set. The policy or task threshold for this field, such as an attendance rate of 80% or a course completion rate of 60%, is derived from industry standard values ​​or task configuration tables.

[0053] pass The numerical value can determine the role a field plays in the current semantic context. For example, when A value significantly greater than zero indicates that the field may be related to the current anomaly or task risk; when When the value is close to zero, it indicates that the field is in a "normal state"; when... A value less than zero indicates that the field may be a supporting factor. This step does not directly provide the three types of labels, but rather... Retaining these values ​​as continuous character strength values ​​allows for further utilization of their expressive power in subsequent task target identification and causal reasoning.

[0054] Ultimately, all fields in each snapshot are converted into semantic fragments. and constitute a set of semantic fragments. All fragments are derived from snapshot data of the previous stage, with no new data sources. Furthermore, all fields undergo three steps: category mapping, bias calculation, and role strength generation, forming a semantic structure that is industry-specific, multi-organizationally understandable, and task-sensitive. The output is a collection of semantic fragments. The semantic fragments in this set will serve as all the input for the next step, structure recognition.

[0055] S3: Based on the semantic category distribution and role strength value of the semantic fragments, match the predefined task category template, identify the target task corresponding to the current snapshot, and filter the key semantic fragment subset that constitutes the target task to form a task structure;

[0056] Specifically, this step aims to build upon the set of semantic fragments generated in the second step. It identifies the potential business task objectives corresponding to each snapshot and constructs a task expression structure that is clear in structure, semantically interpretable, and task-constrained. This information is used by the subsequent inference engine. Unlike traditional task binding methods based on manual configuration or fixed template matching, this step is based on the semantic categories of fields annotated in the previous step. With character strength By constructing a multidimensional mapping in the semantic space and combining three types of structural features, namely "task sensitivity weight", "key field coverage" and "cross-organizational semantic consistency", a high-confidence identification of the task that should be triggered by the current snapshot is completed.

[0057] The input is a set of semantic fragments from the previous stage. Each segment is a quadruple. These represent the standard field name, field value, field semantic category, and role strength, respectively. All fragments originate from the snapshot units generated in the first step. The result, obtained through the second step, has a clear task context, time context, and field semantic background.

[0058] The key idea in this step is through structured analysis. Based on the semantic fragment distribution features, the task label that best suits the current fragment combination is found in the "task category mapping space". Therefore, a task category library is predefined. Each task Modeled as a semantic vector template Each of them This indicates that the task is for the first Importance weights of semantic categories This represents the total number of all semantic categories (e.g., "behavior", "state", "indicator"). The semantic distribution vector for each snapshot. Similarly, the vector is constructed by statistically weighting according to semantic categories, where the weight values ​​are the role strength of the fragment. The specific definition is as follows:

[0059] ;

[0060] in, For all semantic categories in the snapshot, the first one is... A collection of fragments of a class. The number of fragments of this type. Indicates the first The average semantic strength of a class in a snapshot constitutes the snapshot semantic feature vector. The semantic closeness between the current snapshot and a task template is measured by the similarity between vectors. Using weighted cosine similarity and incorporating a task confidence factor, the following task matching function is constructed:

[0061] ;

[0062] in, For the first Snapshots and Tasks semantic matching degree; For the task For the first The weight preferences for class fragments are given by the task definition template; It is a task The complexity regularization term represents the task's tolerance for semantic diversity, and is specifically set as follows:

[0063] ;

[0064] in, The average coverage of key fields in historical executions of the task. The regularization coefficient (empirically set to) ~ This is used to give higher confidence adjustments to tasks with few semantic categories but high importance, to solve the problem of low matching when certain key tasks are not adequately expressed in the semantic dimension, and to ensure that high-risk tasks can still be identified when data coverage is insufficient.

[0065] Based on matching degree Select the task with the highest value. This serves as the target task for the current snapshot. Furthermore, a subset of key semantic fragments constituting this task is selected from the fragment set. The filtering rules are as follows:

[0066] The semantic category is Category;

[0067] Corresponding character strength greater than the segment threshold set by the task .

[0068] This leads to the construction of task structure units. Its meaning is: current snapshot Recognized as a task Its supported fragments are The overall matching strength is .

[0069] S4: Based on the task structure and the preset causal template, construct a causal connection graph between key semantic segments, extract the path with the maximum causal strength as the causal chain, and combine the strategy sensitivity of the field with the length of the causal chain to generate a suggested confidence level.

[0070] Specifically, this step, based on the identified task structure, constructs logical paths with causal meaning within multi-source heterogeneous data, forming a structured causal chain, and generates interpretable and actionable suggestions based on this. Its fundamental goal is to make the task structure output in the previous stage more effective. It can further carry the three core semantic elements of "what happened", "why it happened", and "what should be done".

[0071] The task structure with input as the output of the third step ,in It is the name of the task being identified, such as "Teaching Plan Anomaly Warning" or "Traffic Congestion Anomaly Identification"; It is a set of key semantic fragments involved in this task; This represents the semantic matching strength between the task and the snapshot, reflecting the degree to which the task is activated. In, each semantic fragment It contains four elements This refers to the field name, field value, semantic category, and semantic role strength. For example, for the field "course completion rate", its field value is... The semantic category is "state", and the semantic role strength is This indicates that it plays a moderately strong explanatory role in the current task.

[0072] First, to determine whether an interpretable causal structure exists among these fragments, a semantic connection graph needs to be established. Considering that there are no explicit causal path priors in government and enterprise scenarios, this step introduces a pre-defined fragment causal template from the task template. For example, for the "Teaching Plan Anomaly Warning" task, there might be a situation where: "Student attendance rate is low". "Low course completion rate" A template path such as "abnormal teaching assessment scores". Template The structure is based on semantic categories rather than fixed field names, which allows it to accommodate situations where different organizations upload fields that are inconsistent but semantically similar. For example, "attendance ratio" and "absence rate" are different fields, but both belong to the "behavioral" semantic category and can be mapped to the same node.

[0073] Based on task templates This step scores the connections between all semantic segment pairs within the snapshot, calculating the causal connection strength between segments. The definition is as follows:

[0074] ;

[0075] in, and Each is a fragment and semantic categories, From template Provides a semantic causal structure common in tasks, such as setting the transition from "behavior class" to "state class". The rest are or less The weights; This represents the semantic role coupling degree, set to the smaller of the two values, to ensure that weak segments do not dominate the causal path; This refers to the frequency of anomalies occurring in both fields within historical data, obtained through statistical analysis of historical snapshot data. For example, in an education system, if a decrease in "course completion rate" is accompanied by a decrease in "attendance rate" exceeding [a certain threshold], [the frequency would be higher]. If the co-occurrence frequency is such that the co-occurrence factor is set to... above.

[0076] all Exceeding the threshold The fragments form a directed weighted graph. Here, edges represent possible causal directions, and weights indicate their reliability. A graph traversal algorithm (such as greedy maximum path search) is used to select the path with the strongest causal strength as the causal chain. The maximum chain length can be limited during construction (e.g., This is to enhance the interpretability of results and reduce policy ambiguity.

[0077] To generate recommendations, a recommendation confidence scoring function needs to be defined, combining the semantic fragment strength in the causal chain and the sensitivity settings in the task policy template.

[0078] ;

[0079] in, The confidence score for the final recommendation. The length of the causal chain. For the first in the causal chain The semantic strength of each segment For this field in the task The suggested sensitivity value indicates the controllability or representativeness of the task outcome due to changes in the field value. For example, "Course Completion Rate" could be set to... This indicates that an anomaly in this field directly triggered task activation; while "average student score" might be... This indicates that it is only an indirect characterization factor.

[0080] The final output is the reasoning structure. ,in For causal chain path, This refers to the suggested confidence level. Different confidence level ranges can be mapped to recommendation policy levels using rules, for example... "Strong recommendation for intervention" The first option is "Suggested Monitoring," and the rest are "Information Prompts." The output structure can be used for system interface display, report generation, or policy scheduling module invocation.

[0081] For example, in an educational scenario, a snapshot generated from data uploaded by a school contains the segments "Attendance rate = 72%", "Completion rate = 54%", and "Student score = 68". The system determines that this constitutes a causal chain: "Decrease in attendance rate → decrease in completion rate → decrease in score", and matches it with the task "Teaching Anomaly Warning". The suggested confidence level is [not specified]. Therefore, the system generates a strategy output that strongly recommends implementing attendance management and make-up class plans for this class.

[0082] S5: Based on the causal chain, the suggestion confidence level, and the preset task-field-action mapping template, generate structured strategy suggestions, and convert the strategy suggestions into an executable instruction format for the target business system through the interface registry to complete the strategy triggering;

[0083] Specifically, this step aims to structure the causal reasoning results output from the fourth step. This is further transformed into structured strategy recommendations for real business systems, and through adaptation mechanisms with interfaces of various information systems, automatic triggering or prompt-based interaction of instructions is achieved. The key task of this step is to "transform abstract causal reasoning results into executable business operation instructions," which not only requires a clear structure and standardized format, but also compatibility and deployability with various heterogeneous business platforms.

[0084] The input for this step is the causal reasoning result generated in step four. ,in For example, a causal chain path composed of multiple semantic fragments. Each segment All The forms represent the field name, field value, semantic category, and role strength, respectively. It also includes suggested confidence levels. This is used to measure the extent to which the causal chain explains the current task state, and also reflects the credibility of the policy recommendation output.

[0085] To ensure the standardization, clarity, and effectiveness of the recommendations, we first designed a unified recommendation instruction structure. Each item in this structure must be generated through explicit rules and data sources. (Suggested action set) Derived from the system's built-in task-field-action mapping template The template is jointly defined by domain experts and industry users, stored in YAML or JSON files, and periodically updated or approved manually by the system administrator. During the generation process, the system first traverses the causal chain. Key segments in the text, compared with the template Search and Fields Corresponding suggested actions If the value of the "Course Completion Rate" field is lower than The system will match the "generate make-up class plan" strategy action; when the "vehicle throughput" field drops significantly, the "dynamically optimize signal timing" action can be matched.

[0086] Recommended priority The calculation fully considers the suggestion confidence level. Organizational Executive Role Level Two factors, specifically defined as:

[0087] ;

[0088] in This is an organizational level adjustment factor, set by the industry regulatory body or platform administrator during the system deployment phase. For example, in an education system, the school-level unit can be set to... District-level units are Provincial-level units are This mechanism reflects the business priority strategy that requires higher-level units to respond more quickly and give greater importance to equal recommendations. Through this calculation mechanism, the ability to differentiate the processing priorities for the same problem across different organizations can be established.

[0089] Strategy summary information The generation of summaries is based on semantic information and role strength within the causal chain path. The system generates natural language descriptions through template filling. To avoid introducing a language model, this system combines fragment field names with policy corpus fragments to generate summary sentences, such as: "Due to the decline in attendance affecting course completion rate, it is recommended to arrange make-up classes"; "Vehicle traffic volume has decreased abnormally, it is recommended to temporarily optimize intersection timing." This process is implemented using a field-semantic template table, possessing strong controllability and stability, and does not rely on model inference.

[0090] In strategic actions Once determined, to ensure its effective implementation, it needs to be transformed into an interface data format acceptable to the target business system. This step involves designing a strategy instruction conversion function. Its function is to transform the suggested action content into the standard interface structure of the target business system. :

[0091] ;

[0092] in This indicates the interface number in the business system, such as "teaching plan adjustment interface", "traffic dispatch release interface" or "energy control command interface", which is uniformly managed by the interface registry maintained by the system. These are the parameters required by the interface, usually encapsulated in key-value pairs or JSON format, such as "class time adjustment amount = 4, target class = Grade 1 Class 3" or "signal timing extension = 12 seconds, affected intersection = East intersection". This indicates the time type at which the strategy action should be triggered, and is divided into three categories: "Execute Immediately", "Manual Confirmation", and "Scheduled Push". The time type is determined by the urgency of the strategy and the platform configuration.

[0093] Taking the government education system as an example, a snapshot In the process, the system identified the chain of "low attendance → decreased course completion rate → fluctuating academic performance" through causal reasoning, and combined it with the strategy template to suggest "issuing a tutoring plan" and "sending a notification to parents," with a confidence score of [insert confidence score here]. Recommended priority (A district-level education bureau, grade weight) The system generates a suggestion summary as "The current attendance rate of Class 3, Grade 1 is affecting the course progress. It is recommended to arrange targeted tutoring." The execution structure is constructed as "{Interface ID:042, Parameters: {Class: 'Class 3, Grade 1', Tutoring Hours: 4}, Trigger Time: Immediate}", which is sent directly to the teaching task management system through the interface middleware to achieve automatic closed-loop processing at the system level.

[0094] In one or more embodiments, such as Figure 2 As shown, an intelligent data processing system supporting multi-source heterogeneous data is disclosed, the system comprising:

[0095] The data snapshot construction module is used to aggregate structured data from at least two business systems into data snapshot units based on a preset time window, task identifier, and organization set. Each data snapshot unit contains field-value pairs within the same task, the same time window, and a specified organization. The module maps the field names in the data snapshot units to standard field names and generates a snapshot structure in a unified format based on the field values.

[0096] The semantic fragment generation module is used to perform semantic category labeling on each standard field based on the snapshot structure, calculate the semantic deviation of the field value relative to the historical average of the task, and combine the task threshold and the task sensitivity coefficient to generate a role strength value that represents the strength of the field's role in the current task, thus forming a semantic fragment.

[0097] The task identification module is used to match a predefined task category template based on the semantic category distribution and role strength value of the semantic fragments, identify the target task corresponding to the current snapshot, and filter the subset of key semantic fragments that constitute the target task to form a task structure.

[0098] The causal reasoning module is used to construct a causal connection graph between key semantic segments based on the task structure and the preset causal template, extract the path with the maximum causal strength as the causal chain, and generate a suggested confidence level by combining the strategy sensitivity of the field with the length of the causal chain.

[0099] The strategy generation and triggering module is used to generate structured strategy suggestions based on the causal chain, suggestion confidence, and preset task-field-action mapping template, and to convert the strategy suggestions into an executable instruction format for the target business system through the interface registry to complete the strategy triggering.

[0100] It is worth noting that the specific workflow of the intelligent data processing system supporting multi-source heterogeneous data provided in this embodiment of the invention is the same as that of the intelligent data processing method supporting multi-source heterogeneous data described in the above embodiment, and will not be repeated here.

[0101] This invention also provides an intelligent data processing device supporting multi-source heterogeneous data, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps described in the above embodiment of an intelligent data processing method supporting multi-source heterogeneous data, for example... Figure 1 The steps S1 to S5 described above; or, when the processor executes the computer program, it implements the functions of each module in the above system embodiments.

[0102] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in an intelligent data processing device supporting multi-source heterogeneous data.

[0103] The intelligent data processing device supporting multi-source heterogeneous data can be a desktop computer, laptop, handheld computer, or cloud server, among other computing devices. This device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that the intelligent data processing device may also include input / output devices, network access devices, and buses.

[0104] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASACs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the intelligent data processing device supporting multi-source heterogeneous data, connecting various parts of the device via various interfaces and lines.

[0105] The memory can be used to store the computer program and / or modules. The processor, by running or executing the computer program and / or modules stored in the memory and calling the data stored in the memory, realizes various functions of the intelligent data processing device supporting multi-source heterogeneous data. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the operation of the air conditioner controller, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0106] The module of the intelligent data processing device supporting multi-source heterogeneous data, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0107] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0108] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for intelligent data processing supporting multi-source heterogeneous data, characterized in that, The method includes: For structured data from at least two business systems, the structured data is aggregated into data snapshot units based on a preset time window, task identifier, and organization set. Each data snapshot unit contains field-value pairs within the same task, the same time window, and a specified organization. Map the field names in the data snapshot unit to standard field names, and generate a snapshot structure in a unified format based on the field values; Based on the snapshot structure, each standard field is semantically categorized and the semantic deviation of the field value relative to the historical average of the task is calculated. Combined with the task threshold and the task sensitivity coefficient, a role strength value representing the field's influence in the current task is generated, forming a semantic fragment. The semantic deviation is calculated based on the difference between the field value in the current time window and the reference average of the field in the historical time window under the same task. The generation of the role strength value depends on the field value, task threshold, task sensitivity coefficient, and semantic deviation, and is used to distinguish whether the field represents an abnormal, normal, or supporting factors in the current task. Based on the semantic category distribution and role strength value of the semantic fragments, a predefined task category template is matched to identify the target task corresponding to the current snapshot, and a subset of key semantic fragments constituting the target task is selected to form a task structure. Based on the task structure and the preset causal template, a causal connection graph between key semantic segments is constructed, the path with the maximum causal strength is extracted as the causal chain, and the suggested confidence level is generated by combining the policy sensitivity of the field with the length of the causal chain. Based on the causal chain, the suggestion confidence level, and the preset task-field-action mapping template, a structured strategy suggestion is generated. The strategy suggestion is then converted into an executable instruction format for the target business system through an interface registry to complete the strategy triggering. The action types and target fields included in the strategy suggestion are limited to the fields corresponding to the key semantic segments in the task structure and their pre-associated actions. Fields or actions outside the task structure are not allowed to be referenced.

2. The intelligent data processing method supporting multi-source heterogeneous data according to claim 1, characterized in that, During the construction of the data snapshot unit, only the original fields belonging to the field set associated with the current task are retained, and the original field names are converted into standard field names through a field name mapping table.

3. The intelligent data processing method supporting multi-source heterogeneous data according to claim 1, characterized in that, The task category template is a vector structure weighted by semantic category. Task recognition is achieved by calculating the weighted cosine similarity between the snapshot semantic distribution vector and each task category template.

4. The intelligent data processing method supporting multi-source heterogeneous data according to claim 1, characterized in that, The causal connection graph is constructed based on the causal template between semantic categories preset by the task, the coupling strength of fragment roles, and the frequency of historical co-occurrence anomalies to determine the causal connection strength between fragments.

5. The intelligent data processing method supporting multi-source heterogeneous data according to claim 1, characterized in that, The extraction of the causal chain is limited to a maximum chain length, and the path with the largest sum of causal connection strength is selected as the final causal chain.

6. The intelligent data processing method supporting multi-source heterogeneous data according to claim 1, characterized in that, The priority of the strategy recommendations is determined by the recommendation confidence level and the organizational level adjustment coefficient, with different organizational levels corresponding to different response weights.

7. An intelligent data processing system supporting multi-source heterogeneous data, characterized in that, The system includes: The data snapshot construction module is used to aggregate structured data from at least two business systems into data snapshot units based on a preset time window, task identifier, and organization set. Each data snapshot unit contains field-value pairs within the same task, the same time window, and a specified organization. The module maps the field names in the data snapshot units to standard field names and generates a snapshot structure in a unified format based on the field values. The semantic fragment generation module is used to perform semantic category labeling on each standard field based on the snapshot structure, calculate the semantic deviation of the field value relative to the historical average of the task, and combine the task threshold and task sensitivity coefficient to generate a role strength value that represents the strength of the field's role in the current task, thus forming a semantic fragment. The semantic deviation is calculated based on the difference between the field value in the current time window and the reference average of the field in the historical time window under the same task. The generation of the role strength value depends on the field value, task threshold, task sensitivity coefficient, and semantic deviation, and is used to distinguish whether the field represents an abnormal, normal state, or supporting factor in the current task. The task identification module is used to match a predefined task category template based on the semantic category distribution and role strength value of the semantic fragments, identify the target task corresponding to the current snapshot, and filter the subset of key semantic fragments that constitute the target task to form a task structure. The causal reasoning module is used to construct a causal connection graph between key semantic segments based on the task structure and the preset causal template, extract the path with the maximum causal strength as the causal chain, and generate a suggested confidence level by combining the strategy sensitivity of the field with the length of the causal chain. The strategy generation and triggering module is used to generate structured strategy suggestions based on the causal chain, suggestion confidence, and preset task-field-action mapping template, and to convert the strategy suggestions into an executable instruction format for the target business system through an interface registry to complete the strategy triggering. The action types and target fields included in the strategy suggestions are limited to the fields corresponding to the key semantic segments in the task structure and their pre-associated actions, and references to fields or actions outside the task structure are not allowed.

Images