Big data processing method and system for filtrate detection

Abstract

The present application relates to big data processing technical field, specifically for big data processing method and system for filtrate detection, including the following steps: collecting sample field, reagent field and equipment field, marking processing stage and function, analyzing calling relationship and process grouping, completing field and node matching, constructing big data processing process integration structure, in the present application, by constructing the function attribution, calling order and reference relationship among fields, the structural connection mode of the field in the processing logic is clear, based on the grouping and extension characteristics of the process path, the differentiated field organization form is formed, the continuity of the combined field and the cross node extraction are extracted, the process fragments with correlation are identified, combined with the structure matching between the field and the execution resource, the ordered mapping of the field to the processing node is completed, the logical chain and the execution order are reorganized, the field transmission path and the unified process structure are established, and the coherence and adaptability of the field flow in the detection are enhanced.

Description

Technical Field

[0001] This invention relates to the field of big data processing technology, and in particular to a big data processing method and system for filtrate detection. Background Technology

[0002] Big data processing technology involves core aspects such as the collection, storage, management, analysis, and mining of massive, multi-source, and heterogeneous data. It encompasses data modeling, distributed computing, data cleaning, feature extraction, data fusion, parallel processing, and visualization, with the overall goal of extracting valuable information from large-scale datasets to support decision-making. This technology is widely applied in various scenarios including industrial monitoring, healthcare, financial risk control, intelligent manufacturing, and environmental monitoring. It supports efficient processing of both structured and unstructured data, emphasizing the ability to model complex relationships and respond to dynamic changes. Traditional big data processing methods for filtrate detection refer to data calculation and processing methods used for real-time analysis of filtrate states in industrial processes or biological samples. These methods primarily rely on acquiring physical or chemical parameter data of the filtrate using sensors, followed by screening or anomaly identification using preset numerical thresholds. Common techniques include fixed-rule comparison, multi-index weighted judgment, and linear trend fitting analysis. Data computation and chart presentation are performed using local computing resources, and the perception of data fluctuation trends is mainly based on time-series analysis methods for discrete measurement data.

[0003] Existing technologies for filtrate detection mainly rely on static threshold comparison and single data source processing. They lack the ability to identify multi-level reference relationships between fields and the logical connection between processes. The processing structure is mainly linear analysis, which makes it difficult to support field reuse and path intersection in complex processes. This results in unclear field function positioning, a single data organization method, and the inability to form a clear mapping relationship between process structures. Field calls lack sequential correlation, process execution paths are loosely separated, and the ability to fully express the processing logic chain is lacking. This can easily lead to problems such as node execution confusion, field transmission delays, and data disconnect between tasks. Summary of the Invention

[0004] To achieve the above objectives, the present invention adopts the following technical solution: a big data processing method for filtrate detection, comprising the following steps: S1: Collect sample fields, reagent fields, and equipment fields from the filtrate sampling terminal, titration reaction information, and instrument status monitoring node; mark the processing stage and function of the sample fields, reagent fields, and equipment fields; record the calling order and reference relationship; and construct a field processing map. S2: Call the functions and order in the field processing graph, group the sample field, the reagent field and the device field according to the processing flow, extract the path connection direction, intersection node and extension relationship, and generate a field flowchart; S3: Call the paths and connections in the field flowchart, aggregate duplicate fields, determine the continuity of field combinations, extract field combinations with flow connectivity features, and generate processing flow segments. S4: Call the field combination in the processing flow segment, collect the deployment order parameters and execution channel identifiers of the field processing in the process path endpoint nodes, compare the deployment order and execution channel of the edge nodes, map the field combination to the nodes with execution capabilities, and generate a module mapping sequence; S5: Call the order and path in the module mapping sequence, reorganize the call chain of the field combination, record the transmission path and execution order of the fields between nodes, and construct the big data processing workflow integration structure.

[0005] As a further aspect of the present invention, the field processing graph includes field function attribution markers, field call order information, and field reference relationship mapping; the field flowchart includes sample processing path, reaction processing path, and error compensation path; the processing flow segment includes continuous field combinations, interconnected field combinations, and flow connection nodes; the module mapping sequence includes field deployment order, execution channel configuration, and node processing mapping; and the big data processing flow integration structure includes field logic call chain, field transmission path, and execution order structure.

[0006] As a further aspect of the present invention, the extraction of field combinations with process connectivity features refers to filtering field combinations that can be continuously transmitted and have logical pathways in the processing flow by analyzing the connection paths and connection logic between fields.

[0007] As a further aspect of the present invention, the node with execution capability refers to a functional unit and device node that has actual deployment and operation functions and can perform field combination processing tasks.

[0008] As a further aspect of the present invention, the specific steps of S1 are as follows: S101: Based on the sample field, reagent field and equipment field collected by the filtrate sampling terminal, call the concentration value in the sample field, the composition parameter in the reagent field and the reaction temperature and stirring rate in the equipment field, assign the field to a function according to the preset processing stage parameters, and generate a set of field function assignment tags. S102: Based on the field function attribution tag set, call the corresponding field's call record in the titration reaction information and instrument status monitoring node, construct the field call order based on the timestamp and scheduling index value, identify the reference relationship between fields, and generate a field call order structure table; S103: Call the field pairs in the field call order structure table, combine them with the function tags in the field function attribution tag set, construct the connection matrix between the fields, extract the graph structure representation of the fields, and establish the field processing graph.

[0009] As a further aspect of the present invention, the specific steps of S2 are as follows: S201: Call the field function attribution information and field call order data in the field processing map, classify all field nodes according to the preset sample processing flow, reaction processing flow and error compensation flow flow index parameters, and aggregate the field nodes according to the flow type label to establish a field flow path set. S202: Based on the field node sequence under the path in the field process path set, retrieve the connection direction between adjacent nodes, extract the field connection direction identification information, perform cross-reference detection on field nodes with path affiliation, filter the preset field nodes that meet the path cross structure conditions, and obtain the path connection structure parameter group. S203: Call the path structure information in the field flow path set and the path connection structure parameter group, analyze the physical parameter relationship between the connection direction between nodes in each path, the position of the intersection node, the distance value between the path extension nodes and the flow dependency strength, and construct the logical association diagram between nodes to generate the field flow diagram.

[0010] As a further aspect of the present invention, the specific steps of S3 are as follows: S301: Call the path grouping label and field connection relationship parameters in the field flowchart to identify the field nodes that are repeatedly referenced by the process path in the path, perform aggregation processing through field index value and path belonging information, calculate the frequency of repeated occurrence of field nodes in the path diagram, classify them according to field number, and generate a field repeated reference matrix. S302: Based on the field combination data in the field repeated reference matrix, extract the connection position index of the corresponding field in the differentiated process path, determine whether the connection order of the field combination in the adjacent path is continuous, filter the field combination that meets the preset connection order continuity condition, and obtain the field coherence combination set. S303: Call the field combination sequence in the field coherence combination set, retrieve the starting node and ending node index values ​​in the path, establish the extended structure of the field combination in the process path, extract the field set with continuity and process connectivity characteristics, and obtain the processing process segment.

[0011] As a further aspect of the present invention, the specific steps of S4 are as follows: S401: Call the field combination sequence and field connection order index in the processing flow segment, detect the start node and end node position of the field combination in the flow segment, and expand the field combination in sequence according to the relationship between the fields in the path to form a linear arrangement structure of the fields in the processing flow, and obtain the field combination execution order table. S402: Based on the field combination execution order table, collect the processing deployment order parameters and execution channel identifiers of the field combination in the start and end nodes of the process path, perform a structural comparison between the field combination order and the node deployment order, determine the consistency relationship between the field execution order and the node channel arrangement, filter the fields and corresponding items of the order matching, and generate a node field matching relationship set. S403: Call the field-node correspondence in the node field matching relationship set, map the fields sequentially to the node identifier with the corresponding execution channel according to the field combination order, establish the sequential association structure between fields and nodes, obtain the executable field-to-node mapping link, and generate the module mapping sequence.

[0012] As a further aspect of the present invention, the specific steps of S5 are as follows: S501: Call the node processing order parameters and field combination matching path information in the module mapping sequence, extract the node number and field execution order corresponding to each group of fields, aggregate and reorganize the logical call relationship between fields, establish a chain structure based on the field dependency direction and node scheduling order, and generate a set of field call chains; S502: Based on the field transmission path data in the field call chain set, record the jump position index of the field between nodes and the time order between fields, analyze the continuity of the transmission chain and the connection path between nodes in the field cross-node scheduling, and obtain the field transmission path sequence; S503: Integrate the set of field call chains and the set of structural parameters in the field pass path sequence, combine the execution order and path connection relationship of fields in differentiated nodes, construct a task execution architecture diagram with a link structure, and establish an integrated structure for big data processing flow.

[0013] Big data processing systems for filtrate detection include: The field annotation module is used to implement S1: collecting sample fields, reagent fields, and equipment fields from the filtrate sampling terminal, titration reaction information, and instrument status monitoring node; marking the processing stage and function of the sample fields, reagent fields, and equipment fields; recording the calling order and reference relationship; and constructing a field processing map. The flowchart module is used to implement S2: calling the functions and order in the field processing graph, grouping the sample field, the reagent field and the equipment field according to the processing flow, extracting the path connection direction, intersection nodes and extension relationships, and generating a field flowchart; The field combination module is used to implement S3: call the paths and connections in the field flowchart, aggregate duplicate fields, determine the continuity of field combinations, extract field combinations with flow connectivity features, and generate processing flow segments. The node mapping module is used to implement S4: calling the field combination in the processing flow segment, collecting the deployment order parameters and execution channel identifiers of the field processing in the endpoint nodes of the process path, comparing the deployment order and execution channel of the edge nodes, mapping the field combination to the nodes with execution capabilities, and generating a module mapping sequence; The process integration module is used to implement S5: calling the order and path in the module mapping sequence, reorganizing the call chain of the field combination, recording the transmission path and execution order of the fields between nodes, and constructing a big data processing process integration structure.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, by constructing the functional affiliation, calling order, and reference relationships between fields, the structural connection method of fields in the processing logic is clarified. Based on the grouping and extension characteristics of the process path, a differentiated field organization form is formed. By combining the continuity and cross node extraction of combined fields, related process segments are identified. By combining the structural matching between fields and execution resources, an ordered mapping from fields to processing nodes is completed, the logic chain and execution order are reorganized, and a field transmission path and unified process structure are established, thereby enhancing the coherence and adaptability of field flow in the detection process. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the steps of the present invention; Figure 2 This is a detailed schematic diagram of S1 of the present invention; Figure 3 This is a detailed schematic diagram of S2 of the present invention; Figure 4 This is a detailed schematic diagram of S3 of the present invention; Figure 5 This is a detailed schematic diagram of S4 of the present invention; Figure 6 This is a detailed schematic diagram of S5 of the present invention; Figure 7 This is a system module diagram of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0018] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0019] In practical filtrate testing, especially in the scenario of methionine-zinc chelation rate detection, the test data typically originates from specific chemical analysis steps, including sample dissolution, solid-liquid separation, titration reaction, and result calculation. This type of detection method is based on the slight solubility of methionine in water and its formation of a poorly soluble chelate with metallic zinc. By separately determining the components of the filtrate and filter residue, the zinc and methionine contents are obtained, and the chelation rate is calculated accordingly. This process generates various types of test fields, including titration volume data, reagent concentration parameters, reaction condition parameters, and endpoint determination information.

[0020] However, most existing filtrate testing data processing methods do not provide structured modeling of the generation logic between the above-mentioned testing steps and data fields. They lack a systematic description of the source, generation order, and interdependencies of fields during the testing process. This results in problems such as unclear data sources, weak correlations, and discontinuous process connections in subsequent data processing, making it difficult to meet the needs of multi-source data collaborative processing and dynamic mapping in complex testing processes.

[0021] Therefore, it is necessary to combine the procedural characteristics of specific filtrate detection methods, to uniformly model the sample fields, reagent fields, and equipment fields generated during the detection process, and to couple them with the data processing flow, so as to achieve synergistic matching between the detection method and the data processing structure, and improve the integrity and logical consistency of filtrate detection data processing.

[0022] Please see Figure 1 This invention provides a big data processing method for filtrate detection, comprising the following steps: S1: Based on the sample fields, reagent fields and equipment fields collected from the filtrate sampling terminal, titration reaction information and instrument status monitoring node, mark the function of each field according to the processing stage of each field in the detection task, and record the continuous calling order of the fields in the task processing logic chain and the reference association between fields to construct a field processing map; S2: Call the field function and field calling order in the field processing graph, group the fields according to the sample processing flow, reaction processing flow and error compensation flow, and extract the connection direction, intersection node and path extension relationship of the fields in each path to generate a field flowchart. S3: Call the path grouping and connection relationship in the field flowchart, perform aggregation analysis on the fields repeatedly referenced in the path, and judge the continuity according to the connection position of the field combination in the differentiated process. Extract the field combination with continuity and process connectivity characteristics, and extract the processing process segment. S4: Call the field combination and connection order in the processing flow segment, compare the deployment order and execution channel of field processing in edge nodes, map the field combination to the node processing with execution capability in sequence, and generate the module mapping sequence; S5: Match the node processing order and field combination path in the module mapping sequence, reorganize the logical call chain of all field combinations, record the transmission path and execution order of fields between nodes, and build an integrated structure for big data processing workflow.

[0023] The field processing graph includes field function attribution markers, field call order information, and field reference relationship mapping. The field flowchart includes sample processing path, reaction processing path, and error compensation path. The processing flow segments include continuous field combinations, interconnected field combinations, and process connection nodes. The module mapping sequence includes field deployment order, execution channel configuration, and node processing mapping. The big data processing flow integration structure includes field logical call chain, field transmission path, and execution order structure.

[0024] Please see Figure 2 The specific steps of S1 are as follows: S101: Based on the sample field, reagent field and equipment field collected by the filtrate sampling terminal, call the concentration value in the sample field, the composition parameter in the reagent field and the reaction temperature and stirring rate in the equipment field, assign the field to a function according to the preset processing stage parameters, and generate a set of field function assignment tags. Through the digital interface of the high-precision filtrate sampling terminal, the system captures the underlying data streams of sample, reagent, and equipment fields in real time. The acquisition scope not only covers basic data but also focuses on specific detection indicators, including mass data and volume parameters in sample information, and real-time concentration and precise dosage in reagent parameters. Particularly in titration process data, the system can accurately capture the consumed volume V of disodium ethylenediaminetetraacetate (EDTA-2Na) or sodium thiosulfate standard titration solution, as well as the blank volume V0, while simultaneously recording instrument operating status data. This process first preprocesses the acquired raw signals by removing preset background noise interference bands to ensure data purity. This preprocessing step is actually led by the system's big data processing layer, which uses built-in data cleaning and outlier removal algorithms to initially purify the raw signals, laying the foundation for subsequent high-precision calculations. For the sample field, the molar concentration of chemical substances and the physical form of the solute are extracted. For the reagent field, the molecular weight parameters and purity percentage of its chemical components are analyzed. For the equipment field, the real-time reaction temperature value fed back by the thermocouple inside the reactor and the real-time rotational speed frequency of the stirring motor are read. Subsequently, based on a preset processing stage parameter library, a field classification logic algorithm is invoked to map the above fields to the corresponding functional ranges. This algorithm first calculates the feature matching degree between each field and the preset functional category, that is, it performs a dot product operation between the metadata attributes of the field and the attribute vector of the standard functional template to obtain a matching score. If the matching score exceeds the preset classification benchmark value, the field is classified into the corresponding functional category. For example, when the reaction temperature value is 85 degrees Celsius and the preset thermosensitive reaction range is 80 degrees Celsius to 90 degrees Celsius, the temperature field is marked as "thermal catalytic condition classification". Through the traversal and calculation of all fields, a field function classification tag set containing the functional positioning information of all fields is finally generated.

[0025] S102: Based on the field function belonging tag set, call the call record of the corresponding field in the titration reaction information and instrument status monitoring node, construct the field call order based on the timestamp and scheduling index value, identify the reference relationship between fields, and generate a field call order structure table; Based on the generated field function attribution tag set, a deep search program is initiated on the historical operation logs to locate the log clusters of the titration reaction information storage area and the instrument status monitoring node. At this time, the system utilizes the distributed storage architecture of the data storage layer for data backtracking. This storage layer not only has high-concurrency read and write capabilities but is also specifically optimized for biochemical detection scenarios, enabling efficient integration and storage of multidimensional data on zinc and methionine detection in filtrate and filter residue, ensuring data integrity and traceability. This process extracts all call records for the corresponding field in past reaction cycles by comparing unique identifiers in the tag set. For each call record, the nanosecond-level timestamp at the time of its generation and the global scheduling index value assigned by the central controller are precisely read. Using bubble sort logic, the call records are linearly arranged according to the order of timestamps. If there are cases where the timestamps are completely identical, a secondary sort is performed based on the numerical value of the scheduling index, thereby constructing a strict time series. On this basis, a reference relationship identification operation is further performed, that is, analyzing whether the output value of the previous field is used as the input parameter of the next field in the calculation. Specifically, the data dependency coefficient between adjacent fields is calculated, which is the ratio of the rate of change of the value of the latter field to the rate of change of the value of the former field. If this ratio is within a preset correlation range, it is determined that there is a reference relationship between the two. For example, when an increase in the amount of reagent added causes the reaction temperature to rise positively within 3 seconds, and the correlation coefficient reaches 0.95, it is determined that there is a direct reference between the two, and a field call order structure table describing the temporal and logical dependencies between the fields is generated accordingly.

[0026] S103: Call the field pairs in the field call order structure table, combine them with the function tags in the field function attribution tag set, construct the connection matrix between fields, extract the graph structure representation of the fields, and establish the field processing graph; The system calls each pair of adjacent fields in the field call order structure table, combined with the function labels assigned in the field function attribution label set, to initiate the connection matrix construction program. This program initializes a zero matrix with a dimension equal to the total number of fields multiplied by the total number of fields, defining row indices as output fields and column indices as input fields. It iterates through all field pairs; if field A points to field B, it finds the intersection of the row corresponding to field A and the column corresponding to field B, updating the value at that position from 0 to the specific connection weight value. This connection weight value is obtained by calculating the coupling strength between function labels, i.e., a weighted sum of the function priority value of field A and the function sensitivity value of field B. Subsequently, using graph theory algorithms, the constructed connection matrix is ​​transformed into a visual graph structure representation. In this process, each field is mapped to a vertex in the graph, and non-zero elements in the matrix are mapped to directed edges between vertices, with the thickness of the edges directly corresponding to the magnitude of the connection weight value. Table 1 below shows some of the connection weight calculation data between fields. Through this refined graph construction, the system can provide a precise topological foundation for subsequent intelligent calculations, effectively avoiding logical gaps that may occur in traditional manual recording.

[0027] Table 1. Field Connection Weight Calculation Parameters Field combination identifier Output field function priority Input field functionality sensitivity Coupling strength weighting coefficient Calculated connection weight values SR-01 0.85 0.92 1.5 1.3275 ST-02 0.76 0.88 1.2 0.9840 RE-03 0.91 0.75 1.4 1.1620 EM-04 0.65 0.82 1.0 0.7350 As shown in Table 1, taking field combination SR-01 as an example, the functional priority of the output field is 0.85, the functional sensitivity of the input field is 0.92, and the coupling strength weight coefficient is set to 1.5. A weighted calculation is performed: first, the priority and sensitivity are multiplied to obtain the basic association value of 0.782, then multiplied by the weight coefficient 1.5, resulting in a connection weight value of 1.173 (Note: This is a simplified logic; if a more complex weighted summation logic is used, such as priority multiplied by 0.5 plus sensitivity multiplied by 0.5 and then multiplied by the coefficient, the result would be (0.85 + 0.92) × 1.5 = 1.3275). By establishing the topological relationships between the nodes, a complete field processing graph is finally established.

[0028] Please see Figure 3 The specific steps of S2 are as follows: S201: Call the field function attribution information and field call order data in the field processing map. Based on the preset sample processing flow, reaction processing flow and error compensation flow flow index parameters, classify all field nodes by path and aggregate field nodes by flow type label to establish a field flow path set. The system retrieves all field function attribution information and field call order data encapsulated in the field processing graph and inputs this data into the path classification engine. This engine has built-in standard templates for preset sample processing, reaction processing, and error compensation processes, each containing a specific set of process index parameters. These process templates are custom-developed based on the actual business logic of methionine zinc chelation rate detection, aiming to adapt to the specific needs of this detection project for low-cost and rapid testing. A path matching algorithm is used to scan the node sequences in the graph one by one, calculating their similarity scores with each standard process template. This calculation process is achieved by comparing the overlap between the functional label sequences of the nodes and the label sequences required by the template. For each successful match of a key node, the corresponding matching score is accumulated. When the total matching score of a path segment for a specific process template exceeds a preset classification threshold, the path segment is marked as the corresponding process type. Subsequently, an aggregation operation is performed, logically grouping all field nodes marked as the same type by their physical addresses. For example, all fields related to the "acid-base neutralization" reaction, such as temperature, pH value, and stirring speed, are aggregated into the "reaction processing process" set. By classifying and aggregating all nodes in the graph, a set of field process paths containing multiple independent process definitions is established.

[0029] S202: Based on the field node sequence under the path in the field flow path set, retrieve the connection direction between adjacent nodes, extract the field connection direction identification information, perform cross-reference detection on field nodes with path affiliation, filter the preset field nodes that meet the path cross structure conditions, and obtain the path connection structure parameter group. Based on the established set of field flow paths, the system deeply analyzes the field node sequence under each independent path, using a directed graph traversal algorithm to retrieve the connection direction between adjacent nodes and extract clear field connection direction identifiers, such as "unidirectional progression," "bidirectional feedback," or "loop iteration." On this basis, a cross-reference detection program is initiated, designed to identify whether there are shared or conflicting nodes between different flow paths. All field nodes with path affiliation are traversed, recording the list of path IDs referenced by each node. If a node's path ID list contains two or more different path IDs, and these paths correspond to different flow types, then the node is identified as a potential cross-reference node. These nodes are further filtered to see if they meet the preset path cross-reference structure conditions, i.e., whether their in-degree and out-degree are both greater than or equal to 2. For these complex cross-reference nodes, the system's big data processing layer automatically intervenes, using advanced error correction algorithms to perform secondary verification and smoothing of the data, thereby reducing human calculation errors caused by path conflicts at the source. For example, when a "temperature sensor" field is used simultaneously by the "reaction processing flow" to control heating and by the "error compensation flow" to correct pH readings, that field satisfies the cross-condition. All nodes that meet the condition and their connection parameters are packaged to obtain the path connection structure parameter group.

[0030] S203: Call the path structure information in the field flow path set and path connection structure parameter group, analyze the physical parameter relationship between the connection direction between nodes in each path, the position of the intersection node, the distance value between the path extension nodes and the flow dependency strength, and construct the logical association diagram between nodes to generate the field flow diagram. The core path structure information from the field flow path set and path connection structure parameter group is invoked to launch the logical association analysis engine. This engine first parses the connection direction between nodes in each independent path to confirm the data flow trend; secondly, it accurately locates the geometric position of the intersection nodes in the global topology and analyzes their throughput capacity as data hubs; finally, it analyzes the numerical relationships of path extension nodes, that is, calculates the numerical influence weight of the output value of the end node of the path on the input value of the starting node of the next path. The direction information, position information, and numerical weight information obtained from the above analysis are integrated to construct a logical association graph between nodes. This graph not only shows the physical connections but also describes the logical constraints between nodes through metadata. This step makes the deep-seated logic between filtrate detection data explicit, significantly improving the data utilization rate of subsequent stages. For example, if the analysis shows that for every unit increase in the endpoint value of path A, the starting value of path B needs to decrease by 0.5 units to maintain balance, a logical connection with a negative correlation attribute is generated in the association graph. Finally, this logical association graph is instantiated to generate a visual field flow diagram.

[0031] Please see Figure 4 The specific steps of S3 are as follows: S301: Call the path grouping label and field connection relationship parameters in the field flowchart to identify the field nodes that are repeatedly referenced by the process path in the path. Aggregate the field index value and path belonging information to calculate the frequency of repeated occurrence of the field node in the path diagram, and classify them according to the field number to generate a field repeated reference matrix. Load the field flowchart and read the embedded path grouping labels and field connection parameters. For each independent process path, identify the field nodes repeatedly referenced by the process path, i.e., those nodes that appear more than once during the traversal of the same path. Using the field index value as the primary key, and combining it with path attribution information, perform aggregation processing to calculate the recurrence frequency of each field node in the current path graph. The calculation logic is as follows: for a specific field, count the total number of times it appears in the path sequence, divide it by the total number of nodes contained in the path, and obtain the recurrence frequency coefficient. Subsequently, these frequency data are categorized and organized according to the field number. For example, if a path contains 20 nodes, and the "pressure monitoring" field appears 4 times, then the recurrence frequency coefficient of this field is 0.2. Construct a two-dimensional matrix, where rows represent different process paths, columns represent different field numbers, and cells store the calculated recurrence frequency coefficient. This matrix is ​​the field recurrence reference matrix, which intuitively reflects the degree of reuse and redundancy of each field in different processes.

[0032] S302: Based on the field combination data in the field repeated reference matrix, extract the connection position index of the corresponding field in the differentiated process path, determine whether the connection order of the field combination in the adjacent path is continuous, filter the field combination that meets the preset connection order continuity condition, and obtain the field coherence combination set. The connection position index of the corresponding field in the differentiated process path is extracted, that is, the step in which each high-frequency field appears in the path is recorded. Then, continuity judgment logic is executed to analyze whether the connection order of these field combinations in adjacent paths is continuous. Specifically, the end position index of the same field in path A is compared with the start position index in path B, and the index difference between the two is calculated. If the absolute value of the difference is equal to 1, it is determined to be a direct connection; if the difference is within a preset tolerance range (e.g., less than 3), it is determined to be logically continuous. If the difference exceeds the preset range, but the time interval between the corresponding field appearing in different paths is extremely short and meets the preset minimum time difference threshold (e.g., less than 100ms), then combined with the index difference and time similarity, it can still be determined as a continuous field, forming a fault-tolerant judgment mechanism. This high-precision continuity judgment mechanism ensures the smooth flow of data and is a key link in realizing intelligent processing of filtrate testing data. For example, if field A appears at the end of path 1 (index 10) and again at the beginning of path 2 (index 1), even though the index difference is 9, if the processing time interval between the two is only 80ms, which is lower than the preset time threshold of 100ms, it is still considered to meet the field continuity condition. Finally, all the filtered field combinations and their order information are integrated to obtain the field continuity combination set.

[0033] S303: Call the field combination sequence in the field coherence combination set, retrieve the index values ​​of the start node and end node in the path, establish the extended structure of the field combination in the process path, extract the field set with continuity and process connectivity characteristics, and obtain the processing process segment. The process retrieves the start and end node indices for each path, using this endpoint information to establish an extended structure of field combinations within the process path. This process aims to piece together fragmented, coherent combinations into complete operational segments. A depth-first search algorithm is executed, starting from the end node of a given coherent combination and searching for the start node of the next coherent combination; if they match, they are merged. Simultaneously, the set of fields exhibiting continuity and process connectivity characteristics from these combinations is extracted, filtering out noisy nodes. The completeness of the segments is verified by calculating the process coverage ratio, i.e., the ratio of the extracted segment length to the original path length. If this ratio reaches a preset validity standard (e.g., 0.8), the segment is confirmed as an independent and complete processing unit. Through this process, a series of processing flow segments with clear business meanings are successfully obtained, preparing for subsequent linear arrangement.

[0034] Please see Figure 5 The specific steps of S4 are as follows: S401: Call the field combination sequence and field connection order index in the processing flow segment, detect the start and end node positions of the field combination in the flow segment, and expand the field combination in sequence according to the relationship between the fields in the path to form a linear arrangement structure of the fields in the processing flow, and obtain the field combination execution order table. The linearization process is initiated by calling the field combination sequence and field connection order index within the processing flow segment. First, the start and end nodes of each field combination within the flow segment are detected to determine the operation boundaries. Then, based on the inherent dependencies between fields in the path, the field combinations are sequentially expanded. This expansion process follows topological sorting principles, ensuring that all preceding dependent fields precede subsequent fields. For parallel branch structures, they are converted into linear sequences according to preset priority rules. Finally, a linear arrangement structure of fields in the processing flow is formed, generating a table of field combination execution order containing strictly executed steps. For example, if a flow segment contains two parallel branches, "heating -> stirring" and "acid addition -> pH detection," and "heating" has a higher priority than "acid addition," then the linearized order is "heating -> stirring -> acid addition -> pH detection." Table 2 below shows an example of the linearization result of the field combination execution order.

[0035] Table 2 Linearization of Field Combination Execution Order Execution step number Field combination name Starting node index Termination Node Index Priority weight Linearized execution time (ms) 1 Preheat Start 1001 1005 0.95 0 2 Solvent injection 2001 2004 0.88 500 3 Stir and mix 3001 3010 0.85 1200 4 Status monitoring 4001 4003 0.99 1800 As shown in Table 2, the order of each execution step, the corresponding field combination, the start and end indexes, the priority, and the precise execution time point are clearly defined, ensuring the executability of the process.

[0036] S402: Based on the field combination execution order table, collect the processing deployment order parameters and execution channel identifiers of the field combination in the start and end nodes of the process path, perform a structural comparison between the field combination order and the node deployment order, determine the consistency relationship between the field execution order and the node channel arrangement, filter the fields and corresponding items of the order matching, and generate a node field matching relationship set. The data collection process combines field combinations with processing deployment order parameters and execution channel identifiers at the start and end nodes of the workflow path. These parameters describe the actual execution capacity and resource consumption at the hardware level. The execution structure comparison logic matches the field combination order at the software level with the node deployment order at the hardware level item by item. Specifically, it calculates the matching degree between the processing capacity required for each field and the corresponding node's channel attribute. It determines whether the field execution order and node channel arrangement are consistent, i.e., checks for the logical paradox of "execute first, then deploy." It filters out field and node counterparts that match in order and are resource-compatible, eliminating conflicting entries. For example, if a field requires "high-speed centrifugation" to be executed at the 5th second, but the corresponding centrifuge node is in "self-check" mode at the 5th second (i.e., the channel is occupied), the match is invalid; conversely, if the channel is idle and the function matches, the match is successful. Finally, a set of node field matching relationships containing all valid matching pairs is generated.

[0037] S403: Call the field field matching relationship set to find the correspondence between fields and nodes, map the fields sequentially to the node identifiers with corresponding execution channels according to the order of field combination, establish the sequential association structure between fields and nodes, obtain the executable field-to-node mapping link, and generate the module mapping sequence; Following the logical flow determined by the field combination sequence table, fields are sequentially mapped to node identifiers with corresponding execution channels. This process is equivalent to assigning a specific hardware executor to each abstract software instruction. A sequential association structure between fields and nodes is established to ensure that each field has a unique node responsible for execution, and that the execution order conforms to the process requirements. Simulated testing verifies the feasibility of the mapping link, checking for deadlocks or resource contention. This automated mapping mechanism significantly improves detection efficiency, enabling the system to quickly respond to large-scale detection tasks. If the test passes, this mapping relationship is solidified, resulting in a complete and executable field-to-node mapping link. Finally, a module mapping sequence is generated, which specifies in detail which module should process which field's data at each time step of the processing flow, achieving precise hardware and software coordination.

[0038] Please see Figure 6 The specific steps of S5 are as follows: S501: Call the node processing order parameters and field combination matching path information in the module mapping sequence, extract the node number and field execution order corresponding to each group of fields, aggregate and reorganize the logical call relationship between fields, establish a chain structure based on the field dependency direction and node scheduling order, and generate a set of field call chains; The process calls the node processing order parameters and field combinations in the module mapping sequence to match path information, extracting the node number and execution order corresponding to each field group. Based on this, the logical call relationships between fields are aggregated and reorganized. This process utilizes a linked list construction algorithm to establish a doubly linked structure based on field dependency direction (i.e., data flow direction) and node scheduling order (i.e., control flow direction). For each node, its predecessor and successor nodes are recorded, forming a tight logical closed loop. Special attention is paid to cross-node field calls to ensure data integrity during transmission between different hardware units. Finally, a set of field call chains containing all logical link information is generated. This set not only describes the execution chain of a single process but also includes alternative chains for parallel processing and exception handling.

[0039] S502: Based on the field transmission path data in the field call chain set, record the jump position index of the field between nodes and the time order between fields, analyze the continuity of the transmission chain and the connection path between nodes in the field cross-node scheduling, and obtain the field transmission path sequence; Record the jump position index of the field between nodes and the time order between fields. Initiate a continuity analysis program to calculate the continuity coefficient of the transmission chain in the cross-node scheduling of fields. This coefficient is determined by the physical distance between nodes, network latency, and data conversion time. The calculation logic is as follows: First, obtain the communication latency and data packet transmission time between nodes, sum them to obtain the single jump time; then compare this time with a preset maximum allowable latency threshold. If the time is less than the threshold, the connection is considered continuous and valid. Simultaneously, analyze the stability of the connection path between nodes and identify potential bottlenecks. Through full-link scanning and calculation, a detailed field transmission path sequence is obtained, which annotates the specific latency expectation and path reliability score for each data transmission step.

[0040] S503: Integrate the set of structural parameters in the field call chain and the field passing path sequence, combine the execution order and path connection relationship of fields in differentiated nodes, construct a task execution architecture diagram with a link structure, and establish an integrated structure for big data processing flow; The system invokes the set of field call chains and the set of structural parameters in the field transmission path sequence to perform the final integration operation. It globally coordinates the execution order and path connections of fields at differentiated nodes, constructing a task execution architecture diagram with a link structure. This diagram uses a time axis as the main line, visually arranging the execution tasks, data transmission tasks, and waiting tasks of each node. The theoretical throughput and execution efficiency of the entire architecture are calculated to verify whether it meets the high concurrency requirements of big data processing. This architecture diagram clarifies the convergence and distribution points of the data flow, establishing an integrated structure for the big data processing workflow. The establishment of this structure signifies a complete connection from the bottom-level field acquisition to the top-level process control, realizing intelligent encapsulation and automated execution of complex processing flows. Finally, the system presents the processed and accurate data through the result output layer. This end-to-end integrated architecture fills the gap in unified detection methods in this field, not only realizing intelligent processing of filtrate detection data and effectively reducing human calculation errors, but also significantly improving detection efficiency and data utilization, perfectly meeting the urgent need for low-cost and rapid detection of methionine zinc chelation rate. For example, this integrated structure can integrate 1,000 previously scattered independent data collection points and 50 processing nodes into a unified and scheduled organic whole, significantly improving the collaborative efficiency of data processing.

[0041] Please see Figure 7 A big data processing system for filtrate detection includes: The field annotation module is used to implement S1: collecting sample fields, reagent fields, and equipment fields from the filtrate sampling terminal, titration reaction information, and instrument status monitoring node; marking the processing stage and function of sample fields, reagent fields, and equipment fields; recording the calling order and reference relationship; and constructing a field processing map. The flowchart module is used to implement S2: calling the functions and order in the field processing graph, grouping the sample field, reagent field and equipment field according to the processing flow, extracting the path connection direction, intersection node and extension relationship, and generating the field flowchart; The field combination module is used to implement S3: call the paths and connections in the field flowchart, aggregate duplicate fields, determine the continuity of field combinations, extract field combinations with process connectivity characteristics, and generate processing flow segments. The node mapping module is used to implement S4: calling the combination and order of fields in the process flow segment, collecting the deployment order parameters and execution channel identifiers of the fields in the endpoint nodes of the process path, comparing the deployment order and execution channel of the edge nodes, mapping the field combination to the node with execution capability, and generating the module mapping sequence; The process integration module is used to implement S5: calling the order and path in the module mapping sequence, reorganizing the call chain of field combinations, recording the transmission path and execution order of fields between nodes, and building a big data processing process integration structure.

[0042] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of protection of the technical solution.

Claims

1. A big data processing method for filtrate detection, characterized in that, Includes the following steps: S1: Collect sample fields, reagent fields, and equipment fields from the filtrate sampling terminal, titration reaction information, and instrument status monitoring node; mark the processing stage and function of the sample fields, reagent fields, and equipment fields; record the calling order and reference relationship; and construct a field processing map. S2: Call the functions and order in the field processing graph, group the sample field, the reagent field and the device field according to the processing flow, extract the path connection direction, intersection node and extension relationship, and generate a field flowchart; S3: Call the paths and connections in the field flowchart, aggregate duplicate fields, determine the continuity of field combinations, extract field combinations with flow connectivity features, and generate processing flow segments. S4: Call the field combination in the processing flow segment, collect the deployment order parameters and execution channel identifiers of the field processing in the process path endpoint nodes, compare the deployment order and execution channel of the edge nodes, map the field combination to the nodes with execution capabilities, and generate a module mapping sequence; S5: Call the order and path in the module mapping sequence, reorganize the call chain of the field combination, record the transmission path and execution order of the fields between nodes, and construct the big data processing workflow integration structure.

2. The big data processing method for filtrate detection according to claim 1, characterized in that, The field processing graph includes field function attribution markers, field call order information, and field reference relationship mapping. The field flowchart includes sample processing path, reaction processing path, and error compensation path. The processing flow segment includes continuous field combinations, interconnected field combinations, and flow connection nodes. The module mapping sequence includes field deployment order, execution channel configuration, and node processing mapping. The big data processing flow integration structure includes field logic call chain, field transmission path, and execution order structure.

3. The big data processing method for filtrate detection according to claim 1, characterized in that, The extraction of field combinations with process connectivity features refers to filtering field combinations that can be continuously transmitted and have logical pathways in the processing flow by analyzing the connection paths and connection logic between fields.

4. The big data processing method for filtrate detection according to claim 1, characterized in that, The nodes with execution capabilities refer to functional units and device nodes that have actual deployment and operation functions and can perform tasks by combining executable fields.

5. The big data processing method for filtrate detection according to claim 1, characterized in that, The specific steps of S1 are as follows: S101: Based on the sample field, reagent field and equipment field collected by the filtrate sampling terminal, call the concentration value in the sample field, the composition parameter in the reagent field and the reaction temperature and stirring rate in the equipment field, assign the field to a function according to the preset processing stage parameters, and generate a set of field function assignment tags. S102: Based on the field function attribution tag set, call the corresponding field's call record in the titration reaction information and instrument status monitoring node, construct the field call order based on the timestamp and scheduling index value, identify the reference relationship between fields, and generate a field call order structure table; S103: Call the field pairs in the field call order structure table, combine them with the function tags in the field function attribution tag set, construct the connection matrix between the fields, extract the graph structure representation of the fields, and establish the field processing graph.

6. The big data processing method for filtrate detection according to claim 1, characterized in that, The specific steps of S2 are as follows: S201: Call the field function attribution information and field call order data in the field processing map, classify all field nodes according to the preset sample processing flow, reaction processing flow and error compensation flow flow index parameters, and aggregate the field nodes according to the flow type label to establish a field flow path set. S202: Based on the field node sequence under the path in the field process path set, retrieve the connection direction between adjacent nodes, extract the field connection direction identification information, perform cross-reference detection on field nodes with path affiliation, filter the preset field nodes that meet the path cross structure conditions, and obtain the path connection structure parameter group. S203: Call the path structure information in the field flow path set and the path connection structure parameter group, analyze the physical parameter relationship between the connection direction between nodes in each path, the position of the intersection node, the distance value between the path extension nodes and the flow dependency strength, and construct the logical association diagram between nodes to generate the field flow diagram.

7. The big data processing method for filtrate detection according to claim 1, characterized in that, The specific steps for S3 are as follows: S301: Call the path grouping label and field connection relationship parameters in the field flowchart to identify the field nodes that are repeatedly referenced by the process path in the path, perform aggregation processing through field index value and path belonging information, calculate the frequency of repeated occurrence of field nodes in the path diagram, classify them according to field number, and generate a field repeated reference matrix. S302: Based on the field combination data in the field repeated reference matrix, extract the connection position index of the corresponding field in the differentiated process path, determine whether the connection order of the field combination in the adjacent path is continuous, filter the field combination that meets the preset connection order continuity condition, and obtain the field coherence combination set. S303: Call the field combination sequence in the field coherence combination set, retrieve the starting node and ending node index values ​​in the path, establish the extended structure of the field combination in the process path, extract the field set with continuity and process connectivity characteristics, and obtain the processing process segment.

8. The big data processing method for filtrate detection according to claim 1, characterized in that, The specific steps of S4 are as follows: S401: Call the field combination sequence and field connection order index in the processing flow segment, detect the start node and end node position of the field combination in the flow segment, and expand the field combination in sequence according to the relationship between the fields in the path to form a linear arrangement structure of the fields in the processing flow, and obtain the field combination execution order table. S402: Based on the field combination execution order table, collect the processing deployment order parameters and execution channel identifiers of the field combination in the start and end nodes of the process path, perform a structural comparison between the field combination order and the node deployment order, determine the consistency relationship between the field execution order and the node channel arrangement, filter the fields and corresponding items of the order matching, and generate a node field matching relationship set. S403: Call the field-node correspondence in the node field matching relationship set, map the fields sequentially to the node identifier with the corresponding execution channel according to the field combination order, establish the sequential association structure between fields and nodes, obtain the executable field-to-node mapping link, and generate the module mapping sequence.

9. The big data processing method for filtrate detection according to claim 1, characterized in that, The specific steps of S5 are as follows: S501: Call the node processing order parameters and field combination matching path information in the module mapping sequence, extract the node number and field execution order corresponding to each group of fields, aggregate and reorganize the logical call relationship between fields, establish a chain structure based on the field dependency direction and node scheduling order, and generate a set of field call chains; S502: Based on the field transmission path data in the field call chain set, record the jump position index of the field between nodes and the time order between fields, analyze the continuity of the transmission chain and the connection path between nodes in the field cross-node scheduling, and obtain the field transmission path sequence; S503: Integrate the set of field call chains and the set of structural parameters in the field pass path sequence, combine the execution order and path connection relationship of fields in differentiated nodes, construct a task execution architecture diagram with a link structure, and establish an integrated structure for big data processing flow.

10. A big data processing system for filtrate detection, characterized in that, The system is used to implement the big data processing method for filtrate detection as described in any one of claims 1-9, and the system includes: The field annotation module is used to implement S1: collecting sample fields, reagent fields, and equipment fields from the filtrate sampling terminal, titration reaction information, and instrument status monitoring node; marking the processing stage and function of the sample fields, reagent fields, and equipment fields; recording the calling order and reference relationship; and constructing a field processing map. The flowchart module is used to implement S2: calling the functions and order in the field processing graph, grouping the sample field, the reagent field and the equipment field according to the processing flow, extracting the path connection direction, intersection nodes and extension relationships, and generating a field flowchart; The field combination module is used to implement S3: call the paths and connections in the field flowchart, aggregate duplicate fields, determine the continuity of field combinations, extract field combinations with flow connectivity features, and generate processing flow segments. The node mapping module is used to implement S4: calling the field combination in the processing flow segment, collecting the deployment order parameters and execution channel identifiers of the field processing in the endpoint nodes of the process path, comparing the deployment order and execution channel of the edge nodes, mapping the field combination to the nodes with execution capabilities, and generating a module mapping sequence; The process integration module is used to implement S5: calling the order and path in the module mapping sequence, reorganizing the call chain of the field combination, recording the transmission path and execution order of the fields between nodes, and constructing a big data processing process integration structure.

Images