A data filtering method based on custom process components

Abstract

This invention discloses a data filtering method based on custom process components, belonging to the field of data filtering technology. When dealing with new production scenarios or adjusting existing processes, users can freely combine various functional components through drag-and-drop to define business processes that meet specific needs, avoiding the tedious process of recoding every time a change occurs. Simultaneously, the dynamic judgment and path selection functions of condition nodes enable the system to dynamically adjust subsequent operation paths based on the actual data processing results, further improving the adaptability and responsiveness of the business process. Ultimately, this solution provides an efficient, flexible, and customizable solution, significantly reducing development and maintenance costs and enabling rapid adaptation to changing production needs.

Description

Technical Field

[0001] This invention relates to the field of data filtering technology, and more specifically to a data filtering method based on a custom workflow component. Background Technology

[0002] In industrial manufacturing, the production process is typically divided into multiple operations, each with several steps performed by different machines. However, selecting the appropriate machine for the next processing step is a complex task. This selection process is influenced by various factors, such as product priority, expiration date, product value, and process requirements. In practice, selection, filtering, and sorting are the core operations, involving sorting and filtering data according to different criteria and selecting suitable resources. Existing technologies often rely on static sorting and filtering algorithms, requiring the repetitive writing of the same code for different factories and products, and cannot easily adapt to changing production needs.

[0003] However, the aforementioned existing technologies face the following problems. First, whenever new production scenarios need to be addressed or existing processes need to be adjusted, existing systems typically require developers to write a large number of sorting and filtering algorithms, resulting in long development cycles and low efficiency. Second, existing systems lack flexibility and cannot quickly adapt to different factory needs or different production conditions. Each change or expansion of functionality requires manual modification of the underlying code, failing to meet rapidly changing business requirements. These problems limit the breadth and flexibility of existing technologies in practical applications. Summary of the Invention

[0004] The purpose of this invention is to solve the problems mentioned above and provide a data filtering method based on custom process components.

[0005] This invention proposes a data filtering method based on a custom process component, the method comprising:

[0006] Build a component composition platform, which contains multiple functional components, each identified by a unique ID;

[0007] By defining and connecting functional components through a visual interface, users can combine multiple components into a custom business process diagram according to their needs by drawing flowcharts.

[0008] Generate a unique identifier and initialize the process execution;

[0009] The execution process components are as follows: the first component of the flowchart queries and loads data from the database or data source according to preset conditions, and then each component is executed in sequence. The execution results are stored through data structures and passed to the next component to ensure the continuity and consistency of the data flow.

[0010] During execution, if a component is a condition node, the system determines whether to continue executing the next component based on the processing result of the current data. If the condition is met, a pre-defined component is executed; otherwise, another component is executed, thus realizing the dynamic adjustment of business logic.

[0011] After each component completes its execution, the result is stored in a database or cache system in a specific data structure. The next component retrieves the execution result of the previous step from the storage by obtaining a unique identifier and continues to execute the next data processing step.

[0012] Once all components have completed their execution, the final filtering, selection, and sorting results are output to downstream systems or other applications as the basis for subsequent processing or decision-making.

[0013] Optionally, the components include, but are not limited to, a process filtering component, a process recipe matching component, a QTime sorting component, a priority sorting component, and a sorting component, used to implement data filtering, sorting, and selection operations.

[0014] Optionally, multiple components can be combined into a custom business process diagram according to requirements, including:

[0015] The flowchart defines the execution order and data transfer rules of each component, and users can flexibly adjust the order of components and condition judgments according to business needs.

[0016] Optionally, generating a unique identifier and initializing the process execution includes:

[0017] Each time a flowchart is executed, the system automatically generates a unique identifier (UID) for that flowchart, which is used to record and track the execution information of that flowchart. The UID is used to identify the state of the data flow and component execution process.

[0018] The steps to determine whether to continue executing the next component based on the current data processing results are as follows:

[0019] Calculate the data fluctuation stability index and energy consumption efficiency index based on the execution results of the current component. Add the data fluctuation stability index and energy consumption efficiency index together to obtain the feasibility index for subsequent processes. Determine whether to continue executing the next component based on the feasibility index of subsequent processes.

[0020] Optionally, the calculation steps for the data fluctuation stability index are as follows:

[0021] Extract the raw data time series from the execution result of the current component, where each data point represents a data value at a specific point in time, and the timestamp corresponds one-to-one with the data.

[0022] In a time series, select a sliding window of a fixed size, calculate the standard deviation of the data within that window, calculate the difference between the standard deviation of the current window and the standard deviation of the previous window, and then divide the difference by the standard deviation of the previous window to obtain the rate of change of the standard deviation between adjacent sliding windows.

[0023] Calculate the probability distribution of all standard deviation rates of change, and use the formula for information entropy to calculate the entropy value of the probability distribution of all standard deviation rates of change. Divide the entropy value by the theoretical maximum entropy value to obtain the deviation.

[0024] The time series is divided into multiple sub-intervals, the fluctuation of each sub-interval is calculated, and the similarity of each sub-interval is measured to obtain the fractal dimension.

[0025] Multiply the rate of change of standard deviation, deviation, and fractal dimension, and use the reciprocal of the result as the data volatility stability index.

[0026] Optionally, the calculation steps for the energy consumption efficiency index are as follows:

[0027] The system acquires energy consumption data and task output data at multiple consecutive time points within the current task execution cycle, which are recorded as the instantaneous energy consumption value and unit output value at each time point, respectively. For each time point, the system calculates the unit energy consumption ratio at that time point by squaring the instantaneous energy consumption value at that time point and dividing it by the corresponding unit output value plus one, thereby obtaining the unit energy consumption ratio at that time point and forming a time series of unit energy consumption ratios.

[0028] The time series of unit energy consumption ratio is analyzed for changes in the direction of adjacent values ​​to determine the increasing or decreasing trends between adjacent data points in the series, and to identify the points of change from "increasing to decreasing" or "decreasing to increasing". The total number of all change points in the entire series is counted as the number of inflection points in the series. The number of inflection points is divided by the total time interval length of the series to obtain the inflection point density, which is used to reflect the fluctuation frequency in the change process of unit energy consumption ratio.

[0029] Plot time on the horizontal axis and energy consumption ratio on the vertical axis, map each data point in the time series to a two-dimensional coordinate point; calculate the total length of the curve connecting all data points as the true path length, and calculate the straight-line distance between the start and end points of the sequence as the shortest path length; divide the shortest path length by the true path length to obtain the path compression.

[0030] Analyze the changing trend of unit output value within the task cycle and identify the time period of continuous increase; for each continuous increase output segment, determine whether the corresponding energy consumption value decreases only after the output increases; if there is a time period in which the energy consumption decrease lags behind the output increase, then accumulate the duration of all such lag segments and divide by the total duration of the entire task cycle to obtain the energy consumption lag rate.

[0031] The energy consumption efficiency index is constructed by logically combining inflection point density, path compression, and energy consumption lag rate. First, the inflection point density is added to the energy consumption lag rate and multiplied by the reciprocal of the path compression to form an inefficiency disturbance factor. Then, one is added to this disturbance factor and the reciprocal is taken to obtain the final energy consumption efficiency index.

[0032] Optionally, the steps for determining whether to continue executing the next component based on the availability of subsequent processes include:

[0033] The feasible index is compared with the preset feasible index threshold. If the feasible index is not less than the preset feasible index threshold, the condition is met, and the next predetermined component is executed according to the flowchart.

[0034] If the operable index is less than the preset operable index threshold, it means that the condition is not met; otherwise, another component is executed to achieve dynamic adjustment of the business logic.

[0035] The beneficial effects of this invention are:

[0036] This invention proposes a data filtering method based on custom process components. When dealing with new production scenarios or adjusting existing processes, users can freely combine various functional components through drag-and-drop to define business processes that meet specific needs, avoiding the tedious process of recoding for each change. Simultaneously, the dynamic judgment and path selection functions of condition nodes enable the system to dynamically adjust subsequent operation paths based on the actual data processing results, further improving the adaptability and responsiveness of the business process. Ultimately, this solution provides an efficient, flexible, and customizable solution that significantly reduces development and maintenance costs and can quickly adapt to changing production needs. Attached Figure Description

[0037] The invention will now be further described with reference to the accompanying drawings.

[0038] Figure 1 This is a flowchart of a data filtering method based on a custom process component. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention.

[0041] This invention provides a data filtering method based on a custom process component. See also... Figure 1 , Figure 1 A flowchart illustrating a data filtering method based on a custom process component, provided as an embodiment of the present invention. The method includes the following steps:

[0042] S1: Build a component composition platform. The platform contains multiple functional components, each of which is identified by a unique ID.

[0043] S2: Define and connect functional components through a visual interface. Users can combine multiple components into a custom business process diagram according to their needs by drawing flowcharts.

[0044] S3: Generate a unique identifier (UID) and initialize the process execution;

[0045] S4: Execution process component. The first component of the flowchart queries and loads data from the database or data source according to preset conditions. Subsequently, each component is executed in sequence. The execution results are stored through data structures (such as Map) and passed to the next component to ensure the continuity and consistency of the data flow.

[0046] S5: During execution, if a component is a condition node, the system determines whether to continue executing the next component based on the processing result of the current data. If the condition is met, a pre-defined component is executed; otherwise, another component is executed, thus realizing the dynamic adjustment of business logic.

[0047] S6: After each component completes its execution, the result is stored in a database or cache system in a specific data structure (such as a Map). The next component retrieves the execution result of the previous step from the storage by obtaining the UID and continues to execute the next data processing step.

[0048] S7: After all components have completed their execution, the final filtering, selection, and sorting results are output to downstream systems or other applications as the basis for subsequent processing or decision-making.

[0049] Based on the data filtering method based on custom process components provided by this invention, when dealing with new production scenarios or adjusting existing processes, users can freely combine various functional components by dragging and dropping to define business processes that meet specific needs, avoiding the tedious process of recoding every time a change occurs. Simultaneously, the dynamic judgment and path selection functions of condition nodes enable the system to dynamically adjust subsequent operation paths based on the actual data processing results, further improving the adaptability and responsiveness of the business process. Ultimately, this solution provides an efficient, flexible, and customizable solution, significantly reducing development and maintenance costs and enabling rapid adaptation to changing production needs.

[0050] In one embodiment, S1: Build a component composition platform, the platform containing multiple functional components, each identified by a unique ID, including:

[0051] The components include, but are not limited to, process filtering components, process recipe matching components, QTime sorting components, priority sorting components, and sorting components, which are used to implement data filtering, sorting, and selection operations.

[0052] It's important to note that the goal of this step is to build a multifunctional component combination platform. The core of this platform is the integration of multiple functional components to form a highly flexible and scalable system. Within this platform, each functional component has a unique identifier (ID) for system management, scheduling, and execution. These functional components include, but are not limited to, process filtering components, process recipe matching components, QTime sorting components, priority sorting components, and sorting components. These components work together to meet data filtering, sorting, and selection operations under different business needs. For example, the process filtering component filters a list of equipment that meets the product's process requirements. It can filter equipment based on specific process parameters such as temperature, humidity, and processing time. The process recipe matching component filters suitable equipment based on the product's recipe requirements. It can select the most suitable production equipment based on the recipe requirements of different products (such as raw materials, proportions, etc.). Additionally, the QTime sorting component handles the time requirements between product processes, ensuring that each process is completed on time. For example, some process steps need to be completed within a specific time. Exceeding the time limit may affect product quality. The QTime sorting component prioritizes these products, ensuring that time-sensitive processes are processed promptly. Prioritization components prioritize tasks based on product importance or customer needs, while sorting components can sort equipment or data according to other criteria, thereby ensuring optimal allocation of production resources.

[0053] By flexibly combining these components, users can customize processing flows to suit specific production needs. Whether it's sorting, filtering, or selection, these functional components can be combined to meet diverse business requirements.

[0054] In one implementation, S2: Functional components are defined and connected through a visual interface. Users combine multiple components into a custom business process diagram according to their needs by drawing flowcharts, including:

[0055] The flowchart defines the execution order and data transfer rules of each component, and users can flexibly adjust the order of components and condition judgments according to business needs.

[0056] It's important to note that in this step, users define and connect various functional components through an intuitive visual interface. This interface provides a graphical operating platform, allowing users to easily build business process diagrams simply by dragging and dropping, without writing complex code. By drawing the flowchart, users arrange multiple functional components according to requirements, forming a clear operational flowchart. Each component has a specific position in the flowchart, which defines the execution order, data flow, and transmission rules of each component. This allows the system to execute each component according to the user's design, ensuring data flows smoothly from one component to the next. For example, suppose a user needs to design a production process. First, the process filtering component filters equipment based on process requirements; next, the formula matching component ensures the equipment meets the formula requirements; then, the data passes through the QTime sorting component to ensure the process is completed on time; finally, the priority sorting component adjusts the execution order based on product priority. Users drag and drop these components to adjust their order in the flowchart and define their connections. The data transmission rules between these components are also set in the flowchart, ensuring smooth data flow and that each component receives the necessary input data. Furthermore, users can flexibly adjust the order and conditional judgments of components according to actual business needs. For example, in some cases, the execution order of a component may need to be dynamically adjusted based on the result of the previous component. By adding conditional judgment nodes, users can control the execution path of certain components in the flowchart. For instance, if the screening result of the process filtering component does not meet certain criteria, the process may be transferred to another alternative component for further processing; if the screening result is qualified, the process will continue to execute subsequent steps. This visual design approach not only reduces the complexity of system development but also greatly improves the system's flexibility and ease of use, enabling users to quickly adjust the production process according to specific needs and meet different business scenarios.

[0057] In one implementation, S3: Generating a unique identifier (UID) and initializing the process execution includes:

[0058] Each time the flowchart is executed, the system automatically generates a unique identifier (UID) for the process to record and track the execution information of the process. The UID is used to identify the data flow and the state of the component during execution.

[0059] It's important to note that in this step, each time a user initiates and executes the defined flowchart, the system automatically generates a unique identifier (UID) for that flowchart. This UID is used to identify and track the execution of the flowchart. The UID is a globally unique identifier that ensures all data, states, and results can be accurately tracked and managed throughout the entire flowchart execution process. Whenever the system executes a new business process, it assigns a new UID to that process, ensuring that the execution of all components and data transfer within the process can be clearly recorded and distinguished, avoiding confusion between different process executions. The UID is not only used to identify the process itself but also to track the execution status of each component within the process. For example, when a process jumps from one component to the next, the UID is passed as the "identity" of that process, ensuring that the input and output data of each component are correctly correlated. For instance, in a production scheduling process, when data is transferred from the process filtering component to the process recipe matching component, the UID acts as an identifier, ensuring that the data flow between these two components is orderly and that the status of each component is accurately reflected in the system. For example, suppose a factory's production process includes screening, sorting, and scheduling operations. When a user initiates the process, the system assigns it a UID, such as UID12345. Subsequently, all executing components (such as process filtering, recipe matching, sorting, etc.) associate their execution results and status with UID12345, ensuring that data at each stage can be traced back to this specific execution instance. When the process is completed, the final result is also output along with this UID, provided to downstream systems or other applications as a basis for subsequent operations. The generation and use of UIDs not only improves the accuracy of process management but also provides strong support for data tracking, logging, and troubleshooting. Through UIDs, the system can ensure the integrity and traceability of each executed process and data, greatly improving system stability and maintainability.

[0060] In one embodiment, S4: Execution process component. The first component of the flowchart queries and loads data from the database or data source according to preset conditions. Subsequently, each component is executed in sequence. The execution result is stored through a data structure (such as Map) and passed to the next component to ensure the continuity and consistency of the data flow.

[0061] It's important to note that in this step, the core of the execution flow components is to execute each functional component sequentially according to the user-defined flowchart order, ensuring accurate operation and data transfer for each component. First, the first component in the flowchart queries and loads the required data from a database or other data source based on preset conditions. This data source can be a relational database, a non-relational database, a caching system, or other data storage media, depending on the needs of the business scenario. For example, during production scheduling, the process filtering component might need to query equipment information or process requirements from the database to filter out equipment that meets the conditions. Once the data is loaded, the first component performs its pre-defined operations (such as filtering and processing) and then stores the results using a specific data structure (such as a Map). A Map is a key-value pair storage structure that can efficiently store processed data and pass it to the next component in key-value format. For example, assuming the first component is the process filtering component, its output might be a filtered list of equipment, stored using a Map and passed to the next component in the form of "equipment list":[equipment1, equipment2, equipment3]. Subsequently, the system executes the second component according to the flowchart order. The second component retrieves the execution results (such as a list of equipment) from the previous component's Map and then continues its operations. For example, after receiving the equipment list, the process recipe matching component filters out the equipment that meets the criteria based on the equipment's characteristics and recipe requirements, stores the results in the Map again, and passes them to the next component. The execution results of each component are maintained in the Map structure and passed to the next component through this structure, ensuring the continuity and consistency of the data flow between components. This step-by-step execution and data transfer design ensures that each component receives accurate input and outputs correct results, providing effective data for subsequent components. In this way, the data processing becomes orderly and efficient, ensuring error-free data collaboration between components in complex business processes. For example, in a production scheduling process, the process filtering component queries all equipment information from the database and filters out equipment that meets the process requirements, storing the output as a Map; then, the recipe matching component retrieves this equipment list, further filters it according to the recipe requirements, outputs the equipment that meets the criteria, and finally hands it over to the sorting component for priority sorting, ensuring that equipment scheduling in the production process meets expectations. Each component depends on the output data of the previous component, forming a complete data flow. In this way, the entire process is executed continuously and in an orderly manner, with each component executing sequentially to ensure the correct transmission and processing of data.

[0062] In one embodiment, S5: During execution, if a component is a condition node, the system determines whether to continue executing the next component based on the processing result of the current data. If the condition is met, a predetermined component is executed; otherwise, another component is executed, thereby realizing the dynamic adjustment of business logic.

[0063] In one implementation, the system determines whether to continue executing the next component based on the processing result of the current data as follows:

[0064] Calculate the data fluctuation stability index and energy consumption efficiency index based on the execution results of the current component. Add the data fluctuation stability index and energy consumption efficiency index together to obtain the feasibility index for subsequent processes. Determine whether to continue executing the next component based on the feasibility index of subsequent processes.

[0065] In one embodiment, the calculation steps for the data volatility stability index are as follows:

[0066] Extract the raw data time series from the execution result of the current component, where each data point represents a data value at a specific point in time, and the timestamp corresponds one-to-one with the data.

[0067] In a time series, select a sliding window of a fixed size, calculate the standard deviation of the data within that window, calculate the difference between the standard deviation of the current window and the standard deviation of the previous window, and then divide the difference by the standard deviation of the previous window to obtain the rate of change of the standard deviation between adjacent sliding windows.

[0068] Calculate the probability distribution of all standard deviation rates of change, and use the formula for information entropy to calculate the entropy value of the probability distribution of all standard deviation rates of change. Divide the entropy value by the theoretical maximum entropy value to obtain the deviation.

[0069] The time series is divided into multiple sub-intervals, the fluctuation of each sub-interval is calculated, and the similarity of each sub-interval is measured to obtain the fractal dimension.

[0070] Multiply the rate of change of standard deviation, deviation, and fractal dimension, and use the reciprocal of the result as the data volatility stability index.

[0071] It's important to note that the data acquisition method for each step in calculating the data fluctuation stability index is as follows: First, the raw data time series is extracted from the current component's execution result. This time series reflects data values ​​at different points in time, typically from real-time monitoring devices or sensors. Each data point includes a timestamp and a corresponding data value. For example, if it's temperature sensor data, each data point represents the temperature value at a certain moment, and the timestamp records the specific sampling time. Next, to calculate the standard deviation, we need to select a fixed-size sliding window (e.g., data from the past 10 seconds) from the extracted time series. Within this window, we calculate the standard deviation of the data, representing the fluctuation range of the data during that time period. Then, we compare the standard deviation of the current window with the standard deviation of the previous window, calculate the difference between them, and divide it by the standard deviation of the previous window to obtain the rate of change of the standard deviation. This data is obtained by calculating the standard deviation of the sliding window at each moment. Then, after calculating the rate of change of the standard deviation for each sliding window, we statistically analyze these rates of change to obtain their probability distribution. We use this distribution to calculate the information entropy, which measures the complexity and uncertainty of the distribution of the rate of change of the standard deviation. Specifically, by statistically analyzing the frequency of change of each standard deviation, the entropy value of the distribution of that rate of change is calculated using the formula for information entropy. Then, the deviation is obtained by comparing it with the theoretical maximum entropy value. Finally, the fractal dimension is obtained by dividing the time series into multiple sub-intervals and calculating the data volatility within each sub-interval. Specifically, the time series is divided into several smaller sub-intervals, the volatility (such as mean or standard deviation) of the data within each sub-interval is calculated, and then the volatility characteristics of these sub-intervals are compared to measure their similarity. By calculating and comparing the volatility of these sub-intervals, the fractal dimension of the time series is obtained, representing the self-similarity and complexity of the data. Through these steps, the acquisition method of each dataset is based on time series analysis, ensuring the continuity and stability of the calculation process, thus deriving the data volatility stability index.

[0072] It's important to note that the data volatility stability index is an indicator used to measure the stability and regularity of data volatility within a system. It reflects whether data volatility remains within a controllable range over a period of time and whether abnormal fluctuations exist. Specifically, the data volatility stability index considers the magnitude of data changes, the regularity of volatility, and the presence of unstable or unpredictable fluctuations. The index value typically ranges from 0 to 1. A value closer to 1 indicates more stable data volatility and a more normal operating state for the system; a value closer to 0 indicates greater data volatility, suggesting the system may be in an unstable or abnormal state, posing a risk to continuing subsequent operations. A higher data volatility stability index indicates less data volatility, higher system stability, and that the current execution process is not affected by external interference or unforeseen events, allowing the system to continue operating smoothly. Conversely, a low data volatility stability index indicates greater data volatility in the system, potentially posing a risk of anomalies or malfunctions. Continuing to execute subsequent components in this case may lead to unstable results or erroneous decisions, or even trigger production problems or system crashes. For example, in a production scheduling system, the operating status of equipment can affect the scheduling of subsequent production steps. If the equipment's fluctuation stability index is high (e.g., 0.9), it indicates that the equipment is operating smoothly and its performance is normal. The system can safely proceed to the next step in the production plan. Conversely, if the equipment's fluctuation stability index is low (e.g., 0.3), it indicates that the equipment is unstable and may be at risk of malfunction or performance degradation. Continuing to execute the next step in this case could lead to production interruptions, product quality issues, or even equipment damage. Therefore, a larger fluctuation stability index indicates that the system is operating well and is ready to proceed to the next step; conversely, a smaller fluctuation index suggests that the system may need adjustment, inspection, or repair to avoid problems caused by continuing execution.

[0073] The reason for calculating the data volatility stability index using the above method, rather than traditional methods such as weighted average, single variance index, or simple threshold judgment, is that this method can more comprehensively, deeply, multidimensionally, and dynamically characterize the essential characteristics of data volatility. First, by using the sliding window standard deviation and its rate of change, the severity and speed of local fluctuations in the time series can be captured, reflecting whether the current system exhibits sudden oscillating behavior. Second, by calculating the probability distribution information entropy of the standard deviation rate of change and further normalizing it to deviation, the unpredictability and complexity of the volatility pattern can be measured, thereby determining whether the system exhibits trend-based instability. Third, by using the fractal dimension calculation of the time series, it is possible to identify whether the self-similar structure of the data at multiple scales is disordered, thus assessing whether the system has transitioned from a structurally stable state to a chaotic state. Multiplying these three factors and taking their reciprocals constructs a non-linear amplification response mechanism: once any dimension experiences severe fluctuation, the entropy value deviates significantly, or the fractal dimension becomes high, the index value will decrease significantly, keenly indicating that the system should not continue executing subsequent components. Compared to conventional methods, the greatest advantage of this approach is its dynamic nature, local sensitivity, and multi-dimensional structural understanding. It accurately reflects situations where "overall statistics are stable but local disturbances are severe," avoiding the omission of key anomalies by traditional mean-based methods. This provides a more reliable and refined basis for judgment in practical engineering or decision-making systems. For example, in a scenario where an industrial device maintains overall stability but experiences sudden high-frequency oscillations over a certain period, traditional average volatility might be ineffective. However, this method can promptly capture, quantify, and quickly report such local non-steady states, enhancing the system's proactive response capability and safety margin. In summary, this method is not only logically rigorous and computationally progressive, but it also truly achieves a fusion judgment of "structure, speed, and complexity," making the obtained stability index more meaningful and practical in engineering.

[0074] In one implementation, the calculation steps for the energy consumption efficiency index are as follows:

[0075] The system acquires energy consumption data and task output data at multiple consecutive time points within the current task execution cycle, which are denoted as instantaneous energy consumption value and unit output value at each time point, respectively. For each time point, the system calculates the unit energy consumption ratio at that time point by squaring the instantaneous energy consumption value at that time point and dividing it by the corresponding unit output value plus one. This yields the unit energy consumption ratio at that time point, forming a time series of unit energy consumption ratios, which is used to represent the degree of influence of energy consumption on unit output at each time point.

[0076] The time series of unit energy consumption ratio is analyzed for changes in the direction of adjacent values ​​to determine the increasing or decreasing trends between adjacent data points in the series, and to identify the points of change from "increasing to decreasing" or "decreasing to increasing". The total number of all change points in the entire series is counted as the number of inflection points in the series. The number of inflection points is divided by the total time interval length of the series to obtain the inflection point density, which is used to reflect the fluctuation frequency in the change process of unit energy consumption ratio.

[0077] With time as the horizontal axis and unit energy consumption ratio as the vertical axis, each data point in the time series is mapped to a two-dimensional coordinate point; the total length of the curve connecting all data points is calculated as the true path length, and the straight-line distance between the start and end points of the series is calculated as the shortest path length; the shortest path length is divided by the true path length to obtain the path compression, which is used to measure the fluctuation range and path stability of the unit energy consumption ratio change process.

[0078] Analyze the changing trend of unit output value within the task cycle and identify the time period of continuous increase; for each continuously increasing output segment, determine whether the corresponding energy consumption value decreases only after the output increases; if there is a time period in which the energy consumption decrease lags behind the output increase, then accumulate the duration of all such lag segments and divide by the total duration of the entire task cycle to obtain the energy consumption lag rate, which is used to measure the system's response speed to energy consumption control after the task output efficiency is improved.

[0079] An energy consumption efficiency index is constructed by logically combining inflection point density, path compression, and energy consumption lag rate. First, the inflection point density is added to the energy consumption lag rate and multiplied by the reciprocal of the path compression to form an inefficiency disturbance factor. Then, one is added to this disturbance factor and the reciprocal is taken to obtain the final energy consumption efficiency index. The index value is between 0 and 1. The larger the value, the higher the energy use efficiency, the more agile the system response, and the smaller the fluctuation.

[0080] It should be noted that in the calculation of the above-mentioned energy consumption effectiveness index, the data is acquired in the following ways: Energy consumption data and task output data within the current task execution cycle can be collected in real time through the system's built-in production monitoring module. Energy consumption data refers to the current, voltage, or other energy consumption information recorded at various time points, which is recorded and uploaded by energy consumption sensors such as electricity meter modules and flow meters at a preset sampling frequency. Task output data is recorded in real time by the equipment's production control system (such as PLC, MES, etc.) for the output or processing completed per unit time at each time point, and marked with the task ID and execution time to form a "time-instantaneous energy consumption value - unit output value" pair with precise timestamp alignment. These data are automatically processed by the system and arranged into a structured time series according to the time sequence for subsequent operations such as ratio calculation, path analysis, trend identification, and delay assessment, ensuring that the basic data of each indicator comes from the original and verifiable data source in the actual production process, and has accuracy and traceability.

[0081] It should be noted that the Energy Consumption Effectiveness Index is a comprehensive numerical indicator used to measure the coordination, efficiency, and system response sensitivity between unit output and energy consumption during task execution. This index comprehensively reflects the quality of energy utilization during the current task execution process from multiple dimensions by analyzing the fluctuation trend of the unit energy consumption ratio (represented by inflection point density), the stability of the change trajectory (represented by path compression), and the timeliness of changes in energy consumption response output efficiency (represented by energy consumption lag rate). Specifically, if the unit energy consumption ratio fluctuates frequently throughout the task, has a tortuous trajectory, and energy consumption adjustments lag behind changes in task output, it indicates that the system's energy efficiency control has problems such as large fluctuations, slow adjustments, and poor feedback. In this case, the Energy Consumption Effectiveness Index will tend to be a small value. Conversely, if the unit energy consumption ratio changes smoothly, the path is close to a straight line, and energy consumption decreases rapidly after the task output increases and responds promptly to system adjustments, the index will tend to be 1, indicating high system operating efficiency, accurate energy consumption strategy, and stable and reliable energy use. Therefore, a higher index indicates that the current task performs exceptionally well in terms of energy efficiency per unit output, and the system is in a healthy state of high efficiency, low consumption, and rapid response. This not only ensures the smooth execution of the current task but also provides a stable system foundation and resource guarantee for the execution of the next task component, resulting in higher process continuity and execution feasibility. For example, in a complex equipment scheduling process, if the unit energy consumption ratio steadily decreases over several consecutive execution cycles, the system response speed is fast, and the path changes are stable, then the energy consumption efficiency index remains high. Based on this index, the system can automatically determine that there is no need to wait or reconfigure, and directly schedule subsequent tasks to ensure overall process efficiency and energy cost control. Thus, this index is not only used for energy efficiency assessment of a single task but also a key decision factor in adaptive process advancement.

[0082] It should be noted that the reason for using the above method to calculate the energy consumption efficiency index, instead of the conventional methods such as dividing total energy consumption by total output, weighted average energy efficiency, or linear trend fitting, is mainly because this method can comprehensively capture the true state and potential risks of energy consumption efficiency during task execution from multiple dynamic dimensions, nonlinear characteristics, and response lag behavior. First, by constructing and identifying the "inflection point density" through the time series of unit energy consumption ratios, it can reveal whether there are frequent and unstable fluctuations in system energy efficiency, avoiding the problem of masking the problem of drastic phased energy consumption fluctuations by relying solely on average values. Second, "path compression" measures the overall stability and redundancy of the energy efficiency change process. Its introduction avoids the drawback of traditional trend analysis failing in nonlinear intervals, allowing the risk of system state deviation in tortuous paths to be perceived in a timely manner. Third, "energy consumption lag rate" effectively captures the synchronicity between output efficiency improvement and energy consumption response, making up for the limitations of traditional calculation methods that lack consideration of lag and inertial adjustment mechanisms. These three dimensions are independent yet highly complementary, without involving weighted summation or unknown factors. They can extract the inherent logic and operational characteristics of system energy efficiency control through structural combination without relying on empirical parameter tuning. The resulting energy consumption efficiency index not only retains the ability to judge overall efficiency macroscopically but also possesses sensitivity to microscopic dynamics such as instantaneous anomalies, structural changes, and slow response times. This provides more scientific, accurate, and forward-looking decision support for automated system scheduling, adaptive process management, and resource allocation, significantly outperforming traditional methods in complex task chains.

[0083] In one embodiment, the step of determining whether to continue executing the next component based on the feasibility index of subsequent processes includes:

[0084] The feasible index is compared with the preset feasible index threshold. If the feasible index is not less than the preset feasible index threshold, the condition is met, and the next predetermined component is executed according to the flowchart.

[0085] If the operable index is less than the preset operable index threshold, it means that the condition is not met; otherwise, another component is executed to achieve dynamic adjustment of the business logic.

[0086] It should be noted that in the data filtering method based on process components, to achieve flexibility and intelligence in business logic, the system sets a "feasibility index" as the core indicator for dynamically determining the execution path. Specifically, when the process execution reaches a component node containing conditional judgments, the system first calculates the "feasibility index" of the current task and compares it with a preset "feasibility index threshold." If the feasibility index is greater than or equal to the threshold, it indicates that the stability, energy efficiency, and other comprehensive indicators of the current data processing result have reached the system's expected standards, meaning that subsequent tasks have a good execution foundation. At this time, the system will continue to execute the next predetermined component according to the normal path set in the flowchart, realizing the natural advancement of the business flow. Conversely, if the feasibility index is lower than the threshold, it indicates that the current processing result has a certain degree of instability, inefficiency, or risk of not being suitable for immediate continuation. In this case, the system will automatically trigger the "alternative path" or "branch component" set in the flowchart, which may perform operations such as exception handling, supplementary verification, and data backhaul, thereby realizing dynamic adjustment and exception handling of the business process. For example, if the current component in the flowchart is processing data filtering for a certain process, and the calculated data fluctuation stability index and energy consumption effectiveness index are low, resulting in a combined feasible index below the threshold, the system may skip the subsequent sorting component and instead enter a data optimization component to correct the abnormal data, ensuring the overall quality and robustness of the process. This path diversion mechanism based on index judgment breaks through the rigid constraints of traditional "fixed process" execution logic, enabling the process to flexibly adjust its path when facing different task states, data quality, or differences in system response capabilities, ensuring that the entire execution process is more intelligent, robust, and adaptable.

[0087] In one embodiment, S6: After each component completes its execution, the result is stored in a database or cache system in a specific data structure (such as a Map). The next component retrieves the execution result of the previous step from the storage by obtaining the UID and continues to execute the next data processing step.

[0088] It's important to note that during the execution of component flows, to ensure efficient data transfer and state consistency between components, the system encapsulates and stores the execution results of each component after completion using a unified data structure. Commonly used data structures include Maps, JSON objects, or other key-value pairs, which clearly identify the processing results and status indicators of each field. The system stores this data structure in a pre-defined database or cache system, indexed by the current process's unique identifier (UID). This storage operation includes not only the component's output data but also auxiliary information such as execution timestamps, status codes, exception markers, and performance metrics, facilitating subsequent component calls or fault traceability. When the next component is ready to execute, the system automatically retrieves the corresponding result data from the previous component based on the UID and passes this data as input to the current component, achieving seamless data connection and state continuity across components. For example, in a process from a "process filtering component" to a "priority sorting component," after the filtering component completes, it stores the retained data records in the form of a Map, where the key is the field name, the value is the filtered dataset, and the UID is the storage index. When the sorting component starts, the system automatically retrieves the Map structure from the cache based on the UID and reads the valid data fields to perform the sorting operation. This mechanism effectively avoids the coupling problem caused by directly passing data between components through hard-coded methods, improving the system's modularity, scalability, and concurrent processing capabilities. It is particularly suitable for ensuring stability in complex scenarios such as parallel execution, simultaneous scheduling of multiple processes, and state reconstruction.

[0089] In one embodiment, after the entire custom process component has been executed, the system encapsulates and outputs the processing results of the last component, generating the final filtering, selection, and sorting results. This output is typically in structured data format, such as tables, JSON files, or database records, containing key data fields processed by each functional component, such as the filtered target record set, matched process recipe parameters, and sorted priority queues. These results are transmitted to downstream systems, such as production scheduling systems, quality management platforms, or business decision modules, through pre-defined interfaces to support subsequent automated execution or manual intervention. To ensure the integrity and compatibility of the data output, the system also includes execution logs, process status codes, and related marker fields, enabling downstream systems to accurately identify the current processing stage and context of the data. For example, in a semiconductor manufacturing scenario, wafer batches filtered by the "process filtering component" and the processing order sorted by the "QTime sorting component" are ultimately output to the MES system to drive the next stage of equipment operation instructions. This output mechanism not only ensures the consistency of data in the processing chain, but also provides traceable and verifiable data for subsequent systems, realizing a closed loop from custom process configuration to intelligent decision output, and improving the overall flexibility, transparency and automation level of the system.

[0090] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A data filtering method based on a self-defined process component, characterized in that, The method comprises the following steps: constructing a component combination platform, the platform comprising a plurality of functional components, each functional component being identified by a unique ID; defining and connecting the functional components through a visual interface, a user combining a plurality of components into a custom business flowchart according to requirements through the way of drawing a flowchart; generating a unique identifier and initializing flow execution; executing the flow components, the first component of the flowchart querying and loading data from a database or a data source according to preset conditions, and then each component being executed in sequence, the execution result being stored through a data structure and being passed to the next component to ensure the continuity and consistency of data flow; in the execution process, if a component is a conditional node, the system judges whether to continue executing the next component according to the processing result of the current data, if the condition is established, a predetermined component is executed, otherwise another component is executed, and the dynamic adjustment of business logic is realized; wherein the step of judging whether to continue executing the next component according to the processing result of the current data is: calculating a data fluctuation stability index and an energy consumption effective index according to the execution result of the current component, adding the data fluctuation stability index and the energy consumption effective index to obtain a performable index of the subsequent flow; and judging whether to continue executing the next component according to the performable index of the subsequent flow; the calculation steps of the data fluctuation stability index are: extracting a time series of original data in the execution result of the current component, wherein each data point represents a data value at a specific time point, and the timestamp is one-to-one corresponding to the data; in the time series, a fixed-size sliding window is selected, the standard deviation of the data in the window is calculated, the difference between the standard deviation of the current window and the standard deviation of the previous window is calculated, and then the difference is divided by the standard deviation of the previous window to obtain the change rate of the standard deviation between adjacent sliding windows; calculating the probability distribution of all standard deviation change rates, and calculating the entropy value of the probability distribution of all standard deviation change rates by using the formula of information entropy, and dividing the entropy value by the theoretical maximum entropy value to obtain the deviation degree; dividing the time series into a plurality of subintervals, calculating the fluctuation of each subinterval, and measuring the similarity of each subinterval to obtain the fractal dimension; multiplying the standard deviation change rate, the deviation degree and the fractal dimension, and taking the reciprocal of the multiplication result as the data fluctuation stability index; after each component is executed, the result is stored in a database or a cache system in a specific data structure, and the next component extracts the execution result of the previous step from the storage by obtaining a unique identifier to continue executing the next data processing step; when all components are executed, the final screening, selection and sorting result is output to a downstream system or other application program as a basis for subsequent processing or decision-making.

2. The data filtering method based on self-defined flow components according to claim 1, characterized in that, The components include but are not limited to process filtering components, process formula matching components, QTime sorting components, priority sorting components and sorting components, and are used to realize data filtering, sorting and selection operations.

3. The data filtering method based on self-defined flow components according to claim 1, characterized in that, The custom business flowchart is composed of a plurality of components according to requirements, which comprises: The flowchart defines the execution order and data transmission rules of each component, and the user can flexibly adjust the order and condition judgment of the components according to business requirements.

4. The data filtering method based on self-defined flow components according to claim 1, characterized in that, Generating a unique identifier and initializing flow execution comprises: Each time the flowchart is executed, the system automatically generates a unique identifier UID for the flow, which is used to record and track the execution information of the flow. The UID is used to identify the state of the data flow and component execution process.

5. The data filtering method based on self-defined flow components according to claim 1, characterized in that, The step of calculating the energy consumption efficiency index comprises: Obtaining energy consumption data at multiple consecutive time points in the current task execution period and task output data at the corresponding time points, denoted as the instantaneous energy consumption value and unit output value at each time point, respectively; For each time point, the unit energy consumption ratio at the time point is calculated by dividing the square of the instantaneous energy consumption value at the time point by the result of adding one to the corresponding unit output value, thereby obtaining the unit energy consumption ratio at the time point, forming a time sequence of unit energy consumption ratios; Adjacent value direction change analysis is performed on the time sequence of unit energy consumption ratios to determine the increasing and decreasing trends between each two adjacent data points in the sequence, and to identify the change points where the trend changes from increasing to decreasing or from decreasing to increasing; the number of all change points in the entire sequence is counted as the number of inflection points; the inflection point density is obtained by dividing the number of inflection points by the total time interval length corresponding to the sequence, which is used to reflect the fluctuation frequency in the unit energy consumption ratio change process; Each data point in the time sequence is mapped to a two-dimensional coordinate point with time as the horizontal axis and unit energy consumption ratio as the vertical axis; the total length of the curve formed by connecting all data points is calculated as the real path length, and the straight line distance between the start point and the end point of the sequence is calculated as the shortest path length; the path compression degree is obtained by dividing the shortest path length by the real path length; The trend of the unit output value is analyzed within the task period to identify the time period of continuous rise; for each continuous output rise period, it is determined whether the corresponding energy consumption value decreases after the output rises; if there is a time period where energy consumption lags behind the output rise, the duration of all such lag periods is added and divided by the total duration of the task period to obtain the energy consumption lag rate; The inflection point density, path compression degree and energy consumption lag rate are logically combined to construct the energy consumption efficiency index; first, the inflection point density and the energy consumption lag rate are added and multiplied by the reciprocal of the path compression degree to form a non-efficiency disturbance factor, then the disturbance factor is added by one and the reciprocal is taken to obtain the final energy consumption efficiency index.

6. The data filtering method based on self-defined flow components according to claim 1, characterized in that, The step of judging whether to continue executing the next component according to the subsequent flow comprises: Comparing the performable index with a preset performable index threshold value, if the performable index is not less than the preset performable index threshold value, it means that the condition is established, and the next predetermined component is executed according to the flowchart; If the performable index is less than the preset performable index threshold value, it means that the condition is not established, otherwise another component is executed to realize the dynamic adjustment of business logic.

Images