Stream-oriented computation-based visual computation model construction method

Abstract

The invention relates to the technical field of stream-oriented computing, in particular to a visual computing model construction method based on stream-oriented computing. The method comprises the following steps: acquiring a streaming computing data set; performing data preprocessing on the streaming computation data set to obtain a standard streaming computation data set; extracting calculation nodes of the standard stream-oriented calculation data set to obtain calculation node data; performing correlation analysis on the calculation node data to obtain node correlation data; performing network configuration analysis on the calculation node data to obtain node network configuration data; performing node type analysis on the calculation node data to obtain node type data; and performing node function analysis according to the node type data to obtain node function data. Therefore, by means of the Internet of Things technology, the intelligent control technology, the visualization technology and the block chain technology, the dynamic adjustment and distribution capacity of flow-oriented computing is improved, and the real-time monitoring and feedback capacity of a visual computing model of flow-oriented computing is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of streaming computing technology, and in particular to a method for constructing a visual computing model based on streaming computing. Background Art

[0002] The importance of streaming computing technology lies in its ability to process continuously generated data streams without waiting for the data to be fully stored or loaded. This real-time processing capability makes streaming computing widely used in fields such as financial transaction monitoring, intelligent manufacturing, and IoT device data analysis. Through streaming computing, users can obtain data in real time and make timely decisions instead of waiting for traditional batch processing to complete. Visual computing models play a key role in this process. It displays the flow of computing tasks and the flow of data through an intuitive graphical interface, allowing users to easily understand and manage complex computing processes. This visual approach not only improves the efficiency of user operations, but also helps users discover and solve potential data processing problems and optimize the use of computing resources. When implementing streaming computing, the role of computing nodes and the type of computing tasks are crucial. Computing nodes can play the role of data processing, storage, analysis, or decision support as needed, depending on the characteristics of the data stream and business needs. However, the existing streaming computing technology is limited in real-time performance and responsiveness when facing high load and large-scale data processing, and it is difficult to dynamically adjust resource allocation; and the visual computing model of streaming computing needs to be strengthened in terms of real-time monitoring feedback. Summary of the invention

[0003] Based on this, it is necessary to provide a method for building a visual computing model based on streaming computing to solve at least one of the above technical problems.

[0004] To achieve the above purpose, a method for constructing a visualization computing model based on streaming computing is provided, and the method comprises the following steps:

[0005] Step S1: Obtain a streaming computing data set; perform data preprocessing on the streaming computing data set to obtain a standard streaming computing data set;

[0006] Step S2: extract computing nodes of the standard streaming computing data set to obtain computing node data; perform correlation analysis on the computing node data to obtain node correlation data; perform network configuration analysis on the computing node data to obtain node network configuration data; perform node type analysis on the computing node data to obtain node type data; perform node function analysis based on the node type data to obtain node function data; integrate the node correlation data, node network configuration data and node function data to generate computing node characteristic data;

[0007] Step S3: extract computing tasks of the standard streaming computing data set to obtain computing task data; perform processing path analysis on the computing task data to obtain task processing path data; perform execution priority analysis on the computing task data to obtain task execution priority data; perform resource utilization analysis on the computing task data to obtain high resource demand task data; integrate task processing path data, task execution priority data and high resource demand task data to generate computing task management data;

[0008] Step S4: constructing a visual computing model according to the computing node characteristic data and the computing task management data to obtain a visual computing model;

[0009] Step S5: Perform network topology analysis on computing node data based on the visual computing model to obtain node topology data; perform dynamic network configuration adjustment on node network configuration data based on the node topology data to obtain computing node adjustment data; perform demand analysis on computing task data based on the visual computing model to obtain task demand data; perform intelligent task management control on task processing path data, task execution priority data and high resource demand task data based on the task demand data to generate computing task control data;

[0010] Step S6: Perform real-time visual feedback on the streaming computing through computing node adjustment data and computing task control data, and generate a real-time visual feedback report; upload the real-time visual feedback report to the blockchain network distributed storage to execute the visual computing job of the streaming computing.

[0011] The present invention obtains a streaming computing data set and performs data preprocessing on the streaming computing data set to ensure the consistency and accuracy of the data, thereby obtaining a standard streaming computing data set, providing a reliable basis for subsequent analysis and processing, and ensuring that the data used in the streaming computing process is verified and cleaned, thereby improving the accuracy and reliability of the calculation. By extracting the computing nodes of the standard streaming computing data set, we obtain computing node data, which provides basic data for subsequent analysis and ensures that the characteristics of each computing node are understood from a comprehensive perspective; by performing correlation analysis on the computing node data, we obtain node correlation data, which can help us understand the interactions and dependencies between different nodes. Correlation analysis helps to determine the data flow pattern between nodes, thereby optimizing the efficiency of data processing and transmission; by performing network configuration analysis on the computing node data, we obtain node network configuration data. The network configuration data provides important information about how nodes are connected to the network and how they interact with each other; by performing node type analysis on the computing node data, we obtain node type data, which can help us understand that each type of node has different processing and storage capabilities; by performing node function analysis based on the node type data, we obtain node function data. After determining the function of the node, we can further understand the overall role of the computing node; by integrating node correlation data, node network configuration data and node function data to generate computing node characteristic data, we can fully understand the computing node characteristic data. By extracting computing tasks from standard streaming computing data sets, computing task data is obtained; processing path analysis is performed on computing task data to obtain task processing path data, which can understand the processing process and path of each computing task; execution priority analysis is performed on computing task data to obtain task execution priority data, which can determine the execution order and priority of each task; resource utilization analysis is performed on computing task data to obtain high resource demand task data. By analyzing high resource demand task data, system resources can be reasonably allocated and managed to avoid resource bottlenecks and excessive consumption; task processing path data, task execution priority data and high resource demand task data are integrated to generate computing task management data, which will be used to formulate task scheduling later. Visual computing models are constructed by computing node characteristic data and computing task management data to obtain visual computing models, which can intuitively display the relationship between computing nodes and task execution status, and can more quickly discover potential problems and make decisions through visual presentation, thereby improving overall management efficiency.Through the visual computing model, the network topology analysis of the computing node data is performed to obtain the node topology data. By analyzing the network topology, the physical layout, data transmission path and load conditions between the computing nodes can be understood; based on the node topology data, the node network configuration data is dynamically adjusted to optimize network performance and resource utilization, and the computing node adjustment data is obtained. By adjusting the network configuration in real time, the network bandwidth is allocated, the routing path is adjusted or the connection method between nodes is changed according to actual needs; based on the visual computing model, the computing task data is analyzed for demand and the task demand data is obtained; based on the task demand data, the task processing path data, the task execution priority data and the high resource demand task data are intelligently managed and controlled to realize the intelligent regulation of the task processing path, execution priority and resource demand. Through intelligent task management, the computing task control data is generated, which can automatically respond to the demand changes of different tasks and improve the task execution efficiency and overall responsiveness. By adjusting the data of the computing nodes and the computing task control data, real-time visual feedback is provided to the streaming computing, and a real-time visual feedback report is generated, so that the execution and performance of the streaming computing can be monitored in real time; the real-time visual feedback report is uploaded to the distributed storage of the blockchain network to perform the visual computing job of the streaming computing, which ensures the security and non-tamperability of the report, and further enhances the transparency and trust of the streaming computing. Therefore, the present invention improves the dynamic adjustment and allocation capability of the streaming computing through the Internet of Things technology, intelligent control technology, visualization technology and blockchain technology, and strengthens the real-time monitoring and feedback capability of the visual computing model of the streaming computing.

[0012] Preferably, step S2 comprises the following steps:

[0013] Step S21: performing computing node feature recognition on the standard streaming computing data set to generate computing node feature information;

[0014] Step S22: extracting node data of computing node characteristic information to obtain computing node data;

[0015] Step S23: performing a correlation analysis of the interaction patterns of adjacent computing nodes on the computing node data to obtain node correlation data;

[0016] Step S24: Perform network configuration analysis on the computing node data to obtain node network configuration data;

[0017] Step S25: extracting attribute features of the computation node data to obtain node attribute feature data; classifying the computation node data into node types according to the node attribute feature data to obtain node type data; evaluating the node role according to the node type data to obtain node role evaluation data; and performing functional pathway analysis on the node role evaluation data to obtain node function data;

[0018] Step S26: Integrate the node association data, node network configuration data and node function data to generate computing node characteristic data.

[0019] The present invention uses a standard streaming computing data set to perform computing node feature recognition and generate computing node feature information, which can understand the role and performance of each node in the system, provide basic data for subsequent analysis, extract node data of computing node feature information, and obtain computing node data; perform adjacent computing node interaction mode correlation analysis on computing node data, which helps to understand the connection mode and interaction mode of computing nodes, and obtain node correlation data; perform network configuration analysis on computing node data, and by analyzing the network configuration of nodes, the network topology and network performance parameters can be clarified to obtain node network configuration data; extract attribute features of computing node data to obtain node attribute feature data; and perform node attribute feature analysis on computing nodes according to the node attribute feature data. Node type classification based on data can clarify the patterns and structures in computing node data, group computing node data according to commonalities or established classification standards, and obtain node type data; node role evaluation is performed based on node type data to clarify the role and influence of each node type in the entire streaming computing to obtain node role evaluation data; functional pathway analysis is performed on the node role evaluation data to ensure the specific role of computing nodes in streaming computing and obtain node function data; node association data, node network configuration data and node function data are integrated to generate computing node characteristic data, so that the computing node characteristic data includes the association between nodes, network configuration information and comprehensive node function characteristic data.

[0020] Preferably, step S24 includes the following steps:

[0021] Step S241: extracting and calculating the network characteristics of the node data to obtain the node network characteristic data;

[0022] Step S242: performing network connection analysis on the node network characteristic data to obtain node network connection data;

[0023] Step S243: performing connection status analysis according to the node network connection data to obtain node connection status data;

[0024] Step S244: Analyze the configuration of the node connection status data to obtain node network configuration data.

[0025] The present invention obtains node network feature data by extracting network features of computing node data, which can reflect the specific behaviors and attributes of computing nodes in the network, and is crucial for identifying key nodes in the network, evaluating network load and optimizing data flow; performing network connection analysis on node network feature data helps to understand the physical and logical connections between computing nodes, and clarify the connection mode and frequency between nodes, and obtain node network connection data; performing connection status analysis based on node network connection data can evaluate the quality and reliability of communication between nodes, understand the connection status, obtain node connection status data, and ensure that potential network problems are discovered in time; performing configuration status analysis on node connection status data, evaluating whether the settings of each node meet the requirements of security, performance and manageability, and obtaining node network configuration data, and ensuring that the network configuration meets expectations;

[0026] Preferably, step S3 comprises the following steps:

[0027] Step S31: performing computing task feature recognition on the standard streaming computing data set to generate computing task feature information;

[0028] Step S32: extracting task data of computing task characteristic information to obtain computing task data;

[0029] Step S33: performing data mapping on the computing task data processing information to generate a task processing flow chart; identifying the data flow direction and processing steps on the task processing flow chart to obtain task processing path data;

[0030] Step S34: performing execution priority analysis on the computing task data to obtain task execution priority data;

[0031] Step S35: Perform statistical analysis on the required resource amount of the computing task data to obtain task resource amount statistical data; arrange the task resource amount statistical data in descending order to obtain task resource amount descending data; perform heat map color mapping on the task resource amount descending data to generate a task resource amount heat map; extract the dark coordinate data of the task resource amount heat map to obtain high resource demand task data;

[0032] Step S36: Integrate the task processing path data, task execution priority data and high resource demand task data to generate computing task management data.

[0033] The present invention identifies computing task features through standard streaming computing data sets, generates computing task feature information, and accurately understands the basic properties and data requirements of different computing tasks; extracts task data of computing task feature information to obtain computing task data, which can more effectively understand and process data, thereby improving computing efficiency and accuracy; performs data mapping on computing task data processing information to generate a task processing flow chart; identifies data flow and processing steps on the task processing flow chart, and by identifying data flow and processing steps, can deeply analyze the execution path and key processing steps of the computing task, obtain task processing path data, and help optimize the task process; performs execution priority analysis on computing task data to obtain task execution priority data, which can reasonably arrange the execution order of tasks; and performs required resource quantity statistics on computing task data. Through the task resource quantity analysis, the statistical data of task resource quantity can be obtained, which can accurately calculate the resource quantity required for each task; the statistical data of task resource quantity can be arranged in descending order to obtain the task resource quantity descending data, which can clearly understand the computing tasks with the highest resource demand; the task resource quantity descending data can be mapped to the heat map color, which can intuitively display the resource demand of different tasks and generate the task resource quantity heat map. The task resource quantity heat map shows the difference in resource demand through the change of color depth; the dark coordinate data of the task resource quantity heat map is extracted to obtain the task data with high resource demand, which helps the generation team focus on and deal with the computing tasks with large resource demand; the task processing path data, task execution priority data and task data with high resource demand are integrated to generate computing task management data, which can form a comprehensive computing task management view.

[0034] Preferably, step S34 includes the following steps:

[0035] Step S341: extracting the execution characteristics of the computing task data to obtain task execution characteristic data;

[0036] Step S342: performing load analysis on the task execution characteristic data to obtain task execution load data;

[0037] Step S343: extracting the peak and valley values ​​of the task execution load data to obtain the task execution load peak and valley data;

[0038] Step S344: Calculate the average value of the peak and valley data of the task execution load to obtain the average value of the task execution load;

[0039] Step S345: Compare the task execution load data with the task execution load average value. When the task execution load average value is greater than the task execution load average value, mark the computing task as task execution high load data; when the task execution load value is less than or equal to the task execution load average value, mark the computing task as task execution low load data.

[0040] Step S346: performing high-priority classification on the task execution high-load data to obtain high-priority task data; performing low-priority classification on the task execution low-load data to obtain low-priority task data;

[0041] Step S347: Integrate high-priority task data and low-priority task data to obtain task execution priority data.

[0042] The present invention obtains task execution characteristic data by extracting and calculating the execution characteristics of task data, and can clarify the specific characteristics and execution status of each task; performs load analysis on the task execution characteristic data, and can deeply analyze the execution load status of the task under different time periods or conditions to obtain task execution load data; extracts the peak and valley values ​​of the task execution load data to obtain task execution load peak and valley data, which is helpful to identify the peak and valley values ​​of the load status during the task execution process; calculates the average value of the task execution load peak and valley data to obtain the task execution load average value, and calculates the task execution load average value to provide a benchmark for comparing the high and low states of the task execution load; compares the task execution load data with the task execution load average value, and according to whether the task execution load is higher or lower than the average value, the system can dynamically mark the load status of the task, and according to the load status of the task, arranges high-load tasks and low-load tasks in different priority queues respectively, integrates high-priority task data and low-priority task data, and obtains task execution priority data.

[0043] Preferably, step S4 comprises the following steps:

[0044] Step S41: performing attribute parameter analysis on the calculation node characteristic data to generate node attribute parameter data;

[0045] Step S42: Analyze the status information of the computing task management data to generate task status information data;

[0046] Step S43: constructing a visualization computing model according to the node attribute parameter data and the task status information data to obtain a visualization computing pre-model; and training the visualization computing pre-model using the node attribute parameter data and the task status information data to obtain a visualization computing training model;

[0047] Step S44: Perform model cross-validation evaluation on the visualization computing training model to obtain visualization computing model evaluation data; adjust model parameters of the visualization computing training model using the visualization computing model evaluation data to obtain a visualization computing model.

[0048] The present invention performs attribute parameter analysis on characteristic data of computing nodes to generate node attribute parameter data, which helps to fully understand the capabilities and resource conditions of each computing node and provides a basis for subsequent task scheduling and resource allocation; performs status information analysis on computing task management data to generate task status information data, which can clarify the execution status and resource requirements of each task; constructs a visual computing model based on the node attribute parameter data and the task status information data to obtain a visual computing pre-model, and displays the relationship and status between computing nodes and tasks in a visual manner, helping the generation team to intuitively understand the operation of streaming computing; and uses the node attribute parameter data and the task status information data to perform model training on the visual computing pre-model to obtain a visual computing training model, which accurately reflects the complex interactive relationship between computing nodes and computing tasks, and provides a more accurate basis for subsequent analysis and optimization; performs model cross-validation evaluation on the visual computing training model to obtain visual computing model evaluation data, which can verify the accuracy and robustness of the model and ensure that the model has good generalization ability; adjusts the model parameters of the visual computing training model through the visual computing model evaluation data to further improve the performance of the model and obtain a visual computing model.

[0049] Preferably, step S5 comprises the following steps:

[0050] Step S51: Performing graph theory analysis on the network layout of the computing node data based on the visual computing model to obtain a node network layout diagram; extracting the connection attributes and layout mode of the node network layout diagram to obtain node network layout data; performing network topology analysis on the node network layout data to obtain node topology data;

[0051] Step S52: dynamically adjusting the node network configuration data based on the node topology data to obtain computing node adjustment data;

[0052] Step S53: performing a processing speed requirement analysis on the computing task data based on the visualized computing model to obtain processing speed requirement data; performing a memory requirement analysis on the computing task data based on the visualized computing model to obtain memory requirement data; performing a communication bandwidth requirement analysis on the computing task data based on the visualized computing model to obtain communication bandwidth requirement data;

[0053] Step S54: Integrate the processing speed requirement data, the memory requirement data and the communication bandwidth requirement data to generate task requirement data;

[0054] Step S55: Based on the task requirement data, intelligent task management and control are performed on the task processing path data, the task execution priority data and the high resource requirement task data to generate computing task control data.

[0055] The present invention uses a visual computing model to perform graph theory analysis on the network layout of computing node data to obtain a node network layout diagram, understand the topological structure between computing nodes, and help identify existing performance bottlenecks or optimization potentials; extract the connection properties and layout methods of the node network layout diagram to obtain node network layout data, which can provide data support for subsequent optimization of network configuration and improvement of data transmission efficiency; perform network topology analysis on the node network layout data to obtain node topology data; dynamically adjust the node network configuration data based on the node topology data to ensure that each computing node can communicate and transmit data in the most efficient way when running tasks, and obtain computing node adjustment data, which can improve the response speed of computing nodes; perform processing speed requirement analysis on computing task data based on the visual computing model to obtain processing speed requirement data; The visual computing model performs memory requirement analysis on the computing task data to obtain memory requirement data; based on the visual computing model, the communication bandwidth requirement analysis on the computing task data is performed to obtain communication bandwidth requirement data, so that the visual computing model can accurately understand the specific requirements of each computing task for processing speed, memory and communication bandwidth; integrating the processing speed requirement, memory requirement and communication bandwidth requirement data into task requirement data can more accurately allocate appropriate resources to each task to avoid resource waste and inefficiency; based on the task requirement data, intelligent task management and control are performed on the task processing path data, task execution priority data and high resource requirement task data to ensure that the computing tasks can be executed according to the optimal path, give priority to important tasks, and perform special processing on high resource requirement tasks when necessary, thereby generating computing task control data.

[0056] Preferably, step S52 includes the following steps:

[0057] Step S521: Performing network performance evaluation on the node network configuration data according to the node topology data to obtain node network performance evaluation data;

[0058] Step S522: performing anomaly analysis on the node network performance evaluation data to obtain network performance anomaly data;

[0059] Step S523: extracting delay anomaly features of the network performance anomaly data to obtain delay anomaly data; extracting routing anomaly features of the network performance anomaly data to obtain routing anomaly data, wherein the routing anomaly data includes routing path anomaly and link selection anomaly;

[0060] Step S524: performing load balancing adjustment on the node network configuration data based on the delay abnormality data to obtain node load balancing adjustment data;

[0061] Step S525: Based on the routing anomaly characteristics, the node network configuration data is optimized and adjusted to obtain the node routing optimization data; if the computing node has a routing path anomaly, the routing path of the affected node in the computing node is replanned and the routing table is updated; if the computing node has a link selection anomaly, the dynamic routing protocol is used to intelligently switch the link;

[0062] Step S526: Integrate the node load balancing adjustment data and the node routing optimization data to obtain computing node adjustment data.

[0063] The present invention performs network performance evaluation on node network configuration data through node topology data, evaluates the overall performance level of the current network, thereby obtaining node network performance evaluation data, which provides a basis for subsequent decision-making; performs abnormal analysis on the node network performance evaluation data to obtain network performance abnormality data, which reflects the response speed of the network and the efficiency of data transmission; extracts delay abnormality features of the network performance abnormality data, determines the data of network performance abnormality caused by the delay abnormality, and obtains delay abnormality data; extracts routing abnormality features of the network performance abnormality data, determines the data of network performance abnormality caused by the routing abnormality, and obtains routing abnormality data, wherein the routing abnormality data includes routing path abnormality and link selection abnormality; based on the delay abnormality data Perform load balancing adjustments on the node network configuration data to ensure that the loads of each computing node in the network are balanced, and obtain the node load balancing adjustment data; perform routing optimization adjustments on the node network configuration data based on routing anomaly characteristics to avoid network instability and data packet loss caused by routing path anomalies or link selection problems, and obtain node routing optimization data; if a routing path anomaly occurs in a computing node, re-plan the routing path of the affected node in the computing node and update the routing table; if a link selection anomaly occurs in a computing node, a dynamic routing protocol is used to intelligently switch links; integrate the node load balancing adjustment data and the node routing optimization data to obtain the computing node adjustment data, which reflects the adjustments and optimization measures made after the computing node encounters an abnormal processing situation.

[0064] Preferably, step S55 comprises the following steps:

[0065] Step S551: Perform intelligent path planning on the task processing path data based on the task demand data. If the processing speed demand data is higher than the preset task speed demand threshold, the processing path of the computing task is changed to generate task change path data; if the processing speed demand data is less than or equal to the preset task speed demand threshold, the computing task is not processed to generate task default path data; the task change path data and the task default path data are integrated to obtain task path planning data;

[0066] Step S552: Intelligently schedule the task execution priority data based on the task demand data. If the computing task is a high-priority task, optimal link allocation is performed on the computing task to generate task optimal link allocation data; if the computing task is a low-priority task, link redistribution is performed on the computing task to generate task link redistribution data; integrate the task optimal link allocation data and the task link redistribution data to obtain priority task intelligent scheduling data;

[0067] Step S553: ​​Intelligent resource management is performed on the high resource demand task data based on the task demand data. If the computing task is a high resource demand task and a high priority task, the memory capacity and communication bandwidth rate are increased to generate task memory bandwidth improvement data; if the computing task is a high resource demand task and a low priority task, only the communication bandwidth rate is increased to generate task bandwidth improvement data; the task memory bandwidth improvement data and the task bandwidth improvement data are integrated to obtain task resource intelligent management data;

[0068] Step S554: Integrate the task path planning data, priority task intelligent scheduling data and task resource intelligent management data to obtain computing task control data.

[0069] The present invention uses task demand data to perform intelligent path planning on task processing path data to ensure that computing tasks can be completed on time. If the processing speed demand data is higher than a preset task speed demand threshold, the processing path of the computing task is changed to generate task change path data; if the processing speed demand data is less than or equal to the preset task speed demand threshold, the computing task is not processed to generate task default path data; the task change path data and the task default path data are integrated to obtain task path planning data, which can effectively respond to different task requirements and ensure the efficiency and accuracy of tasks, thereby improving the overall computing resource utilization; based on the task demand data, the task execution priority data is intelligently scheduled to ensure that the computing task can be completed as soon as possible and given priority. If the computing task is a high-priority task, the optimal link allocation is performed on the computing task to generate task optimal link allocation data; if the computing task is a low-priority task, the link of the computing task is redistributed to generate task link redistribution Allocate data; integrate task optimal link allocation data and task link redistribution data to obtain priority task intelligent scheduling data in order to better balance resource usage and task completion time; perform intelligent resource management on high resource demand task data based on task demand data to ensure efficiency and stability during task execution. If the computing task is a high resource demand task and a high priority task, the memory capacity and communication bandwidth rate are increased to generate task memory bandwidth improvement data; if the computing task is a high resource demand task and a low priority task, only the communication bandwidth rate is increased to generate task bandwidth improvement data; integrate task memory bandwidth improvement data and task bandwidth improvement data to obtain task resource intelligent management data, and try to meet the needs of computing tasks while ensuring resource utilization; integrate task path planning data, priority task intelligent scheduling data and task resource intelligent management data to obtain computing task control data, ensure that processing data and decisions can be coordinated, and provide a comprehensive control strategy for computing tasks.

[0070] Preferably, step S6 comprises the following steps:

[0071] Step S61: Perform real-time visual control and monitoring on the streaming computing through computing node adjustment data and computing task control data to obtain real-time visual control data and real-time visual monitoring data;

[0072] Step S62: Integrate the real-time visual control data and the real-time visual monitoring data to obtain a real-time visual feedback report;

[0073] Step S63: Use blockchain technology to upload the real-time visualization feedback report to the blockchain node, and distribute the real-time visualization feedback report through the blockchain node to execute the visualization computing job of streaming computing.

[0074] The present invention performs real-time visual regulation and monitoring of streaming computing through computing node adjustment data and computing task control data. Streaming computing can quickly respond to data changes and task requirements, improve the real-time and responsiveness of streaming computing, thereby obtaining real-time visual regulation data and real-time visual monitoring data; integrating real-time visual regulation data and real-time visual monitoring data to obtain real-time visual feedback reports, provides instant data display and analysis for the production team, helps understand current data processing and status, and thus makes better decisions and optimizes strategies; utilizes blockchain technology to upload real-time visual feedback reports to blockchain nodes, and distributes and stores real-time visual feedback reports through blockchain nodes, thereby ensuring the security and integrity of real-time feedback reports and preventing data from being tampered with or lost, so as to execute visual computing jobs of streaming computing.

[0075] The present invention collects streaming computing data sets and pre-processes the streaming computing data sets to ensure the consistency and accuracy of the streaming computing data sets, forms a standardized streaming computing data set, and provides a reliable basis for subsequent analysis and processing, thereby improving the accuracy and reliability of the streaming computing process. By extracting the computing nodes in the standard streaming computing data set, the computing node data is obtained, and the correlation analysis is performed to reveal the interaction and dependency between different nodes, optimize the data flow mode, and improve the data processing and transmission efficiency. At the same time, the network configuration and type analysis of the computing nodes are performed to understand the connection mode, processing capacity and storage capacity of the nodes, and further determine the functions of the nodes and their overall role. Integrate node correlation, network configuration and functional data to form comprehensive computing node characteristic data, and fully grasp the characteristics of the computing nodes. By extracting computing task data from the standard streaming computing data set, processing path analysis and execution priority analysis are performed to determine the processing path and execution order of each task, analyze high resource demand task data, and reasonably allocate system resources to avoid bottlenecks and excessive consumption. Integrate processing paths, execution priorities and high resource demand data to generate computing task management data for intelligent task scheduling. By building a visual computing model through computing node characteristic data and task management data, the relationship between nodes and task execution status can be intuitively displayed, potential problems can be quickly discovered and decisions can be made, and management efficiency can be improved. The visual model is used to perform network topology analysis to understand the physical layout, data transmission path and load status between nodes, and optimize network performance and resource utilization through dynamic network configuration adjustment. Blockchain technology is further used to upload real-time visual feedback reports to distributed storage to ensure its security and non-tamperability, and enhance the transparency and trust of streaming computing. Therefore, the present invention improves the dynamic adjustment and allocation capabilities of streaming computing through Internet of Things technology, intelligent control technology, visualization technology and blockchain technology, and strengthens the real-time monitoring and feedback capabilities of the visual computing model of streaming computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A schematic diagram of the steps of building a visual computing model based on streaming computing;

[0077] Figure 2 for Figure 1 Detailed implementation steps of step S5;

[0078] Figure 3 for Figure 2 Detailed implementation steps of step S52;

[0079] Figure 4 for Figure 2 Detailed implementation steps of step S55 are shown in the flowchart;

[0080] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0081] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0082] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0083] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0084] To achieve this, please refer to Figures 1 to 4 , a method for constructing a visual computing model based on streaming computing, the method comprising the following steps:

[0085] Step S1: Obtain a streaming computing data set; perform data preprocessing on the streaming computing data set to obtain a standard streaming computing data set;

[0086] Step S2: extract computing nodes of the standard streaming computing data set to obtain computing node data; perform correlation analysis on the computing node data to obtain node correlation data; perform network configuration analysis on the computing node data to obtain node network configuration data; perform node type analysis on the computing node data to obtain node type data; perform node function analysis based on the node type data to obtain node function data; integrate the node correlation data, node network configuration data and node function data to generate computing node characteristic data;

[0087] Step S3: extract computing tasks of the standard streaming computing data set to obtain computing task data; perform processing path analysis on the computing task data to obtain task processing path data; perform execution priority analysis on the computing task data to obtain task execution priority data; perform resource utilization analysis on the computing task data to obtain high resource demand task data; integrate task processing path data, task execution priority data and high resource demand task data to generate computing task management data;

[0088] Step S4: constructing a visual computing model according to the computing node characteristic data and the computing task management data to obtain a visual computing model;

[0089] Step S5: Perform network topology analysis on computing node data based on the visual computing model to obtain node topology data; perform dynamic network configuration adjustment on node network configuration data based on the node topology data to obtain computing node adjustment data; perform demand analysis on computing task data based on the visual computing model to obtain task demand data; perform intelligent task management control on task processing path data, task execution priority data and high resource demand task data based on the task demand data to generate computing task control data;

[0090] Step S6: Perform real-time visual feedback on the streaming computing through computing node adjustment data and computing task control data, and generate a real-time visual feedback report; upload the real-time visual feedback report to the blockchain network distributed storage to execute the visual computing job of the streaming computing.

[0091] The present invention obtains a streaming computing data set and performs data preprocessing on the streaming computing data set to ensure the consistency and accuracy of the data, thereby obtaining a standard streaming computing data set, providing a reliable basis for subsequent analysis and processing, and ensuring that the data used in the streaming computing process is verified and cleaned, thereby improving the accuracy and reliability of the calculation. By extracting the computing nodes of the standard streaming computing data set, we obtain computing node data, which provides basic data for subsequent analysis and ensures that the characteristics of each computing node are understood from a comprehensive perspective; by performing correlation analysis on the computing node data, we obtain node correlation data, which can help us understand the interactions and dependencies between different nodes. Correlation analysis helps to determine the data flow pattern between nodes, thereby optimizing the efficiency of data processing and transmission; by performing network configuration analysis on the computing node data, we obtain node network configuration data. The network configuration data provides important information about how nodes are connected to the network and how they interact with each other; by performing node type analysis on the computing node data, we obtain node type data, which can help us understand that each type of node has different processing and storage capabilities; by performing node function analysis based on the node type data, we obtain node function data. After determining the function of the node, we can further understand the overall role of the computing node; by integrating node correlation data, node network configuration data and node function data to generate computing node characteristic data, we can fully understand the computing node characteristic data. By extracting computing tasks from standard streaming computing data sets, computing task data is obtained; processing path analysis is performed on computing task data to obtain task processing path data, which can understand the processing process and path of each computing task; execution priority analysis is performed on computing task data to obtain task execution priority data, which can determine the execution order and priority of each task; resource utilization analysis is performed on computing task data to obtain high resource demand task data. By analyzing high resource demand task data, system resources can be reasonably allocated and managed to avoid resource bottlenecks and excessive consumption; task processing path data, task execution priority data and high resource demand task data are integrated to generate computing task management data, which will be used to formulate task scheduling later. Visual computing models are constructed by computing node characteristic data and computing task management data to obtain visual computing models, which can intuitively display the relationship between computing nodes and task execution status, and can more quickly discover potential problems and make decisions through visual presentation, thereby improving overall management efficiency.Through the visual computing model, the network topology analysis of the computing node data is performed to obtain the node topology data. By analyzing the network topology, the physical layout, data transmission path and load conditions between the computing nodes can be understood; based on the node topology data, the node network configuration data is dynamically adjusted to optimize network performance and resource utilization, and the computing node adjustment data is obtained. By adjusting the network configuration in real time, the network bandwidth is allocated, the routing path is adjusted or the connection method between nodes is changed according to actual needs; based on the visual computing model, the computing task data is analyzed for demand and the task demand data is obtained; based on the task demand data, the task processing path data, the task execution priority data and the high resource demand task data are intelligently managed and controlled to realize the intelligent regulation of the task processing path, execution priority and resource demand. Through intelligent task management, the computing task control data is generated, which can automatically respond to the demand changes of different tasks and improve the task execution efficiency and overall responsiveness. By adjusting the data of the computing nodes and the computing task control data, real-time visual feedback is provided to the streaming computing, and a real-time visual feedback report is generated, so that the execution and performance of the streaming computing can be monitored in real time; the real-time visual feedback report is uploaded to the distributed storage of the blockchain network to perform the visual computing job of the streaming computing, which ensures the security and non-tamperability of the report, and further enhances the transparency and trust of the streaming computing. Therefore, the present invention improves the dynamic adjustment and allocation capability of the streaming computing through the Internet of Things technology, intelligent control technology, visualization technology and blockchain technology, and strengthens the real-time monitoring and feedback capability of the visual computing model of the streaming computing.

[0092] In the embodiment of the present invention, reference Figure 1 The above is a flow chart of the steps of the method for constructing a visual computing model of streaming computing of the present invention. In this example, the method for constructing a visual computing model of streaming computing includes the following steps:

[0093] Step S1: Obtain a streaming computing data set; perform data preprocessing on the streaming computing data set to obtain a standard streaming computing data set;

[0094] In an embodiment of the present invention, streaming computing data is monitored in real time by sensors to obtain a streaming computing data set, wherein the streaming computing data set includes computing node data and computing task data; data preprocessing is performed on the streaming computing data set, and the data preprocessing includes data cleaning to remove invalid or erroneous data, data denoising to reduce measurement errors, and data standardization to unify the formats and units of data from different sources, thereby obtaining a standard streaming computing data set.

[0095] Step S2: extract computing nodes of the standard streaming computing data set to obtain computing node data; perform correlation analysis on the computing node data to obtain node correlation data; perform network configuration analysis on the computing node data to obtain node network configuration data; perform node type analysis on the computing node data to obtain node type data; perform node function analysis based on the node type data to obtain node function data; integrate the node correlation data, node network configuration data and node function data to generate computing node characteristic data;

[0096] In an embodiment of the present invention, computing nodes of a standard streaming computing data set are extracted to obtain computing node data; a correlation analysis is performed on the computing node data based on the connection between nodes, communication frequency and data interaction factors to obtain node correlation data; a network configuration analysis is performed on the computing node data to obtain node network configuration data, wherein the node network configuration data includes analysis of the IP address, subnet mask, gateway and port status; a node type analysis is performed on the computing node data, and the type classification of each computing node is determined according to the hardware configuration, operating system information and application identifier of the node, thereby obtaining node type data; the function and purpose of each node type, such as data processing node, storage node and scheduling node, are analyzed, and a node function analysis is performed according to the node type data to determine the specific functional characteristics of each node, thereby obtaining node function data; the node correlation data, the node network configuration data and the node function data are integrated to generate computing node characteristic data for subsequent data visualization, analysis and further streaming computing processing.

[0097] Step S3: extract computing tasks of the standard streaming computing data set to obtain computing task data; perform processing path analysis on the computing task data to obtain task processing path data; perform execution priority analysis on the computing task data to obtain task execution priority data; perform resource utilization analysis on the computing task data to obtain high resource demand task data; integrate task processing path data, task execution priority data and high resource demand task data to generate computing task management data;

[0098] In an embodiment of the present invention, computing tasks of a standard streaming computing data set are extracted based on task identifiers and timestamps to obtain computing task data; processing path analysis is performed on the computing task data to analyze the processing path of the computing task, that is, the data flow path from input to output of the task, to obtain task processing path data; execution priority analysis is performed on the computing task data based on task type, data processing requirements and business priority to obtain task execution priority data; resource utilization analysis is performed on the computing task data to analyze the requirements of the computing task for CPU, memory and storage resources to obtain high resource demand task data; task processing path data, task execution priority data and high resource demand task data are integrated to generate computing task management data to support subsequent task scheduling, resource allocation and performance optimization requirements.

[0099] Step S4: constructing a visual computing model according to the computing node characteristic data and the computing task management data to obtain a visual computing model;

[0100] In an embodiment of the present invention, a visual computing model is constructed by computing node characteristic data and computing task management data to obtain a visual computing pre-model; and the visual computing pre-model is trained using the computing node characteristic data and computing task management data to obtain a visual computing training model; the visual computing training model is cross-validated and evaluated to obtain visual computing model evaluation data; and model parameters of the visual computing training model are adjusted using the visual computing model evaluation data to obtain a visual computing model.

[0101] Step S5: Perform network topology analysis on computing node data based on the visual computing model to obtain node topology data; perform dynamic network configuration adjustment on node network configuration data based on the node topology data to obtain computing node adjustment data; perform demand analysis on computing task data based on the visual computing model to obtain task demand data; perform intelligent task management control on task processing path data, task execution priority data and high resource demand task data based on the task demand data to generate computing task control data;

[0102] In an embodiment of the present invention, a network topology analysis is performed on computing node data based on a visual computing model, and the node positions and connection modes in the network topology diagram are analyzed to obtain node topology data; a dynamic network configuration adjustment is performed on node network configuration data based on the node topology data to obtain computing node adjustment data, wherein the dynamic network configuration adjustment includes IP address allocation, subnet division, and routing settings to ensure effective communication and data transmission between computing nodes; a demand analysis is performed on computing task data based on a visual computing model to analyze the computational complexity, running time estimation, and resource demand of the task to obtain task demand data; intelligent task management and control is performed on task processing path data, task execution priority data, and high resource demand task data based on the task demand data to ensure effective management and optimization of computing tasks, and intelligent task management and control is performed using a visual computing model to achieve effective scheduling and management of computing nodes and computing tasks, and generate computing task control data;

[0103] Step S6: Perform real-time visual feedback on the streaming computing through computing node adjustment data and computing task control data, and generate a real-time visual feedback report; upload the real-time visual feedback report to the blockchain network distributed storage to execute the visual computing job of the streaming computing.

[0104] In the embodiment of the present invention, real-time visual monitoring of streaming computing is performed through computing node adjustment data and computing task control data. The real-time visual monitoring includes the network configuration adjustment of the computing node and the execution status of the computing task, and real-time visual monitoring data is obtained; the real-time visual monitoring data is converted into a visual feedback report format to generate a real-time visual feedback report. The visual feedback report format includes charts, dynamic graphs and real-time data streams to reflect the current running status and efficiency of streaming computing. The encrypted real-time visual feedback report is uploaded to the blockchain network using an adapted data upload interface, and multiple copies of the data are saved on each node of the network to ensure data backup and fault tolerance and prevent data loss or tampering.

[0105] Preferably, step S2 comprises the following steps:

[0106] Step S21: performing computing node feature recognition on the standard streaming computing data set to generate computing node feature information;

[0107] Step S22: extracting node data of computing node characteristic information to obtain computing node data;

[0108] Step S23: performing a correlation analysis of the interaction patterns of adjacent computing nodes on the computing node data to obtain node correlation data;

[0109] Step S24: Perform network configuration analysis on the computing node data to obtain node network configuration data;

[0110] Step S25: extracting attribute features of the computation node data to obtain node attribute feature data; classifying the computation node data into node types according to the node attribute feature data to obtain node type data; evaluating the node role according to the node type data to obtain node role evaluation data; and performing functional pathway analysis on the node role evaluation data to obtain node function data;

[0111] Step S26: Integrate the node association data, node network configuration data and node function data to generate computing node characteristic data.

[0112] In the embodiment of the present invention, frequency analysis and pattern recognition feature engineering technology are applied to identify computing node features of a standard streaming computing data set, wherein the computing node features include the CPU usage, memory occupancy, running time and processing power of the node, thereby generating computing node feature information; extracting node data of the computing node feature information to obtain computing node data; evaluating the correlation of interaction patterns of adjacent computing nodes on the computing node data through centrality and closeness analysis methods to obtain node correlation data, clarifying that computing nodes have close interaction relationships in streaming computing, as well as key paths and communication patterns between adjacent computing nodes; analyzing the network topology, communication protocol, bandwidth and delay parameters of the computing node data to obtain to the node network configuration data; extract the hardware configuration, operating system type and storage capacity of the computing node data to obtain the node attribute feature data; classify the computing node data into node types according to the node attribute feature data to obtain the node type data, wherein the node type data includes high-performance nodes, storage nodes and communication nodes; evaluate the node role according to the node type data, evaluate the importance and role of the node type data in the overall computing task, obtain the node role evaluation data, perform functional pathway analysis on the node role evaluation data, obtain the node function data; integrate the node association data, the node network configuration data and the node function data to generate the computing node characteristic data, and ensure the accuracy and completeness of the computing node characteristic data.

[0113] Preferably, step S24 includes the following steps:

[0114] Step S241: extracting and calculating the network characteristics of the node data to obtain the node network characteristic data;

[0115] Step S242: performing network connection analysis on the node network characteristic data to obtain node network connection data;

[0116] Step S243: performing connection status analysis according to the node network connection data to obtain node connection status data;

[0117] Step S244: Analyze the configuration of the node connection status data to obtain node network configuration data.

[0118] In an embodiment of the present invention, information related to network characteristics is extracted from computing node data, the network characteristic information including the node's IP address, MAC address, bandwidth utilization and network delay, thereby obtaining node network characteristic data; network connection analysis is performed on the node network characteristic data to determine the physical and logical connection modes between computing nodes, such as direct connection and connection through a switch router; the number of connections, connection frequency and connection type of the nodes, such as wired connection and wireless connection, are counted to obtain node network connection data; connection status analysis is performed based on the node network connection data to determine the stability and reliability of the connection, such as whether there are disconnection, delay and packet loss problems, thereby obtaining node connection status data; configuration status analysis is performed on the node connection status data to analyze whether the configuration meets expectations, such as whether the network parameters are set correctly and whether the network equipment is working normally, and key network configuration information is extracted, such as device model, configuration file and device status, thereby obtaining node network configuration data.

[0119] Preferably, step S3 comprises the following steps:

[0120] Step S31: performing computing task feature recognition on the standard streaming computing data set to generate computing task feature information;

[0121] Step S32: extracting task data of computing task characteristic information to obtain computing task data;

[0122] Step S33: performing data mapping on the computing task data processing information to generate a task processing flow chart; identifying the data flow direction and processing steps on the task processing flow chart to obtain task processing path data;

[0123] Step S34: performing execution priority analysis on the computing task data to obtain task execution priority data;

[0124] Step S35: Perform statistical analysis on the required resource amount of the computing task data to obtain task resource amount statistical data; arrange the task resource amount statistical data in descending order to obtain task resource amount descending data; perform heat map color mapping on the task resource amount descending data to generate a task resource amount heat map; extract the dark coordinate data of the task resource amount heat map to obtain high resource demand task data;

[0125] Step S36: Integrate the task processing path data, task execution priority data and high resource demand task data to generate computing task management data.

[0126] In an embodiment of the present invention, frequency analysis and pattern recognition feature engineering technology are applied to identify computing task features of a standard streaming computing data set to generate computing task feature information; wherein the computing task feature information includes the type of task, data volume, and processing time; the task identifier and feature value of the computing task feature information are extracted to obtain computing task data; data mapping is performed on the computing task data processing information to generate a task processing flowchart, which reflects the dependencies and execution order between computing tasks; the data flow direction and processing steps of the task processing flowchart are identified to obtain task processing path data, which describes the execution path of the computing task in the entire process and the related data transfer process; based on the importance and urgency of the task, the computing task data is analyzed for execution priority to obtain task execution priority data. Perform statistical analysis on the required resources of computing task data to determine the resources required for computing tasks, such as CPU, memory, and storage space, so as to obtain task resource statistics; sort the task resource statistics in descending order to obtain task resource descending data; use the color mapping technology of the heat map to visualize the task resource descending data and generate a task resource heat map; extract the dark coordinate data of the task resource heat map, which represents computing tasks with high resource requirements, so as to obtain high resource demand task data. Integrate task processing path data, task execution priority data, and high resource demand task data to generate computing task management data for subsequent formulation and optimization of computing task execution strategies and resource allocation plans.

[0127] Preferably, step S34 includes the following steps:

[0128] Step S341: extracting the execution characteristics of the computing task data to obtain task execution characteristic data;

[0129] Step S342: performing load analysis on the task execution characteristic data to obtain task execution load data;

[0130] Step S343: extracting the peak and valley values ​​of the task execution load data to obtain the task execution load peak and valley data;

[0131] Step S344: Calculate the average value of the peak and valley data of the task execution load to obtain the average value of the task execution load;

[0132] Step S345: Compare the task execution load data with the task execution load average value. When the task execution load average value is greater than the task execution load average value, mark the computing task as task execution high load data; when the task execution load value is less than or equal to the task execution load average value, mark the computing task as task execution low load data.

[0133] Step S346: performing high-priority classification on the task execution high-load data to obtain high-priority task data; performing low-priority classification on the task execution low-load data to obtain low-priority task data;

[0134] Step S347: Integrate high-priority task data and low-priority task data to obtain task execution priority data.

[0135] In the embodiment of the present invention, the execution characteristics of the computing task data are extracted, and the execution characteristics include the execution time, resource consumption and data transmission rate of the task, so as to obtain the task execution characteristic data; the task execution characteristic data is subjected to load analysis by using a statistical analysis method, and the load analysis includes the average value, variance and distribution characteristics, so as to obtain the task execution load data; the peak and valley values ​​of the task execution load data are taken to obtain the task execution load peak and valley data, and the task execution load peak and valley data reflects the highest and lowest load conditions of the task in different time periods; the average value is calculated based on the task execution load peak and valley data, and is realized by simple averaging and weighted averaging to obtain the average value of the task execution load; the task execution load is calculated based on the task execution load peak and valley data. The data is compared with the task execution load average value. When the task execution load average value is greater than the task execution load average value, the computing task is marked as task execution high load data; when the task execution load value is less than or equal to the task execution load average value, the computing task is marked as task execution low load data; according to the task load mark, the high load data is divided into high priority to ensure that these tasks can be executed first, and the low load data is divided into low priority to ensure that the execution of high load tasks is not affected by low load tasks; the high priority and low priority task data that have been prioritized are integrated together to form the final task execution priority data set for subsequent task scheduling and management operations.

[0136] Preferably, step S4 comprises the following steps:

[0137] Step S41: performing attribute parameter analysis on the calculation node characteristic data to generate node attribute parameter data;

[0138] Step S42: Analyze the status information of the computing task management data to generate task status information data;

[0139] Step S43: constructing a visualization computing model according to the node attribute parameter data and the task status information data to obtain a visualization computing pre-model; and training the visualization computing pre-model using the node attribute parameter data and the task status information data to obtain a visualization computing training model;

[0140] Step S44: Perform model cross-validation evaluation on the visualization computing training model to obtain visualization computing model evaluation data; adjust model parameters of the visualization computing training model using the visualization computing model evaluation data to obtain a visualization computing model.

[0141] In an embodiment of the present invention, attribute parameter analysis is performed on computing node characteristic data, and the attribute parameter analysis includes statistical processor frequency analysis, memory usage analysis, and network bandwidth analysis, so as to generate node attribute parameter data; status information analysis is performed on computing task management data, and the status information analysis includes task completion rate and task execution time distribution, so as to generate task status information data; a visual computing model is constructed based on the node attribute parameter data and the task status information data. First, a visual computing model is constructed based on the node attribute parameter data and the task status information data to obtain a visual computing pre-model, and then the pre-model is trained using the node attribute parameter data and the task status information data. During the training process, the parameters of the model are continuously optimized through a back propagation algorithm to minimize the error between the predicted output and the actual output. The model contains multiple hidden layers, each with several neurons. Each neuron processes the input signal through an activation function, such as ReLU (Rectified Linear Unit) or Sigmoid function. After training, the model is cross-validated and evaluated. The model is trained using the training set, and the model performance is evaluated on the validation set. The evaluation indicators include accuracy, recall, F1 score, and mean square error (MSE). According to the evaluation data, the parameters of the training model are adjusted, and the adjustment operation includes adjusting the learning rate; increasing or decreasing the number of neurons in the hidden layer, changing the activation function and the optimization algorithm, so as to improve the generalization ability of the model. The adjusted model is trained and evaluated again until a satisfactory performance standard is achieved, and finally a visual computing model is obtained.

[0142] As an example of the present invention, refer to Figure 2 As shown, in this example, step S5 includes:

[0143] Step S51: Performing graph theory analysis on the network layout of the computing node data based on the visual computing model to obtain a node network layout diagram; extracting the connection attributes and layout mode of the node network layout diagram to obtain node network layout data; performing network topology analysis on the node network layout data to obtain node topology data;

[0144] Step S52: dynamically adjusting the node network configuration data based on the node topology data to obtain computing node adjustment data;

[0145] Step S53: performing a processing speed requirement analysis on the computing task data based on the visualized computing model to obtain processing speed requirement data; performing a memory requirement analysis on the computing task data based on the visualized computing model to obtain memory requirement data; performing a communication bandwidth requirement analysis on the computing task data based on the visualized computing model to obtain communication bandwidth requirement data;

[0146] Step S54: Integrate the processing speed requirement data, the memory requirement data and the communication bandwidth requirement data to generate task requirement data;

[0147] Step S55: Based on the task requirement data, intelligent task management and control are performed on the task processing path data, the task execution priority data and the high resource requirement task data to generate computing task control data.

[0148] In the embodiment of the present invention, based on the visual computing model, the computing node data is analyzed using a graph theory method to determine the connection relationship between the computing nodes, the network topology structure and the layout method, and obtain the node network layout diagram; the connection attributes and layout method of the node network layout diagram are extracted to obtain the node network layout data; the node network layout data is subjected to network topology analysis, and the network topology analysis includes centrality, clustering coefficient and network diameter, so as to obtain the node topology data; the node network configuration data is dynamically adjusted based on the node topology data to obtain the computing node adjustment data, wherein the dynamic network configuration adjustment includes routing selection, load balancing and network bandwidth adjustment; the computing tasks are adjusted based on the visual computing model. Perform processing speed requirement analysis on the data to obtain processing speed requirement data; perform memory requirement analysis on the computing task data based on the visual computing model to obtain memory requirement data; perform communication bandwidth requirement analysis on the computing task data based on the visual computing model to obtain communication bandwidth requirement data; integrate processing speed requirement data, memory requirement data and communication bandwidth requirement data to generate task requirement data, which includes specific values ​​of various computing task requirements and computing task characteristics; perform intelligent task management and control on task processing path data, task execution priority data and high resource requirement task data based on the task requirement data to optimize task allocation and resource utilization efficiency, thereby generating computing task control data.

[0149] As an example of the present invention, refer to Figure 3 As shown, in this example, step S52 includes:

[0150] Step S521: Performing network performance evaluation on the node network configuration data according to the node topology data to obtain node network performance evaluation data;

[0151] Step S522: performing anomaly analysis on the node network performance evaluation data to obtain network performance anomaly data;

[0152] Step S523: extracting delay anomaly features of the network performance anomaly data to obtain delay anomaly data; extracting routing anomaly features of the network performance anomaly data to obtain routing anomaly data, wherein the routing anomaly data includes routing path anomaly and link selection anomaly;

[0153] Step S524: performing load balancing adjustment on the node network configuration data based on the delay abnormality data to obtain node load balancing adjustment data;

[0154] Step S525: Based on the routing anomaly characteristics, the node network configuration data is optimized and adjusted to obtain the node routing optimization data; if the computing node has a routing path anomaly, the routing path of the affected node in the computing node is replanned and the routing table is updated; if the computing node has a link selection anomaly, the dynamic routing protocol is used to intelligently switch the link;

[0155] Step S526: Integrate the node load balancing adjustment data and the node routing optimization data to obtain computing node adjustment data.

[0156] In an embodiment of the present invention, network performance evaluation is performed on node network configuration data according to node topology data, and the network performance evaluation includes bandwidth utilization, network load and performance bottleneck, so as to obtain node network performance evaluation data; abnormal analysis is performed on the node network performance evaluation data through a data analysis technology algorithm to obtain network performance abnormality data; delay abnormality features of the network performance abnormality data are extracted to obtain delay abnormality data; routing abnormality features of the network performance abnormality data are extracted to obtain routing abnormality data, wherein the routing abnormality data includes routing path abnormality and link selection abnormality; load balancing adjustment is performed on the node network configuration data based on the delay abnormality data to obtain node load balancing adjustment data, The load balancing adjustment includes adjusting the data flow distribution between computing nodes to optimize the overall network performance and reduce latency; performing routing optimization and adjustment on the node network configuration data based on routing anomaly characteristics to obtain node routing optimization data; if a routing path anomaly occurs in a computing node, the routing path of the affected node in the computing node is replanned and the routing table is updated; if a link selection anomaly occurs in a computing node, a dynamic routing protocol is used to intelligently switch links to ensure fast data packet transmission of the computing node and stable network performance; integrating node load balancing adjustment data and node routing optimization data to obtain computing node adjustment data, which is used to optimize the overall computing node network performance and improve data transmission efficiency.

[0157] As an example of the present invention, refer to Figure 4 As shown, in this example, step S55 includes:

[0158] Step S551: Perform intelligent path planning on the task processing path data based on the task demand data. If the processing speed demand data is higher than the preset task speed demand threshold, the processing path of the computing task is changed to generate task change path data; if the processing speed demand data is less than or equal to the preset task speed demand threshold, the computing task is not processed to generate task default path data; the task change path data and the task default path data are integrated to obtain task path planning data;

[0159] Step S552: Intelligently schedule the task execution priority data based on the task demand data. If the computing task is a high-priority task, optimal link allocation is performed on the computing task to generate task optimal link allocation data; if the computing task is a low-priority task, link redistribution is performed on the computing task to generate task link redistribution data; integrate the task optimal link allocation data and the task link redistribution data to obtain priority task intelligent scheduling data;

[0160] Step S553: ​​Intelligent resource management is performed on the high resource demand task data based on the task demand data. If the computing task is a high resource demand task and a high priority task, the memory capacity and communication bandwidth rate are increased to generate task memory bandwidth improvement data; if the computing task is a high resource demand task and a low priority task, only the communication bandwidth rate is increased to generate task bandwidth improvement data; the task memory bandwidth improvement data and the task bandwidth improvement data are integrated to obtain task resource intelligent management data;

[0161] Step S554: Integrate the task path planning data, priority task intelligent scheduling data and task resource intelligent management data to obtain computing task control data.

[0162] In an embodiment of the present invention, intelligent path planning is performed on task processing path data based on task demand data, and task processing path data is checked based on the acquired task demand data to see if it is higher than a preset task speed demand threshold. If so, a change in the task processing path is triggered to adapt to the high speed demand, thereby generating task change path data; if the processing speed demand data is less than or equal to the preset threshold, no path change is performed, and task default path data is generated; finally, the task change path data and the task default path data are integrated to form task path planning data. Intelligent scheduling is performed on task execution priority data based on task demand data, and for high-priority tasks, optimal link allocation is performed to ensure that the computing task is completed under optimal conditions, thereby generating task optimal link allocation data; for low-priority tasks, link redistribution is performed to optimize resource utilization, and task link redistribution data is generated; finally, the task optimal link allocation data and the task link redistribution data are integrated to form priority task intelligent scheduling data. Based on the task demand data, intelligent resource management is performed on the task data with high resource demand. Different resource improvement strategies are adopted according to the priority and high resource demand of the computing task. For tasks with high resource demand and high priority, the memory capacity and communication bandwidth rate are improved to meet the needs of task execution, thereby generating task memory bandwidth improvement data. For tasks with high resource demand but low priority, only the communication bandwidth rate is improved to balance resource input and task demand, thereby generating task bandwidth improvement data. Finally, the task memory bandwidth improvement data and task bandwidth improvement data are integrated to form task resource intelligent management data. The task path planning data, priority task intelligent scheduling data and task resource intelligent management data are integrated to obtain the computing task control data to ensure that the task can be effectively managed and scheduled according to the expected path, priority and resource management strategy during execution.

[0163] Preferably, step S6 comprises the following steps:

[0164] Step S61: Perform real-time visual control and monitoring on the streaming computing through computing node adjustment data and computing task control data to obtain real-time visual control data and real-time visual monitoring data;

[0165] Step S62: Integrate the real-time visual control data and the real-time visual monitoring data to obtain a real-time visual feedback report;

[0166] Step S63: Use blockchain technology to upload the real-time visualization feedback report to the blockchain node, and distribute the real-time visualization feedback report through the blockchain node to execute the visualization computing job of streaming computing.

[0167] In an embodiment of the present invention, real-time visual monitoring of streaming computing is performed through computing node adjustment data and computing task control data, and the real-time visual monitoring includes the network configuration adjustment status of the computing node and the execution status of the computing task, so as to obtain real-time visual monitoring data; the real-time visual monitoring data is converted into a visual feedback report format to generate a real-time visual feedback report; the real-time visual feedback report is uploaded to the blockchain node using blockchain technology, and the data transmission security is ensured by using smart contracts and encryption algorithms, and the real-time visual feedback report is distributedly stored through the blockchain node to ensure that the data is backed up on multiple nodes to execute the visual computing job of streaming computing.

[0168] The present invention collects streaming computing data sets and pre-processes the streaming computing data sets to ensure the consistency and accuracy of the streaming computing data sets, forms a standardized streaming computing data set, and provides a reliable basis for subsequent analysis and processing, thereby improving the accuracy and reliability of the streaming computing process. By extracting the computing nodes in the standard streaming computing data set, the computing node data is obtained, and the correlation analysis is performed to reveal the interaction and dependency between different nodes, optimize the data flow mode, and improve the data processing and transmission efficiency. At the same time, the network configuration and type analysis of the computing nodes are performed to understand the connection mode, processing capacity and storage capacity of the nodes, and further determine the functions of the nodes and their overall role. Integrate node correlation, network configuration and functional data to form comprehensive computing node characteristic data, and fully grasp the characteristics of the computing nodes. By extracting computing task data from the standard streaming computing data set, processing path analysis and execution priority analysis are performed to determine the processing path and execution order of each task, analyze high resource demand task data, and reasonably allocate system resources to avoid bottlenecks and excessive consumption. Integrate processing paths, execution priorities and high resource demand data to generate computing task management data for intelligent task scheduling. By building a visual computing model through computing node characteristic data and task management data, the relationship between nodes and task execution status can be intuitively displayed, potential problems can be quickly discovered and decisions can be made, and management efficiency can be improved. The visual model is used to perform network topology analysis to understand the physical layout, data transmission path and load status between nodes, and optimize network performance and resource utilization through dynamic network configuration adjustment. Blockchain technology is further used to upload real-time visual feedback reports to distributed storage to ensure its security and non-tamperability, and enhance the transparency and trust of streaming computing. Therefore, the present invention improves the dynamic adjustment and allocation capabilities of streaming computing through Internet of Things technology, intelligent control technology, visualization technology and blockchain technology, and strengthens the real-time monitoring and feedback capabilities of the visual computing model of streaming computing.

[0169] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0170] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for constructing a visual computing model based on streaming computing. It is characterized in that The following steps are involved: Step S1: Obtain a streaming computing data set; Perform data preprocessing on the streaming computing data set to obtain a standard streaming computing data set; Step S2: extract the computing nodes of the standard streaming computing data set to obtain computing node data; Perform correlation analysis on computing node data to obtain node correlation data; Perform network configuration analysis on computing node data to obtain node network configuration data; Perform node type analysis on computing node data to obtain node type data; perform node function analysis based on node type data to obtain node function data; integrate node association data, node network configuration data and node function data to generate computing node characteristic data; Step S3: extract the computing tasks of the standard streaming computing data set to obtain computing task data; Performing a processing path analysis on the computing task data to obtain task processing path data; performing an execution priority analysis on the computing task data to obtain task execution priority data; Perform resource utilization analysis on computing task data to obtain high resource demand task data; integrate task processing path data, task execution priority data and high resource demand task data to generate computing task management data; Step S4: constructing a visual computing model according to the computing node characteristic data and the computing task management data to obtain a visual computing model; Step S5: Performing network topology analysis on the computing node data based on the visual computing model to obtain node topology data; Dynamically adjust the node network configuration data based on the node topology data to obtain the computing node adjustment data; Based on the visual computing model, the computing task data is analyzed to obtain the task demand data; based on the task demand data, the task processing path data, the task execution priority data and the high resource demand task data are intelligently managed and controlled to generate computing task control data; Step S6: Perform real-time visual feedback on the streaming computing through computing node adjustment data and computing task control data, and generate a real-time visual feedback report; Upload the real-time visualization feedback report to the blockchain network distributed storage to perform streaming visualization computing jobs.

2. The method for constructing a visual computing model based on streaming computing according to claim 1, It is characterized in that Step S2 includes the following steps: Step S21: performing computing node feature recognition on the standard streaming computing data set to generate computing node feature information; Step S22: extracting node data of computing node characteristic information to obtain computing node data; Step S23: performing a correlation analysis of the interaction patterns of adjacent computing nodes on the computing node data to obtain node correlation data; Step S24: Perform network configuration analysis on the computing node data to obtain node network configuration data; Step S25: extracting attribute features of the computation node data to obtain node attribute feature data; classifying the computation node data into node types according to the node attribute feature data to obtain node type data; evaluating the node role according to the node type data to obtain node role evaluation data; and performing functional pathway analysis on the node role evaluation data to obtain node function data; Step S26: Integrate the node association data, node network configuration data and node function data to generate computing node characteristic data.

3. The method for constructing a visual computing model based on streaming computing according to claim 2, It is characterized in that Step S24 includes the following steps: Step S241: extracting and calculating the network characteristics of the node data to obtain the node network characteristic data; Step S242: performing network connection analysis on the node network characteristic data to obtain node network connection data; Step S243: performing connection status analysis according to the node network connection data to obtain node connection status data; Step S244: Analyze the configuration of the node connection status data to obtain node network configuration data.

4. The method for constructing a visualization computing model based on streaming computing according to claim 1, It is characterized in that Step S3 includes the following steps: Step S31: performing computing task feature recognition on the standard streaming computing data set to generate computing task feature information; Step S32: extracting task data of computing task characteristic information to obtain computing task data; Step S33: performing data mapping on the computing task data processing information to generate a task processing flow chart; identifying the data flow direction and processing steps on the task processing flow chart to obtain task processing path data; Step S34: performing execution priority analysis on the computing task data to obtain task execution priority data; Step S35: Perform statistical analysis on the required resource amount of the computing task data to obtain task resource amount statistical data; arrange the task resource amount statistical data in descending order to obtain task resource amount descending data; perform heat map color mapping on the task resource amount descending data to generate a task resource amount heat map; extract the dark coordinate data of the task resource amount heat map to obtain high resource demand task data; Step S36: Integrate the task processing path data, task execution priority data and high resource demand task data to generate computing task management data.

5. The method for constructing a visual computing model based on streaming computing according to claim 4, It is characterized in that Step S34 includes the following steps: Step S341: extracting the execution characteristics of the computing task data to obtain task execution characteristic data; Step S342: performing load analysis on the task execution characteristic data to obtain task execution load data; Step S343: extracting the peak and valley values ​​of the task execution load data to obtain the task execution load peak and valley data; Step S344: Calculate the average value of the peak and valley data of the task execution load to obtain the average value of the task execution load; Step S345: Compare the task execution load data with the task execution load average value. When the task execution load average value is greater than the task execution load average value, mark the computing task as task execution high load data; when the task execution load value is less than or equal to the task execution load average value, mark the computing task as task execution low load data. Step S346: performing high-priority classification on the task execution high-load data to obtain high-priority task data; performing low-priority classification on the task execution low-load data to obtain low-priority task data; Step S347: Integrate high-priority task data and low-priority task data to obtain task execution priority data.

6. The method for constructing a visualization computing model based on streaming computing according to claim 1, It is characterized in that Step S4 includes the following steps: Step S41: performing attribute parameter analysis on the calculation node characteristic data to generate node attribute parameter data; Step S42: Analyze the status information of the computing task management data to generate task status information data; Step S43: constructing a visualization computing model according to the node attribute parameter data and the task status information data to obtain a visualization computing pre-model; and training the visualization computing pre-model using the node attribute parameter data and the task status information data to obtain a visualization computing training model; Step S44: Perform model cross-validation evaluation on the visualization computing training model to obtain visualization computing model evaluation data; adjust model parameters of the visualization computing training model using the visualization computing model evaluation data to obtain a visualization computing model.

7. The method for constructing a visualization computing model based on streaming computing according to claim 1, It is characterized in that Step S5 includes the following steps: Step S51: Performing graph theory analysis on the network layout of the computing node data based on the visual computing model to obtain a node network layout diagram; extracting the connection attributes and layout mode of the node network layout diagram to obtain node network layout data; performing network topology analysis on the node network layout data to obtain node topology data; Step S52: dynamically adjusting the node network configuration data based on the node topology data to obtain computing node adjustment data; Step S53: performing a processing speed requirement analysis on the computing task data based on the visualized computing model to obtain processing speed requirement data; performing a memory requirement analysis on the computing task data based on the visualized computing model to obtain memory requirement data; performing a communication bandwidth requirement analysis on the computing task data based on the visualized computing model to obtain communication bandwidth requirement data; Step S54: Integrate the processing speed requirement data, the memory requirement data and the communication bandwidth requirement data to generate task requirement data; Step S55: Based on the task requirement data, intelligent task management and control are performed on the task processing path data, the task execution priority data and the high resource requirement task data to generate computing task control data.

8. The method for constructing a visual computing model based on streaming computing according to claim 7, It is characterized in that Step S52 includes the following steps: Step S521: Performing network performance evaluation on the node network configuration data according to the node topology data to obtain node network performance evaluation data; Step S522: performing anomaly analysis on the node network performance evaluation data to obtain network performance anomaly data; Step S523: extracting delay anomaly features of the network performance anomaly data to obtain delay anomaly data; extracting routing anomaly features of the network performance anomaly data to obtain routing anomaly data, wherein the routing anomaly data includes routing path anomaly and link selection anomaly; Step S524: performing load balancing adjustment on the node network configuration data based on the delay abnormality data to obtain node load balancing adjustment data; Step S525: Based on the routing anomaly characteristics, the node network configuration data is optimized and adjusted to obtain the node routing optimization data; if the computing node has a routing path anomaly, the routing path of the affected node in the computing node is replanned and the routing table is updated; if the computing node has a link selection anomaly, the dynamic routing protocol is used to intelligently switch the link; Step S526: Integrate the node load balancing adjustment data and the node routing optimization data to obtain computing node adjustment data.

9. The method for constructing a visual computing model based on streaming computing according to claim 7, It is characterized in that Step S55 includes the following steps: Step S551: Perform intelligent path planning on the task processing path data based on the task demand data. If the processing speed demand data is higher than the preset task speed demand threshold, the processing path of the computing task is changed to generate task change path data; if the processing speed demand data is less than or equal to the preset task speed demand threshold, the computing task is not processed to generate task default path data; the task change path data and the task default path data are integrated to obtain task path planning data; Step S552: Intelligently schedule the task execution priority data based on the task demand data. If the computing task is a high-priority task, optimal link allocation is performed on the computing task to generate task optimal link allocation data; if the computing task is a low-priority task, link redistribution is performed on the computing task to generate task link redistribution data; integrate the task optimal link allocation data and the task link redistribution data to obtain priority task intelligent scheduling data; Step S553: ​​Intelligent resource management is performed on the high resource demand task data based on the task demand data. If the computing task is a high resource demand task and a high priority task, the memory capacity and communication bandwidth rate are increased to generate task memory bandwidth improvement data; if the computing task is a high resource demand task and a low priority task, only the communication bandwidth rate is increased to generate task bandwidth improvement data; the task memory bandwidth improvement data and the task bandwidth improvement data are integrated to obtain task resource intelligent management data; Step S554: Integrate the task path planning data, priority task intelligent scheduling data and task resource intelligent management data to obtain computing task control data.

10. The method for constructing a visualization computing model based on streaming computing according to claim 1, It is characterized in that Step S6 includes the following steps: Step S61: Perform real-time visual control and monitoring on the streaming computing through computing node adjustment data and computing task control data to obtain real-time visual control data and real-time visual monitoring data; Step S62: Integrate the real-time visual control data and the real-time visual monitoring data to obtain a real-time visual feedback report; Step S63: Use blockchain technology to upload the real-time visualization feedback report to the blockchain node, and distribute the real-time visualization feedback report through the blockchain node to execute the visualization computing job of streaming computing.

Images