Cloud service infrastructure unit construction system, method, electronic device, and storage medium

Abstract

The application provides a computing network service basic unit construction system, method, electronic equipment and storage medium, belongs to the technical field of computing network, and the system comprises a resource aggregation module, a data fusion module, a model training module and an ability opening module. The computing network service basic unit construction system provided by the application realizes the access and unified management of multi-source information technology resources through the resource aggregation module; the data fusion module is used for fusing multiple resource management data to obtain multiple fusion data; the model training module is used for training artificial intelligence based on the multiple fusion data to obtain multiple large models; and the ability opening module is used for opening the multiple large models to external users. The combination of the four modules can improve the resource utilization, data fusion degree and model training efficiency in the information technology construction and operation mode, provide better computing network service ability for the industry and the industry, and effectively promote the digital transformation of the industry and the industry.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computing power network, and particularly relates to a computing power network service basic unit construction system and method, electronic equipment and storage medium. BACKGROUND

[0002] As a new network architecture, the computing power network aims to provide efficient and flexible computing services for users by integrating and optimizing computing resources. As an infrastructure, the computing power network has great value in resource scheduling efficiency, resource heterogeneous collaboration and data privacy security. However, as a provider of capabilities and services, there is no clear path for the computing power network to effectively promote the digital transformation of the industry and realize the effective landing of computing power network as value and computing power network as capability. SUMMARY

[0003] The present application provides a computing power network service basic unit construction system and method, electronic equipment and storage medium to solve the problem that the existing computing power network cannot effectively promote the digital transformation of the industry.

[0004] In a first aspect, the present application provides a computing power network service basic unit construction system, comprising a resource aggregation module, a data fusion module, a model training module and a capability opening module.

[0005] The resource aggregation module is configured to access multi-source information technology resources and uniformly manage the multi-source information technology resources to obtain a plurality of resource management data.

[0006] The data fusion module is configured to fuse the plurality of resource management data to obtain a plurality of fusion data.

[0007] The model training module is configured to train artificial intelligence based on the plurality of fusion data to obtain a plurality of large models.

[0008] The capability opening module is configured to open the plurality of large models to external users.

[0009] In one embodiment, the resource aggregation module comprises a resource access unit and a resource management unit.

[0010] The resource access unit is configured to perceive multi-source information technology resources and uniformly access the multi-source information technology resources; the multi-source information technology resources at least include enterprise internal computing power network resources, enterprise external computing power network resources and social collaborative computing power network resources.

[0011] The resource management unit is configured to integrate the multi-source information technology resources based on the multi-source information technology resources and manage the integrated multi-source information technology resources to obtain a plurality of resource management data.

[0012] In one embodiment, the data fusion module comprises a data filtering unit, a data cleaning unit and a data conversion unit.

[0013] The data filtering unit is configured to filter the plurality of resource management data according to preset conditions to obtain a plurality of filtered data.

[0014] The data cleaning unit is configured to clean error data and duplicate data in the plurality of filtered data to obtain a plurality of cleaned data.

[0015] The data conversion unit is configured to convert the plurality of cleaned data according to preset requirements to obtain a plurality of converted data.

[0016] In one embodiment, the data fusion module further comprises a data fusion unit.

[0017] The data fusion unit is configured to fuse the plurality of converted data according to different types to obtain a plurality of fused data.

[0018] In one embodiment, the model training module comprises a capability model training unit.

[0019] The capability model training unit is configured to train artificial intelligence according to different prediction tasks based on the plurality of fused data to obtain a plurality of large models.

[0020] In one embodiment, the model training module further comprises a capability model optimization unit.

[0021] The capability model optimization unit is configured to determine a model optimization strategy for each large model, and to optimize each large model based on the model optimization strategy.

[0022] In one embodiment, the capability opening module comprises an application programming interface service unit.

[0023] The application programming interface service unit is configured to call a target model from the plurality of large models according to a large model calling requirement of an external user, and to provide the target model to the external user.

[0024] In a second aspect, the present application further provides a method for constructing an algorithm network service basic unit, comprising:

[0025] Accessing a plurality of information technology resources, and uniformly managing the plurality of information technology resources to obtain a plurality of resource management data;

[0026] Fusing based on the plurality of resource management data to obtain a plurality of fused data;

[0027] Training artificial intelligence based on the plurality of fused data to obtain a plurality of large models.

[0028] The plurality of large models are opened to external users.

[0029] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for constructing a basic unit of an algorithm network service as described above when executing the program.

[0030] In a fourth aspect, the present application further provides a storage medium, comprising a non-transitory computer-readable storage medium, and a computer program stored on the storage medium, wherein the computer program is executable on a processor to implement the steps of the method for constructing a basic unit of an algorithm network service as described above.

[0031] The system and method for constructing a basic unit of an algorithm network service, the electronic device, and the storage medium provided by the present application can realize the access and unified management of multi-source information technology resources through the resource aggregation module, thereby improving the utilization rate and flexibility of various resources; the data fusion module can fuse multiple resource management data to obtain multiple fusion data, thereby ensuring the comprehensiveness and accuracy of the data and providing reliable data for model training; the model training module can perform artificial intelligence training based on the multiple fusion data to obtain diversified and high-performance large models, thereby improving the model training efficiency; the capability opening module can open the trained large models to external users, so that the external users can call the large models to meet the needs of the external users and provide good services and experiences for the external users. In combination with the four modules, the resource utilization rate, data fusion degree, and model training efficiency in the information technology construction and operation mode can be improved, better algorithm network service capabilities can be provided for the industry and the industry, thereby effectively promoting the digital transformation of the industry and the industry, and realizing the effective landing of algorithm network as value and algorithm network as capability. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0033] Figure 1 is a structural schematic diagram of the system for constructing a basic unit of an algorithm network service provided by the present application;

[0034] Figure 2 is a flowchart of the method for constructing a basic unit of an algorithm network service provided by the present application;

[0035] Figure 3 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0036] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0037] The terms 'first','second', and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein.

[0038] The technical solutions of the present application will be described below with reference to the drawings in the present application. Figures 1-3 The present application provides a grid service basic unit construction system, method, electronic device and storage medium.

[0039] Figure 1 The present application provides a grid service basic unit construction system. As shown in Figure 1 The grid service basic unit construction system provided by the present application comprises a resource aggregation module, a data fusion module, a model training module and a capability opening module.

[0040] The resource aggregation module is configured to access multi-source information technology resources and uniformly manage the multi-source information technology resources to obtain a plurality of resource management data.

[0041] The data fusion module is configured to fuse based on the plurality of resource management data to obtain a plurality of fusion data.

[0042] The model training module is configured to perform artificial intelligence training based on the plurality of fusion data to obtain a plurality of large models.

[0043] The capability opening module is configured to open the plurality of large models to external users.

[0044] It should be noted that the computing power network as a new network architecture aims to provide efficient and flexible computing services for users by integrating and optimizing computing resources. The computing power network is adapted according to different application scenarios (such as cloud computing, edge computing, high-performance computing, etc.). Different application scenarios may have different requirements for computing resources, network latency, data privacy, etc. The computing power network may have limitations in meeting diverse needs. Therefore, the present application considers that the computing power network in various application scenarios is different, and the global computing power network can be divided into regional computing power network, enterprise computing power network, industrial computing power network and industry computing power network, thereby providing corresponding computing power network for regions, enterprises, industries and industries, and providing corresponding computing power network is the basic unit of the computing network service, and the regions, enterprises, industries and industries can construct the basic unit of the computing network service. The computing network service basic unit fully considers and applies the characteristics of the computing power network to provide regional computing power network, enterprise computing power network, industrial computing power network and industry computing power network for regions, enterprises, industries and industries. Therefore, the computing power network native service can be considered to be constructed by a plurality of field and regional computing network service basic units, each computing network service basic unit includes a resource aggregation module, a data fusion module, a model training module and a capability opening module, and solves the problems of low resource utilization, low data fusion degree and low model training efficiency in the original information technology (IT) construction and operation mode.

[0045] Specifically, the resource aggregation module can utilize the computing network resource perception technology to perceive the existence and state of various IT resources, i.e. the existence and state of multi-source IT resources, which include physical resources (such as servers, storage devices), virtual resources (such as virtual machines, containers), cloud resources (such as cloud services, cloud storage) and the like. Through perception access, unified access of various IT resources can be achieved.

[0046] The resource aggregation module can also uniformly manage multi-source IT resources to obtain a plurality of resource management data, wherein the uniform management of multi-source IT resources can include resource registration, classification, monitoring, management, maintenance, etc. Through uniform management, centralized control and unified scheduling of resources can be achieved, improving management efficiency and convenience.

[0047] The resource aggregation module can also utilize the orchestration and scheduling technology to dynamically schedule multi-source IT resources, which includes automatic allocation of resources, load balancing, fault recovery, elastic scaling, etc. The utilization efficiency and flexibility of resources can be improved, and the security and reliability of the original facilities can be improved.

[0048] Further, the data fusion module is a key component in the computing power network service base unit. It utilizes the data integration, filtering, cleaning, conversion, and management capabilities of the computing power network and big data platform to integrate, filter, clean, convert, and manage data from different sources, formats, and structures for subsequent data analysis and processing. Therefore, the data fusion module can fuse multiple resource management data according to different types to obtain various fused data. This data fusion process aims to solve data silos and data fragmentation by integrating data scattered in various data sources into a unified data warehouse, providing a reliable data foundation for business decision-making, data analysis, and machine learning applications. The construction and operation of the data fusion module are of great significance for improving the usability, reliability, and value of data, and are one of the key links in modern big data and artificial intelligence applications.

[0049] Further, the model training module is an important component in the computing power network responsible for artificial intelligence model training. The model training module utilizes a large amount of computing resources, including computing servers, image processors, neural networks, and various distributed computing frameworks and deep learning frameworks, as well as machine learning libraries, algorithms, and other tools, to process and train data to train machine learning models that can perform specific tasks. Therefore, the model training module can train artificial intelligence based on various fused data to obtain various large models. The work of the model training module covers multiple links from data preprocessing, feature engineering, model selection, model training to model evaluation, and also includes large model optimization and deployment. Therefore, through continuous optimization and improvement of the model training module, the accuracy and performance of large models can be effectively improved to better meet business needs.

[0050] Further, the capability exposure module is a key component in the computing power network, responsible for exposing trained large models to external users and applications in the form of APIs or services, so that external users or applications can call the required large models to implement inference, prediction, or other tasks without training models themselves. The capability exposure module can also expose computing, storage, network, and other resources in the computing power network to external users and applications in the form of APIs or services. Therefore, the capability exposure layer can enable the integration of computing power networks and large models with external systems, achieve resource sharing and capability interoperability, and promote the application and development of artificial intelligence technology.

[0051] Further, the resource aggregation module includes a resource access unit and a resource management unit.

[0052] The resource access unit is configured to perceive multi-source information technology resources and uniformly access the multi-source information technology resources; the multi-source information technology resources at least include enterprise internal algorithm network resources, enterprise external algorithm network resources and social collaborative algorithm network resources.

[0053] The resource management unit is configured to integrate based on the multi-source information technology resources, and perform resource management on the integrated multi-source information technology resources to obtain a plurality of resource management data.

[0054] Specifically, the resource access unit can perceive multi-source IT resources by using algorithm network resource perception technology, and uniformly access the multi-source IT resources, wherein the multi-source IT resources at least include enterprise internal algorithm network resources, enterprise external algorithm network resources and social collaborative algorithm network resources. The enterprise internal algorithm network resources refer to various IT resources owned by an organization or enterprise, such as internal servers, storage devices, database systems and application programs, etc., which are used to support business applications and informatization construction within the organization. The enterprise external algorithm network resources refer to IT resources hosted by the organization or enterprise on a cloud platform or other external service providers, such as cloud servers, cloud storage, cloud databases, software operation service applications, etc., which are used by the organization or enterprise to expand their business capabilities and service scope by subscribing and using these external resources. The social collaborative algorithm network resources refer to IT resources from other organizations or individuals in society, such as open application programming interfaces (APIs), third-party services, shared data, etc., which can be obtained through cooperation, sharing or exchange, and can provide rich functions and services for the organization or enterprise.

[0055] Therefore, the resource access unit perceives and uniformly accesses various IT resources from multiple sources, so that the system can cross different networks and boundaries, which helps to improve the resource utilization efficiency and flexibility of the system, and also provides the enterprise with more extensive resource selection and utilization opportunities.

[0056] The resource management unit can integrate based on the multi-source IT resources, and perform resource management on the integrated multi-source IT resources to obtain a plurality of resource management data. The resource management mode includes but is not limited to resource management, application management, event management, performance management and operation and maintenance management.

[0057] Resource management refers to the management and monitoring of multiple IT resources in the system, including at least the management and monitoring of cloud platforms, virtual machines, physical machines, and other resources. This involves the management of resource configuration, allocation, scheduling, and other aspects to ensure that resources can be allocated as needed and used reasonably. Application management refers to the management and monitoring of various applications running in the system, including at least application management, application monitoring, and image management. This involves the management of application performance, availability, security, and other aspects to ensure that applications can run stably and meet user needs. Event management refers to the management and handling of various events occurring in the system, including at least event collection, event analysis, and event handling. This involves the management of event monitoring, analysis, response, and other aspects to ensure that the system can timely discover and solve problems and ensure stable operation. Performance management refers to the management and optimization of the performance of various components and functions of the system, including at least performance collection, performance analysis, and performance warning. This involves the management of system performance indicators, including monitoring, analysis, and tuning, to ensure efficient operation and provide good user experience. Operation and maintenance management refers to the management and execution of daily operation and maintenance work of the system, including at least inspection, migration, and capacity expansion. This involves the management of system operation status, including monitoring, maintenance, and updating, to ensure continuous and stable operation of the system. These resource management methods are interconnected and support each other, forming a complete framework for system resource management and providing protection for stable operation and efficient management of the system.

[0058] Further, the data fusion module includes a data filtering unit, a data cleaning unit, and a data conversion unit.

[0059] The data filtering unit is configured to filter the plurality of resource management data according to a preset condition to obtain a plurality of filtered data.

[0060] The data cleaning unit is configured to clean error data and duplicate data in the plurality of filtered data to obtain a plurality of cleaned data.

[0061] The data conversion unit is configured to convert the plurality of cleaned data according to a preset requirement to obtain a plurality of converted data.

[0062] Specifically, the data filtering unit can filter the plurality of resource management data according to a preset condition to obtain a plurality of filtered data, wherein the preset condition is set according to actual conditions. Therefore, the filtering process can filter and filter the resource management data according to specific conditions or standards, and only select the required digital data.

[0063] Further, the data cleaning unit can clean the error data and the duplicate data in the plurality of screening data to obtain a plurality of cleaning data. Therefore, the cleaning process can eliminate error data and duplicate data to ensure the accuracy and consistency of the cleaning data.

[0064] Further, the data conversion unit can perform data conversion processing on the plurality of cleaning data according to a preset requirement to obtain a plurality of conversion data, wherein the preset requirement is set according to the actual situation. Therefore, the data conversion processing can convert the cleaning data from one format to another format to obtain the conversion data to meet the needs of subsequent data analysis and processing.

[0065] Therefore, the combination of the data filtering unit, the data cleaning unit and the data conversion unit gradually pre-processes the resource management data, and finally obtains conversion data that meets the requirements, ensuring the accuracy, consistency and availability of the data, and providing a reliable data basis for subsequent data analysis and processing.

[0066] Further, the data fusion module further comprises a data fusion unit;

[0067] The data fusion unit is configured to fuse the plurality of conversion data according to different types to obtain a plurality of fusion data.

[0068] Specifically, the data fusion unit can fuse the plurality of conversion data according to different types according to the requirements and application scenarios to obtain a plurality of fusion data, so that the generated plurality of fusion data can provide more comprehensive and accurate information support for the system, and help the system to realize more efficient and intelligent operation and decision-making.

[0069] The plurality of fusion data at least includes business data, asset data, management data, operation and maintenance data, and industry data. The business data relates to the data generated by the core business activities of the organization or enterprise, which reflects the operation status and market situation of the organization or enterprise. The asset data includes the relevant data of various assets owned by the organization or enterprise, which reflects the resource allocation and asset status of the organization or enterprise. The management data includes various data generated in the management process of the organization or enterprise, which reflects the management efficiency and operation situation of the organization or enterprise. The operation and maintenance data relates to the running state and performance indicators of the system or device, which reflects the running status and stability of the system or device. The industry data includes various data related to the industry, which reflects the development trend and market environment of the industry.

[0070] Therefore, the data fusion unit can improve the fusion degree and availability of the data by fusing and aggregating the system management data to form a data warehouse, and provide comprehensive and applicable basic data for subsequent model training.

[0071] Further, the model training module comprises a capability model training unit.

[0072] The capability model training unit is configured to train artificial intelligence according to different prediction tasks based on the plurality of fusion data to obtain a plurality of large models.

[0073] Specifically, the capability model training unit is configured to train artificial intelligence according to different prediction tasks based on the plurality of fusion data to obtain a plurality of large models. During the model training process, interactive, visualized and automated functions are added to improve the efficiency of model training. Through an interactive interface or a command line tool, developers can interact with the capability model training unit by inputting parameters, selecting algorithms, monitoring the training process, and other ways to communicate and adjust the capability model training unit in real time to meet the needs. Through visualized tools and technologies, the process, results and related indicators of model training are presented to developers, which can include loss curves, accuracy curves, model structure visualization, data distribution visualization, etc. during the training process, to help developers more intuitively understand the model training situation and make corresponding adjustments and optimizations. Through automated tools and technologies, manual operations and repetitive operations of developers in the model training process are reduced, which can include functions such as automatic parameter tuning, automatic model selection, automatic deployment, etc. to improve the efficiency and reliability of model training.

[0074] By adding interactive, visualized and automated capabilities, the capability model training unit can better meet the needs of users, improve the efficiency and quality of model training, and accelerate the landing and promotion of artificial intelligence applications.

[0075] Further, the model training module further comprises a capability model optimization unit.

[0076] The capability model optimization unit is configured to determine a model optimization strategy for each large model, and to optimize each large model based on the model optimization strategy.

[0077] Specifically, the capability model optimization unit can determine a model optimization strategy for each large model, and optimize each large model based on the model optimization strategy, wherein the optimization methods include but are not limited to model acceleration, model compression and model conversion. Model acceleration is to improve the inference speed and efficiency of the model by optimizing the model structure, algorithm or hardware acceleration. Model compression is to reduce the storage space and computational complexity of the model to deploy and run the model in a resource-limited environment. Model conversion is to convert the original model into a model that runs efficiently on different hardware or platforms.

[0078] Through model acceleration, model compression, and model transformation, etc. Optimization can make large models more efficient, reliable and easy to deploy in various application scenarios.

[0079] In the model training module, a knowledge management unit is also developed, which involves model repository, risk management and model evaluation. Model repository is a platform or system for centralized storage, management and sharing of machine learning models. In the model repository, various types of models can be stored, including trained models, pre-trained models, open source models, etc. The model repository usually includes model version management, model metadata management, permission control, etc. to ensure the traceability, security and manageability of the model. In the process of model development and deployment, various risks need to be managed and controlled to ensure the robustness, reliability and security of the model. Risk management includes evaluation and monitoring of data quality, model performance, privacy protection, security vulnerabilities, etc. Through risk management, potential problems in the process of model development and deployment can be found and solved in time, reducing the impact of model-related risks on business. Model evaluation is the process of evaluating the performance and quality of machine learning models, which usually includes steps such as division of training set and test set, selection of evaluation indicators, cross-validation, etc. In the model evaluation process, the accuracy, precision, recall, F1 value, etc. of the model need to be considered comprehensively, as well as the generalization ability and stability of the model. Through model evaluation, the performance of the model can be quantified, and the model can be optimized and improved.

[0080] Considering the model repository, risk management and model evaluation, etc., a complete knowledge management system can be built, which helps to improve the efficiency and quality of model development and deployment, and promotes the application and development of artificial intelligence technology.

[0081] Further, the capability opening module includes an application programming interface service unit;

[0082] The application programming interface service unit is configured to invoke a target model in the plurality of large models according to a large model invocation requirement of an external user, and provide the target model to the external user.

[0083] Specifically, the API interface service unit can provide a service interface for external users or applications to call the trained large model, and the external users or applications can dispatch a model that meets the large model calling demand through the service interface. Further, the API interface service unit can call a target model in multiple large models according to the large model calling demand of the external users or applications, and provide the target model to the external users or applications through the service interface. The external users or applications include, but are not limited to, digital government, smart city, industrial internet, and digital people. The digital government can apply the target model to emergency command, comprehensive management and environmental protection, and one-network management / governance application scenarios. The smart city can apply the target model to smart transportation, smart medical care, and smart education application scenarios. The industrial internet can apply the target model to industrial internet, smart mine, and smart energy / twin carbon application scenarios. The digital people can apply the target model to intelligent customer service and metaverse digital character creation.

[0084] Moreover, once the target large model is optimized, the external users or applications will call the optimized target large model, which means that the model may be improved in performance, accuracy, or other indicators to better meet user needs.

[0085] It should be noted that the algorithm network service basic unit construction system is also provided with a security guarantee mechanism (such as identity security, scheduling security, data security, resource security, algorithm network security, etc.) and a standard specification system (such as technical interface, sharing specification, cleaning mechanism, supervision and review, incentive mechanism, etc.) for ensuring the security, reliability, and standard operation of the algorithm network, promoting resource sharing and cooperative development.

[0086] The algorithm network service basic unit construction system provided by the application can realize the access and unified management of multi-source information technology resources through the resource aggregation module, improve the utilization rate and flexibility of various resources, fuse multiple resource management data based on the data fusion module to obtain various fusion data, ensure the comprehensiveness and accuracy of the data, and provide reliable data for model training, train artificial intelligence based on the multiple fusion data through the model training module to obtain diversified and high-performance large models, improve the model training efficiency, and open the trained large models to external users through the capability opening module for the external users to call the large models to meet the needs of the external users and provide good services and experiences. Combined with the four modules, the resource utilization rate, data fusion degree, and model training efficiency in the information technology construction and operation mode can be improved, better algorithm network service capabilities can be provided for industries and industries, thereby effectively promoting the digital transformation of industries and industries, and realizing the effective landing of algorithm network as value and algorithm network as capability.

[0087] Figure 2is a flowchart of the method for constructing the algorithm network service basic unit provided by the present application. Referring to Figure 2 The method for constructing the algorithm network service basic unit provided by the present application can include:

[0088] Step 100: Accessing multi-source information technology resources and uniformly managing the multi-source information technology resources to obtain a plurality of resource management data;

[0089] Step 200: Fusing based on the plurality of resource management data to obtain a plurality of fusion data;

[0090] Step 300: Artificial intelligence training based on the plurality of fusion data to obtain a plurality of large models;

[0091] Step 400: Opening the plurality of large models to external users.

[0092] It should be noted that the computing power network is a new network architecture, which aims to integrate and optimize computing resources to provide efficient and flexible computing services for users. The computing power network is adapted to different application scenarios (such as cloud computing, edge computing, high-performance computing, etc.). Different application scenarios may have different requirements for computing resources, network latency, data privacy, etc. The computing power network may have limitations in meeting diverse needs. Therefore, the present application considers that the computing power network in various application scenarios is different, and the global computing power network can be divided into regional computing power network, enterprise computing power network, industrial computing power network and industry computing power network, thereby providing corresponding computing power network for regions, enterprises, industries and industries, and providing corresponding computing power network is the algorithm network service basic unit. The regional, enterprise, industrial and industry algorithm network service basic unit can be constructed, fully considering and applying the characteristics of the computing power network to provide regional computing power network, enterprise computing power network, industrial computing power network and industry computing power network. Therefore, the computing power network native service can be considered to be constructed by a plurality of field and regional algorithm network service basic units, each algorithm network service basic unit includes a resource aggregation module, a data fusion module, a model training module and a capability opening module, solving the problems of low resource utilization, low data fusion degree and low model training efficiency in the original information technology (Information Technology, IT) construction and operation mode.

[0093] Specifically, the resource aggregation module can utilize the algorithm network resource perception technology, which can perceive the existence and state of various IT resources, i.e. the existence and state of multi-source IT resources, which include physical resources (such as servers, storage devices), virtual resources (such as virtual machines, containers), cloud resources (such as cloud services, cloud storage) and the like. Through perception access, unified access of various IT resources can be achieved.

[0094] The resource aggregation module can also uniformly manage the multi-source IT resources to obtain multiple resource management data. The uniform management of the multi-source IT resources can include registration, classification, monitoring, management, maintenance, etc. of the resources. Through the uniform management, centralized control and unified scheduling of the resources can be realized, and management efficiency and convenience can be improved.

[0095] The resource aggregation module can also use orchestration and scheduling technology to dynamically schedule the multi-source IT resources, which includes automatic allocation, load balancing, fault recovery, elastic scaling, etc. of the resources, so as to improve the utilization efficiency and flexibility of the resources and improve the safety and reliability of the original facilities.

[0096] Further, the data fusion module is a key component of the computing network service foundation unit. It uses the data integration, filtering, cleaning, conversion, and management capabilities of the computing power network and big data platform to integrate, filter, clean, convert, and manage data from different sources, formats, and structures for subsequent data analysis and processing. Therefore, the data fusion module can fuse multiple resource management data according to different types to obtain various fusion data. This data fusion process is to solve the problems of data islands and data fragmentation, integrate data scattered in various data sources into a unified data warehouse, and provide a reliable data foundation for business decision-making, data analysis, and machine learning applications. The construction and operation of the data fusion module are of great significance to improve the availability, reliability, and value of data, and are one of the key links of modern big data and artificial intelligence applications.

[0097] Further, the model training module is an important component of the computing power network responsible for artificial intelligence model training. The model training module uses a large amount of computing resources, including computing servers, image processors, neural networks, various distributed computing frameworks and deep learning frameworks, machine learning libraries, algorithms, and other tools to process and train data to train machine learning models that can perform specific tasks. Therefore, the model training module can perform artificial intelligence training based on various fusion data to obtain various large models. The work of the model training module covers multiple links from data preprocessing, feature engineering, model selection, model training to model evaluation, and also includes large model optimization and deployment. Therefore, through the continuous optimization and improvement of the model training module, the accuracy and performance of the large model can be effectively improved to better meet business needs.

[0098] Further, the capability opening module is a key component in the computing power network, responsible for opening the trained large model in the form of API or service to external users and applications, so that external users or applications can call the required large model to realize inference, prediction or other tasks without training the model themselves. The capability opening module can also open the computing, storage, network and other resources in the computing power network to external users and applications in the form of API or service. Therefore, the capability opening layer can enable the computing power network and large model to integrate with external systems, realize resource sharing and capability interoperability, and promote the application and development of artificial intelligence technology.

[0099] Further, the resource aggregation module includes a resource access unit and a resource management unit;

[0100] The resource access unit is configured to perceive multi-source information technology resources and uniformly access the multi-source information technology resources; the multi-source information technology resources at least include enterprise internal computing network resources, enterprise external computing network resources and social collaborative computing network resources.

[0101] The resource management unit is configured to integrate based on the multi-source information technology resources, and manage the integrated multi-source information technology resources to obtain a plurality of resource management data.

[0102] Specifically, the resource access unit can perceive multi-source IT resources using computing network resource perception technology, and uniformly access multi-source IT resources, wherein the multi-source IT resources at least include enterprise internal computing network resources, enterprise external computing network resources and social collaborative computing network resources. The enterprise internal computing network resources refer to various IT resources owned by an organization or enterprise, such as internal servers, storage devices, database systems and applications, etc., which are used to support business applications and informatization construction within the organization. The enterprise external computing network resources refer to IT resources hosted by the organization or enterprise on a cloud platform or other external service providers, such as cloud servers, cloud storage, cloud databases, software operation service applications, etc. The organization or enterprise expands its business capabilities and service scope by subscribing and using these external resources. The social collaborative computing network resources refer to IT resources from other organizations or individuals in society, such as open application programming interfaces (APIs), third-party services, shared data, etc. These resources can be obtained through cooperation, sharing or exchange, and can provide rich functions and services for organizations or enterprises.

[0103] Therefore, the resource access unit perceives and uniformly accesses various IT resources from multiple sources, enabling the system to cross different networks and boundaries, helping to improve the resource utilization efficiency and flexibility of the system, and also providing enterprises with more extensive resource selection and utilization opportunities.

[0104] The resource management unit can integrate the multi-source IT resources based on the multi-source IT resources and manage the integrated multi-source IT resources to obtain a plurality of resource management data. The resource management mode includes, but is not limited to, resource management, application management, event management, performance management, and operation and maintenance management.

[0105] Resource management refers to the management and monitoring of multi-source IT resources in the system, including at least the management and monitoring of resources such as cloud platforms, virtual machines, and physical machines. This involves management of resource configuration, allocation, scheduling, and the like to ensure that resources can be allocated as needed and used reasonably. Application management refers to the management and monitoring of various application programs running in the system, including at least application management, application monitoring, and image management. This involves management of application performance, availability, security, and the like to ensure that applications can run stably and meet user needs. Event management refers to the management and processing of various events occurring in the system, including at least event collection, event analysis, and event processing. This involves management of event monitoring, analysis, response, and the like to ensure that the system can discover and solve problems in a timely manner and ensure stable operation of the system. Performance management refers to the management and optimization of the performance of various components and functions of the system, including at least performance collection, performance analysis, and performance warning. This involves management of system performance indicators for monitoring, analysis, and tuning to ensure efficient operation of the system and provide good user experience. Operation and maintenance management refers to the management and execution of daily operation and maintenance of the system, including at least inspection, migration, and capacity expansion. This involves management of system operation status for monitoring, maintenance, and updating to ensure continuous and stable operation of the system. These resource management modes are interrelated and support each other, forming a complete framework for system resource management and providing protection for stable operation and efficient management of the system.

[0106] Further, the data fusion module includes a data filtering unit, a data cleaning unit, and a data conversion unit.

[0107] The data filtering unit is configured to filter the plurality of resource management data according to a preset condition to obtain a plurality of filtered data.

[0108] The data cleaning unit is configured to clean error data and duplicate data in the plurality of filtered data to obtain a plurality of cleaned data.

[0109] The data conversion unit is configured to convert the plurality of cleaned data according to a preset requirement to obtain a plurality of converted data.

[0110] Specifically, the data filtering unit can filter the plurality of resource management data according to preset conditions to obtain a plurality of filtered data, wherein the preset conditions are set according to actual situations. Therefore, the filtering process can filter the resource management data according to specific conditions or standards, and only select the required digital data.

[0111] Further, the data cleaning unit can clean the error data and the repeated data in the plurality of filtered data to obtain a plurality of cleaned data. Therefore, the cleaning process can eliminate the error data and the repeated data, and ensure the accuracy and consistency of the cleaned data.

[0112] Further, the data conversion unit can perform data conversion processing on the plurality of cleaned data according to preset requirements to obtain a plurality of converted data, wherein the preset requirements are set according to actual situations. Therefore, the data conversion processing can convert the cleaned data from one format to another format to obtain the converted data, so as to meet the needs of subsequent data analysis and processing.

[0113] Therefore, the combination of the data filtering unit, the data cleaning unit and the data conversion unit gradually pre-processes the resource management data, and finally obtains the converted data meeting the requirements, so as to ensure the accuracy, consistency and availability of the data, and provide a reliable data basis for subsequent data analysis and processing.

[0114] Further, the data fusion module further comprises a data fusion unit.

[0115] The data fusion unit is configured to fuse the plurality of converted data according to different types to obtain a plurality of fused data.

[0116] Specifically, the data fusion unit can fuse the plurality of converted data according to different types according to requirements and application scenarios to obtain a plurality of fused data, so that the plurality of fused data generated can provide more comprehensive and accurate information support for the system, and help the system to realize more efficient and intelligent operation and decision-making.

[0117] The plurality of fused data at least includes business data, asset data, management data, operation and maintenance data, and industry data. The business data relates to data generated by the core business activities of an organization or enterprise, which reflects the operation status and market situation of the organization or enterprise. The asset data includes relevant data of various assets owned by the organization or enterprise, which reflects the resource allocation and asset status of the organization or enterprise. The management data includes various data generated in the management process of the organization or enterprise, which reflects the management efficiency and operation situation of the organization or enterprise. The operation and maintenance data relates to the running state and performance indicators of systems or devices, which reflects the running status and stability of the systems or devices. The industry data includes various data related to the industry, which reflects the development trend and market environment of the industry.

[0118] Therefore, the data fusion unit improves the fusion degree and availability of data by fusing and aggregating system management data to form a data warehouse, thereby providing comprehensive and applicable basic data for subsequent model training.

[0119] Further, the model training module includes a capability model training unit.

[0120] The capability model training unit is configured to train artificial intelligence based on the plurality of fused data according to different prediction tasks to obtain a plurality of large models.

[0121] Specifically, the capability model training unit is configured to train artificial intelligence based on the plurality of fused data according to different prediction tasks to obtain a plurality of large models. During model training, interactive, visual, and automated functions are added to improve the efficiency of model training. Through an interactive interface or command line tool, developers can interact with the capability model training unit by inputting parameters, selecting algorithms, monitoring the training process, and other ways to communicate and adjust the capability model training unit in real time to meet the needs. Through visualization tools and techniques, the process, results, and related indicators of model training are presented to developers, which can include loss curves, accuracy curves, model structure visualization, data distribution visualization, and other functions during training, helping developers more intuitively understand the model training situation and make corresponding adjustments and optimizations. Through automated tools and techniques, developers' manual operations and repetitive operations during model training are reduced, which can include functions such as automatic parameter tuning, automatic model selection, and automatic deployment, improving the efficiency and reliability of model training.

[0122] By adding interactive, visual, and automated capabilities, the capability model training unit can better meet the needs of users, improve the efficiency and quality of model training, and accelerate the landing and promotion of artificial intelligence applications.

[0123] Further, the model training module further comprises a capability model optimization unit;

[0124] The capability model optimization unit is configured to determine a model optimization strategy for each large model respectively, and to optimize each large model based on the model optimization strategy.

[0125] Specifically, the capability model optimization unit can determine a model optimization strategy for each large model respectively, and to optimize each large model based on the model optimization strategy, wherein the optimization methods include but are not limited to model acceleration, model compression and model transformation. Model acceleration is to improve the inference speed and efficiency of the model by optimizing the model structure, algorithm or hardware acceleration. Model compression is to reduce the storage space and computational complexity of the model to deploy and run the model in a resource-limited environment. Model transformation is to convert the original model into a model that runs efficiently on different hardware or platforms.

[0126] Through model acceleration, model compression and model transformation, the large model can be more efficient, reliable and easy to deploy in various application scenarios.

[0127] In the model training module, a knowledge management unit is also developed, which involves model repository, risk management and model evaluation. The model repository is a platform or system for centralized storage, management and sharing of machine learning models. In the model repository, various types of models can be stored, including trained models, pre-trained models, open source models, etc. The model repository usually includes model version management, model metadata management, permission control and other functions to ensure the traceability, security and manageability of the model. During the model development and deployment process, various risks need to be managed and controlled to ensure the robustness, reliability and security of the model. Risk management includes evaluation and monitoring of data quality, model performance, privacy protection, security vulnerabilities, etc. Through risk management, potential problems in the model development and deployment process can be found and solved in time, reducing the impact of model-related risks on business. Model evaluation is the process of evaluating the performance and quality of machine learning models, which usually includes steps such as division of training set and test set, selection of evaluation indicators, cross-validation, etc. During the model evaluation process, the accuracy, precision, recall, F1 value and other indicators of the model need to be considered comprehensively, as well as the generalization ability and stability of the model. Through model evaluation, the performance of the model can be quantified, and the model can be optimized and improved.

[0128] By considering the model repository, risk management and model evaluation, a complete knowledge management system can be built, which helps to improve the efficiency and quality of model development and deployment, and promotes the application and development of artificial intelligence technology.

[0129] Further, the capability opening module comprises an application programming interface service unit;

[0130] The application programming interface service unit is configured to invoke a target model from a plurality of large models according to a large model invocation requirement of an external user, and provide the target model to the external user.

[0131] Specifically, the API interface service unit can provide a service interface for the external user or application program to invoke the trained large model. The external user or application program dispatches a model that meets the large model invocation requirement through the service interface. Further, the API interface service unit invokes a target model from a plurality of large models according to the large model invocation requirement of the external user or application program, and provides the target model to the external user or application program through the service interface. The external user or application program includes but is not limited to digital government, smart city, industrial internet, and digital human. The digital government can apply the target model to emergency command, comprehensive management and environmental protection, and one-stop management / governance application scenarios. The smart city can apply the target model to smart transportation, smart medical care, and smart education application scenarios. The industrial internet can apply the target model to industrial internet, smart mine, and smart energy / twin carbon application scenarios. The digital human can apply the target model to intelligent customer service and metaverse digital character creation.

[0132] Moreover, once the target large model is optimized, the external user or application program will invoke the optimized target large model, which means that the model may have improved performance, accuracy, or other indicators to better meet user needs.

[0133] It should be noted that the algorithm network service base unit construction system is also provided with a security guarantee mechanism (such as identity security, dispatching security, data security, resource security, algorithm network security, etc.) and a standard specification system (such as technical interface, sharing specification, cleaning mechanism, supervision and review, incentive mechanism, etc.) to ensure the security, reliability, and standard operation of the algorithm power network, and promote resource sharing and cooperative development.

[0134] The application provides a network algorithm service basic unit construction method, multi-source information technology resources are accessed and uniformly managed through a resource aggregation module, the utilization rate and flexibility of various resources can be improved, fusion is performed on the basis of multiple resource management data through a data fusion module, various fusion data are obtained to ensure the comprehensiveness and accuracy of the data and provide reliable data for model training, artificial intelligence training is performed on the basis of the various fusion data through a model training module, diversified and high-performance large models are obtained, and the model training efficiency is improved, the various large models trained are opened to external users through an ability opening module, the large models are called by the external users, the needs of the external users are met, good services and experiences are provided for the external users, the resource utilization rate, data fusion degree and model training efficiency in the information technology construction and operation mode are improved, better network algorithm service capabilities are provided for the industry and the industry, the digital transformation of the industry and the industry is effectively promoted, and the effective landing of the network algorithm as value and the network algorithm as capability is realized.

[0135] Figure 3 is a structural schematic diagram of an electronic device provided by the application, as shown in Figure 3 The electronic device can include a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 complete mutual communication through the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute the network algorithm service basic unit construction method, the method comprising: accessing multi-source information technology resources, and uniformly managing the multi-source information technology resources to obtain multiple resource management data; fusing on the basis of the multiple resource management data to obtain various fusion data; performing artificial intelligence training on the basis of the various fusion data to obtain various large models; and opening the various large models to external users.

[0136] In addition, the logic instructions in the memory 330 described above can be realized in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0137] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, and the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer can execute the algorithm service unit construction method provided by the above-mentioned embodiments, and the method comprises the following steps: accessing multi-source information technology resources, and uniformly managing the multi-source information technology resources to obtain a plurality of resource management data; fusing based on the plurality of resource management data to obtain a plurality of fusion data; training artificial intelligence based on the plurality of fusion data to obtain a plurality of large models; and opening the plurality of large models to external users.

[0138] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, and the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer can execute the algorithm service unit construction method provided by the above-mentioned embodiments, and the method comprises the following steps: accessing multi-source information technology resources, and uniformly managing the multi-source information technology resources to obtain a plurality of resource management data; fusing based on the plurality of resource management data to obtain a plurality of fusion data; training artificial intelligence based on the plurality of fusion data to obtain a plurality of large models; and opening the plurality of large models to external users.

[0139] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on a plurality of network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0140] Those skilled in the art can clearly understand the implementation of the embodiments by means of software and necessary general hardware platforms through the description of the above embodiments, and the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in each embodiment or some parts of the embodiment.

[0141] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A system for constructing basic units for computing network services, characterized in that, It includes a resource aggregation module, a data fusion module, a model training module, and a capability opening module; The resource aggregation module is used to access multi-source information technology resources and to manage these resources in a unified manner, thereby obtaining multiple resource management data. The data fusion module is used to fuse the multiple resource management data to obtain various fused data. The model training module is used to perform artificial intelligence training based on the multiple fused data to obtain multiple large models; The capability opening module is used to open up the various large models to external users; The resource aggregation module includes a resource access unit and a resource management unit; The resource access unit is used to sense multi-source information technology resources and uniformly access the multi-source information technology resources; the multi-source information technology resources include at least enterprise internal computing network resources, enterprise external computing network resources, and social collaborative computing network resources; The resource management unit is used to integrate the multi-source information technology resources and to manage the integrated multi-source information technology resources to obtain multiple resource management data.

2. The computer network service basic unit construction system according to claim 1, characterized in that, The data fusion module includes a data filtering unit, a data cleaning unit, and a data conversion unit; The data filtering unit is used to filter the multiple resource management data according to preset conditions to obtain multiple filtered data. The data cleaning unit is used to clean up erroneous and duplicate data in the multiple filtered data to obtain multiple cleaned data. The data conversion unit is used to perform data conversion processing on the multiple cleaned data according to preset requirements to obtain multiple converted data.

3. The computer network service basic unit construction system according to claim 1, characterized in that, The data fusion module further includes a data fusion unit; The data fusion unit is used to fuse the multiple transformed data according to different types to obtain various fused data.

4. The computer network service basic unit construction system according to claim 1, characterized in that, The model training module includes a capability model training unit; The capability model training unit is used to perform artificial intelligence training based on the multiple fused data and according to different prediction tasks to obtain multiple large models.

5. The computer network service basic unit construction system according to claim 4, characterized in that, The model training module also includes a capability model optimization unit; The capability model optimization unit is used to determine the model optimization strategies for various large models, and to fine-tune the various large models based on the optimization strategies.

6. The computer network service basic unit construction system according to claim 1, characterized in that, The capability opening module includes an application programming interface service unit; The application programming interface service unit is used to call the target model among multiple large models according to the large model calling requirements of external users, and provide the target model to the external users.

7. A method for constructing a basic unit for computing network services, characterized in that, include: Access to multi-source information technology resources and unified management of these resources yield multiple resource management data. By fusing the aforementioned multiple resource management data, various types of fused data are obtained; Artificial intelligence training is performed based on the aforementioned fused data to obtain various large models; Make these various large models available to external users; The access to multi-source information technology resources and the unified management of these resources result in multiple resource management data, including: The system senses and accesses multi-source information technology resources in a unified manner; these multi-source information technology resources include at least internal enterprise computing network resources, external enterprise computing network resources, and social collaborative computing network resources. Based on the integration of the multi-source information technology resources, and the resource management of the integrated multi-source information technology resources, multiple resource management data are obtained.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the computer network service basic unit construction method as described in claim 7.

9. A storage medium comprising a non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the computer network service basic unit construction method as described in claim 7.

Images