Information processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses an information processing method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining first data, and analyzing and processing the first data by utilizing a first model to obtain an analysis result; the first data comprises at least one of image data, text data and voice data; the analysis result comprises at least one task; calling the capability corresponding to each task to process the corresponding task to obtain a processing result; and obtaining an execution result according to the at least one processing result, and outputting information related to the execution result.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an information processing method, device, electronic device and storage medium. Background Art

[0002] With the booming development of the data center industry, data centers are taking on more and more data storage, processing and transmission tasks. As the core of critical information infrastructure, the integrity and intelligence of the data center operation system are crucial to maintaining the security of important equipment and data. Once there is a problem with the status of the computer room, the scope of impact will be very wide. However, traditional data center management methods are heavily dependent on manual work and are restricted by the site environment. This is not only inefficient and hinders business development, but also cannot fully monitor the ongoing work and problems in the computer room in real time, so it faces many challenges.

[0003] With the remarkable progress of general artificial intelligence technology represented by large language models, artificial intelligence (AI) has transformed from an auxiliary means to a key infrastructure and core capability to support economic and social transformation, which provides solid technical support for digital operation and maintenance. However, in the complex network operation and maintenance scenarios, although the data center room can provide real-time monitoring data and alarm information, it is relatively weak in intelligent decision support and can only provide basic data display and simple analysis functions, and it is difficult to provide in-depth decision-making suggestions. Summary of the invention

[0004] To solve the existing technical problems, the embodiments of the present invention provide an information processing method, device, electronic device and storage medium.

[0005] To achieve the above object, the technical solution of the embodiment of the present invention is implemented as follows:

[0006] In a first aspect, an embodiment of the present invention provides an information processing method, which is applied to a data center, and includes:

[0007] Obtaining first data, and analyzing and processing the first data using a first model to obtain an analysis result; the first data includes at least one of the following: image data, text data, and voice data; the analysis result includes at least one task;

[0008] Call the capability corresponding to each task to process the corresponding task and obtain the processing result;

[0009] An execution result is obtained according to at least one processing result, and first information is output, where the first information is information related to the execution result.

[0010] In the above scheme, calling the capability corresponding to each task to process the corresponding task includes: calling at least one second model to process the corresponding task, and each second model has the capability to process the corresponding task.

[0011] In the above scheme, the first data is analyzed and processed using the first model to obtain an analysis result; including: using the first model to identify at least one second information of the first data and extracting at least one third information from the first data; determining the third information corresponding to each second information, and obtaining an analysis result containing the corresponding second information and third information; wherein the second information represents the processing intention of the first data; and the third information includes task-related parameters.

[0012] In the above solution, before analyzing and processing the first data using the first model, the method further includes:

[0013] Processing the first data using a third model to obtain an initial target word;

[0014] The first data is processed based on the initial target word.

[0015] In the above solution, the analyzing and processing the first data by using the first model includes:

[0016] The first data after being processed is analyzed and processed using a first model.

[0017] In the above solution, the step of processing the first data using the third model to obtain the initial target word includes:

[0018] The third model is used to perform the following operations on the first data:

[0019] Performing feature extraction on the first data to obtain a first feature; the first data includes image data and / or text data; the first feature includes a first image feature and / or a first text feature;

[0020] Performing low-rank decomposition processing on the first feature to obtain a second feature; the second feature includes a second image feature and / or a second text feature;

[0021] Key information of the first data is obtained based on the second feature, and an initial target word corresponding to the key information is determined based on a preset vocabulary library.

[0022] In the above scheme, when the second feature includes a second image feature and a second text feature, obtaining the key information of the first data based on the second feature includes: fusing the second image feature and the second text feature to obtain a fused feature, and obtaining the key information of the first data based on the fused feature.

[0023] In the above solution, when the first data includes image data, the processing of the first data based on the initial target word includes at least one of the following processing:

[0024] Image cropping, image rotation, image reduction, image enlargement, color dithering.

[0025] In the above solution, the step of using the first model to identify at least one second information of the first data and extracting at least one third information from the first data includes:

[0026] Using the first model, identifying a plurality of first-category words in the first data that represent processing intentions, and extracting a plurality of second-category words that represent task-related parameters;

[0027] The frequency of occurrence of each word in the first category of words and the second category of words is counted respectively, and the first category of words with a frequency higher than a first threshold is used as the at least one second information, and the second category of words with a frequency higher than a second threshold is used as the at least one third information.

[0028] In the above solution, the step of determining the third information corresponding to each second information includes:

[0029] The at least one second information and the at least one third information are classified according to a preset classification rule, and the third information corresponding to each second information is determined according to the classification result.

[0030] In the above scheme, the method also includes: generating training data according to the analysis result, and training the second model using the training data and the first data.

[0031] In the above solution, the outputting of the first information includes:

[0032] directly outputting or displaying the execution result; and / or,

[0033] A knowledge base is queried according to the execution result to obtain fourth information related to the execution result, and the fourth information is output or displayed, where the fourth information includes at least one of the following: fault information, root cause location information, and solution information.

[0034] In the above solution, the obtaining of the first data includes at least one of the following:

[0035] Obtain image data and / or voice data collected by cameras in the data center computer room;

[0036] Obtain image data and / or voice data collected by a camera set up by the inspection robot;

[0037] Obtaining image data and / or voice data collected by a terminal device;

[0038] Obtain at least one of performance data, resource data, alarm data, and industrial parameter data during equipment operation;

[0039] Get the text data entered by the user.

[0040] In the above solution, the outputting of the first information includes:

[0041] Outputting and displaying information related to the execution result through a display component; and / or,

[0042] The first information is sent to the terminal, so that the terminal can output and display information related to the execution result.

[0043] In a second aspect, an embodiment of the present invention further provides an information processing method, which is applied to a terminal and includes:

[0044] receiving first information sent by a data center, wherein the first information is information related to an execution result obtained by the data center invoking a capability corresponding to each task in at least one task to process the corresponding task and integrating at least one processing result, wherein the at least one task is obtained by the data center analyzing and processing first data using a first model, and the first data includes at least one of the following: image data, text data, and voice data;

[0045] Output and display information related to the execution result.

[0046] In the above solution, the output and display of information related to the execution result includes:

[0047] Output first image data and / or first audio data, wherein the first image data and / or the first audio data correspond to information related to the execution result, and the first image data is used to overlay and display a virtual screen corresponding to the first image data in a real scene.

[0048] In a third aspect, an embodiment of the present invention further provides an information processing device, which is applied to a data center, and comprises: a data acquisition unit, a processing unit and a first output unit; wherein,

[0049] The data acquisition unit is used to obtain first data, wherein the first data includes at least one of the following: image data, text data, and voice data;

[0050] The processing unit is used to analyze and process the first data using the first model to obtain an analysis result; the analysis result includes at least one task; and is also used to call the capability corresponding to each task to process the corresponding task to obtain a processing result; and obtain an execution result according to at least one processing result;

[0051] The first output unit is used to output first information, where the first information is information related to the execution result.

[0052] In a fourth aspect, an embodiment of the present invention further provides an information processing device, which is applied to a terminal and includes: a communication unit and a second output unit; wherein:

[0053] The communication unit is used to receive first information sent by a data center, the first information being information related to an execution result obtained by the data center invoking a capability corresponding to each task in at least one task to process the corresponding task and integrating at least one processing result, the at least one task being obtained by the data center analyzing and processing first data using a first model, the first data comprising at least one of the following: image data, text data, and voice data;

[0054] The second output unit is used to output and display information related to the execution result.

[0055] In a fifth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the information processing method described in the first aspect or the second aspect of the embodiment of the present invention.

[0056] In a sixth aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the information processing method described in the first aspect or the second aspect of the embodiment of the present invention are implemented.

[0057] In a seventh aspect, an embodiment of the present invention further provides a computer program product, comprising computer program instructions, which enable a computer to execute the steps of the information processing method described in the first aspect or the second aspect of the embodiment of the present invention.

[0058] The information processing method, device, electronic device and storage medium provided by the embodiment of the present invention obtain first data, analyze and process the first data using a first model to obtain an analysis result; the first data includes at least one of the following: image data, text data, voice data; the analysis result includes at least one task; the capability corresponding to each task is called to process the corresponding task to obtain a processing result; the execution result is obtained according to at least one processing result, and information related to the execution result is output. The technical solution of the embodiment of the present invention is adopted, and the multimodal data (first data) is processed using the first model to realize task decomposition and intent recognition; and then the corresponding tasks are processed by calling the capabilities corresponding to each task to provide targeted solutions, thereby realizing intelligent decision-making of the data center. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of a system architecture used by an information processing method according to an embodiment of the present invention;

[0060] Figure 2 A schematic diagram of a flow chart of an information processing method according to an embodiment of the present invention;

[0061] Figure 3 Schematic diagram of a process of processing first data using a third model in an information processing method according to an embodiment of the present invention;

[0062] Figure 4 Another schematic diagram of a flow chart of an information processing method in an embodiment of the present invention;

[0063] Figure 5 is a schematic diagram of a process of training a second model in an information processing method according to an embodiment of the present invention;

[0064] Figure 6 A schematic diagram of an application of the information processing method according to an embodiment of the present invention;

[0065] Figure 7 It is another flowchart of the information processing method according to the embodiment of the present invention;

[0066] Figure 8 Another application schematic diagram of the information processing method according to an embodiment of the present invention;

[0067] Fig. 9 The structure diagram of the information processing device according to the embodiment of the present invention is shown in FIG. Figure 1 ;

[0068] Fig.10 The structure diagram of the information processing device according to the embodiment of the present invention is shown in FIG. Figure 2 ;

[0069] Fig.11The figure is a schematic diagram of the hardware structure of the electronic device according to the embodiment of the present invention. DETAILED DESCRIPTION

[0070] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0071] It should be understood that the terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0072] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable in appropriate circumstances, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0073] Figure 1 Schematic diagram of the system architecture used by the information processing method of the embodiment of the present invention; Figure 1 As shown, in order to improve the operating efficiency, safety and stability of the data center computer room, the system architecture used by the information processing method of this embodiment mainly includes:

[0074] 1) Power environment monitoring system

[0075] The power environment monitoring system is the core of the data center computer room monitoring, mainly for centralized monitoring and management of the power and environment in the computer room. The intelligent devices or subsystems monitored by the system include but are not limited to: power supply equipment (such as high-voltage power distribution, low-voltage power distribution, diesel generator sets, distribution cabinets, uninterruptible power supply (UPS), DC power supply system, batteries, etc.), air conditioning equipment (including precision air conditioning and ordinary air conditioning, through intelligent interfaces or modified circuits, to monitor their operating status, temperature and humidity parameters, and control their start and stop), environmental parameters (such as temperature and humidity, water leakage (through water leakage detection sensors), fresh air equipment, etc.). These monitoring data are collected and uploaded in real time to achieve 24-hour uninterrupted monitoring to ensure the stability and safety of the computer room environment.

[0076] 2) Security monitoring system

[0077] The security monitoring system is mainly used to ensure the safety of the computer room in the data center, including: video monitoring system (through high-definition cameras to conduct all-round monitoring of the computer room, support multi-screen display and video playback, to facilitate accident tracing and evidence preservation), access control system (to achieve access control, entry and exit information logging, security and anti-theft and alarm functions of the computer room), infrared monitoring and alarm system (through infrared detectors to monitor abnormal conditions in the computer room, such as human intrusion, etc., and trigger the alarm mechanism).

[0078] 3) Intelligent operation and maintenance inspection system

[0079] The intelligent operation and maintenance inspection system mainly uses robots, industrial cameras, infrared thermal imagers and other equipment to realize remote automatic and manual inspection of equipment in the computer room. The system can capture high-resolution images and intelligently identify equipment in the cabinet; use infrared thermal imagers to achieve mobile temperature measurement of equipment in the cabinet to ensure full coverage of temperature monitoring; automatically generate infrared thermal images, set warning intervals, and realize real-time temperature display of any point on the video screen.

[0080] 4) IoT Cloud Platform

[0081] The IoT cloud platform is one of the important means to realize intelligent monitoring of data center computer rooms. By connecting to the industrial IoT system, data collection and cloud configuration are realized, so as to remotely monitor real-time data and issue remote control commands. The platform can collect various parameter data of computer room equipment in real time for storage and analysis; it provides remote viewing and control functions, and managers can view the status of equipment at any time through mobile phones or computers and control its start or stop. When the equipment is abnormal or fails, the system automatically alarms and pushes the alarm information to the management personnel for timely processing.

[0082] It is understandable that the IoT cloud platform has a data analysis and intelligent alarm system, which can analyze and process the collected data in real time, generate various reports and charts, and provide data support for management decisions. By setting alarm thresholds and alarm rules, the intelligent alarm function is realized. When an abnormal situation is detected, the system can automatically trigger the alarm mechanism and notify the management personnel in various ways (such as sound, SMS, email, etc.).

[0083] Among them, the services based on the IoT cloud platform can push relevant information through three-dimensional panoramic virtual visualization. Specifically, by three-dimensional modeling of the computer room environment, layout, equipment, and pipelines, seamless integration with the monitoring platform can be achieved. This technology can provide an intuitive operation interface and enhance the user's operating experience. It realizes the data and status display and remote control functions of dynamic environment equipment (full name: power equipment environment monitoring equipment), security equipment, and intelligent inspection equipment.

[0084] It should be noted that the system architecture used in the technical solution of the embodiment of the present invention includes systems or devices not limited to Figure 1 As shown in , systems or devices in other data centers are also within the protection scope of the embodiments of the present invention.

[0085] An embodiment of the present invention provides an information processing method. Figure 2 FIG. 1 is a flow chart of an information processing method according to an embodiment of the present invention; Figure 2 As shown, the method includes:

[0086] Step 101: obtaining first data, and analyzing and processing the first data using a first model to obtain an analysis result; the first data includes at least one of the following: image data, text data, and voice data; the analysis result includes at least one task;

[0087] Step 102: calling the capability corresponding to each task to process the corresponding task and obtain the processing result;

[0088] Step 103: Obtain an execution result according to at least one processing result, and output first information, where the first information is information related to the execution result.

[0089] The information processing method of this embodiment is applied to a data center or a server belonging to the data center. In some optional embodiments, the data center may be located in the cloud, that is, the corresponding functions or services of the data center are provided by using the Internet of Things cloud platform, and various data of the equipment in the computer room can be collected through the interface for storage and analysis to provide corresponding services. In other optional embodiments, the data center may also be located on the side, and the data center or the server belonging to the data center can be called an edge server; under this deployment method, the data center can directly collect various data of the equipment in the computer room through the interface for storage and analysis, and provide corresponding services; or the Internet of Things cloud platform can be used to collect various data of the equipment in the computer room for storage, and the data center can then obtain the required data from the Internet of Things cloud platform for analysis and provide corresponding services.

[0090] In this embodiment, the data center obtains the first data, and then performs subsequent analysis and processing based on the first data to obtain corresponding execution results. Wherein, the first data includes at least one of the following: image data, text data, and voice data, and the first data can also be called multimodal data. The technical solution of the embodiment of the present invention utilizes the complementarity and fusion between different modal data, and integrates multimodal data for understanding and perception, thereby obtaining at least one task.

[0091] In some optional embodiments, obtaining the first data includes at least one of the following: obtaining image data and / or voice data collected by a camera in a computer room of a data center; obtaining image data and / or voice data collected by a camera set up by an inspection robot; obtaining image data and / or voice data collected by a terminal device; obtaining at least one of performance data, resource data, alarm data, and industrial parameter data during the operation of the equipment; and obtaining text data input by a user.

[0092] In this embodiment, the first data may come from a variety of sources. Exemplarily, the first data may be image data and / or voice data collected by a fixed camera in a computer room of a data center. Exemplarily, the first data may be image data and / or voice data collected by a camera of an inspection robot in a computer room. Exemplarily, the first data may be at least one of performance data, resource data, alarm data, and industrial parameter data during the operation of the equipment. Exemplarily, the first data may be image data and / or voice data collected by a terminal device, and the terminal device may be a mobile phone, a computer, a wearable device, etc.; wherein the wearable device may be a smart watch, a mixed reality (MR, Mixed Reality) device, an augmented reality (AR, Augmented Reality) glasses, etc. Exemplarily, the first data may be text data input by a user; in actual applications, a user may input text data through an input component (such as a keyboard, a touch screen, etc.) of a data center, or a user may input text data through a terminal (such as a mobile phone, a computer, etc.) and send the text data to the data center.

[0093] Exemplarily, if the first data obtained by the data center includes multiple types, or the first data obtained by the data center comes from multiple sources, then the multiple types of first data or the first data from multiple sources have correlation. As an example, the correlation may be a correlation in time, such as corresponding to multiple types of first data or first data from multiple sources at the same time or in the same time range. Taking the first data as an example, the text data input by the user and the image data (such as collected by the camera in the central computer room and / or collected by the camera of the inspection robot), the text data input by the user corresponds to the first time, then the image data may be the image data corresponding to the first time, or may be the image data within a time range before and after the first time, or may be the image data within the time period of the first time (for example, if the first time is within 8:00-9:00, then the time period may be 8:00-9:00). As another example, the correlation may also be a correlation in area, such as corresponding to multiple types of first data or first data from multiple sources within the same area, and the area may be the same computer room, the same community or the same area range, which is not limited in this embodiment.

[0094] Exemplarily, the data center may simultaneously obtain multiple types of first data or first data from multiple sources. For example, operation and maintenance personnel may package multiple types of first data or first data from multiple sources as first data for task analysis and processing.

[0095] Exemplarily, the data center can obtain multiple first data or first data from multiple sources. For example, the operation and maintenance personnel can input a first data, such as text data, and the data center can obtain other first data from related devices or systems based on the relevant information of the input text data (such as the time and / or area involved in the text data, etc.), such as obtaining image data associated with the relevant information of the text data from the stored image data, or obtaining at least one of the performance data, resource data, alarm data, and industrial parameter data of the device associated with the relevant information of the text data from the stored device-related data (such as at least one of performance data, resource data, alarm data, and industrial parameter data).

[0096] In other optional embodiments, the first data obtained by the data center may also be a type of first data, or first data from a single path. For example, the first data is image data collected by a camera of a patrol robot in a computer room, or image data collected by a fixed camera in a computer room of a data center, etc. The data center may also use the first model to analyze and process the first data to obtain an analysis result including at least one task.

[0097] Exemplarily, the data center may passively receive the first data, for example, an operation and maintenance personnel inputs the first data; or, the data center may actively obtain the first data, for example, the data center may actively obtain image data and / or audio data collected by a camera in the computer room, etc.

[0098] In this embodiment, the first model may also be referred to as a large model or a basic model, which refers to a machine learning model or a neural network model with large-scale parameters and a complex computing structure. The first model (or large model) may include a large visual model, a large language model, and a large multimodal model. In this embodiment, the multimodal large model is mainly aimed at, which can process a variety of different types of data, such as multimodal data such as text, image, audio, etc., and uses a multimodal attention mechanism, a multimodal feature fusion network, etc. to fuse the features of the visual model, the language model, the auditory model, and the communication model; after modeling the multimodal sequence data, the pre-trained parameters of the large visual model, the large language model, the auditory model, and the communication model are migrated to the multimodal large model using transfer learning technology, and the parameters of all models are trained on the multimodal large model to achieve end-to-end optimization of the multimodal large model. Thereby achieving tasks such as image segmentation, face recognition, image annotation, image analysis, voiceprint recognition, open question answering, and multimodal reasoning.

[0099] In this embodiment, the data center uses the first model to analyze and process the first data to obtain at least one task, and then processes the corresponding task by calling the capability corresponding to each task to obtain the processing result. Among them, the data center may have multiple capabilities, each of which can be realized by a corresponding computer program or model (recorded as a second model or a small model), and each capability can be called a "digital employee". The multiple capabilities (such as the functions of the second model or the small model) may include: target detection, target recognition, target tracking, target classification, face detection, alarm query, indicator query, event query, network planning, network construction, network maintenance, network optimization, supervision notification, service safety quality inspection, dress code quality inspection, equipment resource acceptance, equipment status identification, etc., and the above multiple capabilities are not limited in this embodiment. The data center obtains the processing result of each "digital employee" by calling the "digital employee" corresponding to each capability, and then integrates at least one processing result (such as natural language integration) to obtain the execution result.

[0100] In some optional embodiments of the present invention, the first data is analyzed and processed using the first model to obtain an analysis result; including: using the first model to identify at least one second information of the first data and extracting at least one third information from the first data; determining the third information corresponding to each second information, and obtaining an analysis result containing the corresponding second information and third information; wherein the second information represents the processing intention of the first data; and the third information includes task-related parameters.

[0101] In this embodiment, the first model is used to identify the second information and the third information of the first data; wherein the second information represents the processing intention of the first data, and the processing intention represents the intention or purpose of obtaining or inputting the first data. In this embodiment, the accuracy of intention recognition can be improved by obtaining multiple first data or first data from multiple sources. wherein the third information represents the task-related parameters contained in the first data. further, according to the processing intention and the task-related parameters, the processing task represented by the first data is disassembled, specifically, the processing intention is aligned with the task-related parameters, so as to obtain at least one task (or subtask). taking the first data as the text data input by the user "please query the top 10 base stations with alarm volume in M ​​places in the past x days, and send supervision text messages to the corresponding personnel according to the network element charter relationship" as an example, the text data is analyzed and processed by the first model, and the second information obtained may include: 1. alarm ranking; 2. charterer query; 3. text message notification...etc.; the third information obtained may include: 1. recent x days; 2. M places; 3. alarm volume; 4. Top 10; 5. base station...etc. Then, through the intention recognition and extraction of task-related parameters, the processing tasks can be broken down into: 1. Query the number of alarms; 2. Query charter information; 3. Send supervision text messages... etc. In other optional embodiments, if the first data is audio data, the audio data can be converted into text data through the first model, and then the text data can be analyzed and processed to obtain at least one second information and extract at least one third information. In other optional embodiments, if the first data is image data, the image data can be analyzed, identified and deeply understood through the first model to obtain multiple information representing the content of the image data, and then at least one second information can be obtained from the multiple information and at least one third information can be extracted.

[0102] In some optional embodiments, the method of using the first model to identify at least one second information of the first data and extracting at least one third information from the first data includes: using the first model to identify multiple first-category words in the first data that represent processing intentions, and extracting multiple second-category words that represent task-related parameters; respectively counting the frequency of occurrence of each word in the first category of words and the second category of words, and taking the first category words with a frequency higher than a first threshold as the at least one second information, and taking the second category words with a frequency higher than a second threshold as the at least one third information.

[0103] In this embodiment, the data center uses the first model to perform target detection or recognition, or deep understanding on the first data, and outputs structured information containing the content of the first data; wherein the structured information may include entity information, attribute information, and relationship information. The output structured information includes a plurality of first-category words representing processing intentions and a plurality of second-category words representing task-related parameters; then, by counting the word frequency, counting the number of occurrences of each word, removing irrelevant low-frequency words, retaining high-frequency words, taking the first-category words with a frequency higher than the first threshold as the at least one second information, and taking the second-category words with a frequency higher than the second threshold as the at least one third information.

[0104] In some optional embodiments, determining the third information corresponding to each second information includes: classifying the at least one second information and the at least one third information according to a preset classification rule, and determining the third information corresponding to each second information according to the classification result.

[0105] In this embodiment, for at least one second information representing a processing intention and at least one third information representing a task-related parameter, the information is classified using a predefined classification rule to better understand the content or structure of the first data and provide guidance for the disassembly of subsequent tasks. The at least one second information and the at least one third information are classified using a predefined classification rule, and the third information corresponding to each second information is determined based on the classification result, that is, the processing intention is aligned with the task-related parameter, and then the corresponding task is generated based on the aligned second information and third information.

[0106] In the embodiment of the present invention, the data center can set a variety of capabilities (or second models or small models) for different scenarios or tasks. The capabilities (or second models or small models) are insufficient in generalization due to their high specialization and efficiency for specific tasks. Relatively speaking, the first model (or large model) shows good versatility and adaptability in diversified scenarios due to its scale and complexity, but there is still room for improvement in in-depth professional knowledge and logical reasoning capabilities. The embodiment of the present invention uses the advanced parsing capabilities of the first model (or large model) to accurately identify user intentions and automatically schedule corresponding capabilities to provide targeted solutions. The first model (or large model) then integrates the output results of each capability for comprehensive analysis to enhance the accuracy and reliability of the service. Under this framework, the feedback generated by the capability when performing professional tasks will also be used to enrich the knowledge reserve of the first model (or large model) and improve its professional level in specific fields. This two-way knowledge exchange not only enhances the security and efficiency of the model, but also promotes knowledge optimization and sharing across model levels.

[0107] In some optional embodiments of the present invention, before analyzing and processing the first data using the first model, the method further includes: processing the first data using a third model to obtain initial target words; and processing the first data based on the initial target words.

[0108] Correspondingly, the analyzing and processing the first data using the first model includes: analyzing and processing the processed first data using the first model.

[0109] In this embodiment, before the data center uses the first model to analyze and process the first data, it can use the third model to pre-process the first data to obtain the initial target words. Exemplarily, the third model can be a high-precision target detection model based on the Transformer architecture, or it can be called a multimodal detector. The first data is preliminarily analyzed by the third model to identify the key entities or concepts in the first data, and then the initial target words are generated according to the vocabulary library inside the third model, and then the first data is preliminarily processed according to the initial target words; the processed first data is then input into the first model for analysis and processing.

[0110] In this embodiment, the third model mainly processes image data and text data. In some optional embodiments, when the first data includes image data and / or text data, the third model can be directly used to process the first data to obtain the initial target word. In other optional embodiments, when the first data also includes voice data, the voice data is converted into text data, and then the third model can be used to process the image data and / or text data.

[0111] Exemplarily, this embodiment mainly processes image data and text data simultaneously through the third model, analyzes and identifies the image data using text features, identifies key entities or concepts in the image data, and then generates initial target words based on the vocabulary library within the third model.

[0112] In some optional embodiments, when the first data includes image data, the processing of the first data based on the initial target word includes at least one of the following processing: image cropping, image rotation, image reduction, image enlargement, and color jittering.

[0113] In this embodiment, the image data can be processed using image segmentation technology according to the initialization target word, including at least one of the following processing: image cropping, image rotation, image reduction, image enlargement, color jitter, etc., to enhance the generalization ability of the model and ensure that the processed image data is more focused on the content represented by the initialization target word. Further, the cropped image data is input into the first model for processing.

[0114] In an embodiment of the present invention, the first model can be an open-set based Grounding-dino multimodal pre-training model, which deeply understands the input data through its internal mechanism and outputs structured information.

[0115] In some optional embodiments, the use of the third model to process the first data to obtain an initial target word includes: using the third model to perform the following operations on the first data: extracting features from the first data to obtain a first feature; the first data includes image data and / or text data; the first feature includes a first image feature and / or a first text feature; performing low-rank decomposition processing on the first feature to obtain a second feature; the second feature includes a second image feature and / or a second text feature; obtaining key information of the first data based on the second feature, and determining the initial target word corresponding to the key information based on a preset vocabulary library.

[0116] In this embodiment, the third model is used to extract features from the first data to obtain the first feature. Exemplarily, when the first data includes image data, a convolutional neural network can be used to extract features from the image data to obtain the first image feature; when the first data includes text data, a pre-trained word embedding model and a self-attention mechanism can be used to capture semantic features of different granularities, or Q-Former can be used to obtain the semantic features of the text data, that is, to obtain the first text feature. Further, the first feature is subjected to low-rank decomposition to obtain the second feature. Exemplarily, a method such as singular value decomposition (SVD, Singular Value Decomposition) can be used to perform low-rank decomposition on the first feature (including the first image feature and / or the first text feature), retain the main information component (that is, retain the larger singular value or principal component), remove noise and redundant information, and obtain the second feature. Further, the second image feature and the second text feature are used to obtain key information representing the image data and / or text data, and the key information is mapped with the vocabulary library preset in the third model to obtain the corresponding initial target word.

[0117] In some optional embodiments, when the second feature includes a second image feature and a second text feature, obtaining the key information of the first data based on the second feature includes: fusing the second image feature and the second text feature to obtain a fused feature, and obtaining the key information of the first data based on the fused feature.

[0118] In this embodiment, when the second feature includes a second image feature and a second text feature, the second image feature and the second text feature after low-rank decomposition are fused, and the image feature is fused with the text feature through a multimodal attention mechanism to enhance the representation of sentence semantics, ensure that the fusion process is as simple and efficient as possible, and reduce computational overhead.

[0119] Figure 3 FIG. 4 is a schematic diagram of a process flow of processing first data by using a third model in an information processing method according to an embodiment of the present invention; Figure 3 As shown, taking the first data including image data and text data as an example, the process of processing the image data and text data using the third model may include:

[0120] 1. Data preprocessing process: The data preprocessing process may mainly include standardization, processing missing values, etc., so as to obtain data that is more suitable for processing by the third model. Among them, the data preprocessing process preprocesses the image data and text data to obtain processed image data and text data.

[0121] 2. Encode the text data using the text encoding module and encode the image data using the image encoding module, perform feature extraction on the encoded text data and perform feature extraction on the encoded image data to obtain multi-scale image features.

[0122] Exemplarily, a convolutional neural network is used to extract image features from image data, and a Q-Former is used to obtain semantic features of text data, that is, to obtain text features.

[0123] 3. Perform low-rank decomposition on the extracted image features and text features, retain the main information components, and remove noise and redundant information.

[0124] For example, methods such as SVD may be used to perform low-rank decomposition on image features and text features.

[0125] 4. Fuse the image features and text features after low-rank decomposition, and obtain key information based on the fused features.

[0126] In this example, the DINO detection head can be used to fuse image features and text features. Specifically, the DINO detection method corresponds to a possible target category for each position on the feature map; in order to determine the specific position of the target, for example, a sliding window method is used to slide a window on the feature map, and the similarity between the feature vector in the window and the target category is calculated. By comparing the similarities of different windows, the position of the target is determined. Exemplarily, the image features after low-rank decomposition in this example can be one or more feature maps; each text feature is used as a target category through the DINO detection head, and the similarity between the feature vector at each position in the feature map and the text feature as the target category is calculated to determine the position of the target, that is, to obtain the key information of the image features and text features, and map the key information with the vocabulary in the preset vocabulary library to obtain the corresponding initial target word.

[0127] In some optional embodiments, additional auxiliary tasks may be introduced for the third model, such as key point detection, attribute recognition, etc. Figure 3 The auxiliary detection head in provides more supervision information. The auxiliary task shares the feature layer with the target task (such as feature extraction, low-rank processing, image-text fusion, etc.), and the overall performance of the third model is improved through joint training.

[0128] In some optional embodiments, for the third model, during the training phase, training and verification may be performed on multiple data sets to evaluate the detection accuracy and type matching accuracy of the model; the model structure and parameter configuration may be adjusted based on the verification results to further optimize the model performance.

[0129] In some optional embodiments, the outputting of the first information includes: directly outputting or displaying the execution result; and / or, querying a knowledge base according to the execution result to obtain fourth information related to the execution result, and outputting or displaying the fourth information, wherein the fourth information includes at least one of the following: fault information, root cause location information, and solution information.

[0130] In this embodiment, the data center processes the corresponding task by calling the capability corresponding to each task, or processes the corresponding task by calling the second model corresponding to the task, to obtain the processing result of each capability or the second model on the task; the data center integrates at least one processing result to obtain the execution result. As an execution method, the execution result can be the task execution result after natural language integration of at least one processing result, and the data center can directly output or display the task execution result. As another implementation method, the data center can query relevant information on the task execution result based on a pre-established knowledge base, provide targeted solutions, or provide fault information, or provide targeted root cause location information, etc., so as to output or display the query results of the knowledge base.

[0131] For example, the knowledge base can be a knowledge base constructed based on multiple types of instructions such as computer room operation and maintenance corpus and operation and maintenance cases, computer room monitoring and operation and maintenance, and the content of the knowledge base can be updated according to actual conditions. Then the data center can realize complex fault identification and root cause location based on the processing result of at least one task.

[0132] In some optional embodiments, the outputting of the first information includes: outputting and displaying information related to the execution result through a display component; and / or sending the first information to a terminal so that the terminal can output and display information related to the execution result.

[0133] In this embodiment, as an implementation method, the data center may have a display component, such as a large display screen, a touch screen, and the like. Then the data center may directly use its own display component to display information related to the execution result. As another implementation method, the data center may send a first information to a terminal based on a communication interface or a mobile communication network, so that the terminal can output and display information related to the execution result. Among them, the information related to the execution result may specifically refer to the processing result of the task using at least one capability or a second model, and the execution result obtained after integrating at least one processing result; or it may refer to at least one of the fault information, root cause location information, and solution information related to the execution result obtained by querying the knowledge base based on the execution result.

[0134] As an example, in the operation and maintenance scenario, taking the problem of operation and maintenance personnel inputting text as an example, the processing flow of the information processing method of the embodiment of the present invention can refer to Figure 4 As shown, it may specifically include:

[0135] 1. After the operation and maintenance personnel input the question, the data center uses the large model (i.e., the first model) to identify the content input by the user, recognize the user's intention, and extract the "entity" information.

[0136] Here, the user intention may be equivalent to the second information, and the "entity" information may be equivalent to the third information. Exemplarily, the "entity" information may be extracted using natural language processing (NLP).

[0137] 2. Align the “entity” information with the intent and generate at least one task.

[0138] 3. According to the intent, the corresponding capabilities (or the second model) are dispatched to process the corresponding tasks, and the "entity" information is used as a scheduling parameter to realize business needs.

[0139] 4. Obtain the task processing results of each capability (or the second model), use the large language model (LLM) to complete natural language integration, and obtain the execution results.

[0140] Optionally, the data center may feed back the execution results to the user.

[0141] 5. Query the knowledge base through the execution results of step 4 to obtain targeted solutions and provide feedback on the solutions to users.

[0142] In some optional embodiments of the present invention, calling the capability corresponding to each task to process the corresponding task includes: calling at least one second model to process the corresponding task, each second model having the capability corresponding to the processing task.

[0143] In this embodiment, the data center may have multiple capabilities, each task may correspond to one capability, or multiple tasks may correspond to one capability, or multiple tasks may correspond to one capability, and each capability may be implemented by a corresponding computer program or model. Such a model has a small number of parameters and may also be referred to as a small model or a second model. In this embodiment, after obtaining at least one task through the first model, the second model corresponding to each task is called to process the corresponding task using the second model.

[0144] Exemplarily, when the data center uses the first model to analyze and process the first data to obtain at least one task, the data center generates at least one task based on the identified processing intent (i.e., the second information) and the extracted task-related parameters (i.e., the third information). Furthermore, the data center can call the corresponding second model based on at least one second information (i.e., the processing intent) to process the corresponding task based on the corresponding second model.

[0145] In some optional embodiments, the method further includes: generating training data according to the analysis result, and training the second model using the training data and the first data.

[0146] In this embodiment, the data center generates training data based on the analysis results of the first data by the first model (ie, at least one task included in the analysis results), and uses the training data and the first data to train the corresponding second model.

[0147] Figure 5 Schematic diagram of the process of training the second model in the information processing method of an embodiment of the present invention; Figure 5 As shown, it may specifically include:

[0148] 1. Obtain input data, process the input data using the third model to obtain an initial target word; and process the first data based on the initial target word.

[0149] Here, the processing is mainly performed on the input data that is image data, and at least one of image cropping, image rotation, image reduction, image enlargement and color jittering is performed on the image data based on the initial target word to enhance the generalization ability of the model and ensure that the cropped image area is more focused on the content represented by the target word.

[0150] It should be noted that if the input data is not image data, this step is not performed.

[0151] 2. Use the first model to process the data and output structured information.

[0152] Here, the first model can be used to identify the intent information (equivalent to the second information) and the "entity" information (equivalent to the entity information) of the data, and multiple structured information can be obtained based on the intent information and the entity information. Among them, the structured information may include entities, attributes, relationships, etc. For example, refer to Figure 6 As shown, the first data is image data, which is an image collected by a fixed camera in the computer room. In addition to the deployed computer room, a person appears in the image. The first model is used to analyze and identify the image data, and the unstructured result can be: the video describes the scene in the computer room, where there are two rows of black cabinets that are normally closed, and a woman wearing a blue safety helmet, a blue reflective vest, white pants, and black shoe covers walks in the middle of the cabinets towards the key equipment in the distance with a mobile phone, that is, the unstructured recognition result includes the description information of the image data. Through the recognition of the intent of the image data by the first model, the output structured results may include: alarm, illegal intrusion of personnel, and regional location: computer building.

[0153] 3. Perform word frequency analysis on the structured information. Specifically, perform word frequency statistics on the structured information output by the first model to determine the number of times each word appears, remove irrelevant or low-frequency words, and retain high-frequency, meaningful words to reduce the impact of noise on subsequent steps.

[0154] 4. Classify the retained high-frequency words according to the predefined classification rules. The purpose of classification is to better understand the structure of the image content.

[0155] 5. Based on the classification results of high-frequency words, select a suitable template according to different application scenarios or task requirements, fill the high-frequency words into the template, and generate at least one task by filling in the words and classification information; the at least one task is used as a prompt for training the second model.

[0156] It can be understood that the output of the first model is at least one task.

[0157] In this embodiment, the output result of the first model is used as a supervisory signal to guide the training of the second model. During the training process, transfer learning, incremental learning and other technologies are used to make full use of the prior knowledge of the large model to accelerate the convergence process of the small model; and the second model is automatically trained, tested and deployed, and the self-iteration and continuous update of the second model can be completed without human participation; according to the feedback of new data, the parameters of the small model are continuously updated to improve its generalization and robustness, reducing the complexity and cost of data center operations, and providing an efficient, intelligent and energy-saving solution for data center computer room monitoring.

[0158] Based on the above embodiments, an embodiment of the present invention further provides an information processing method. Figure 7 FIG. 1 is another flow chart of the information processing method according to an embodiment of the present invention; Figure 7 As shown, the method includes:

[0159] Step 201: receiving first information sent by a data center, wherein the first information is information related to an execution result obtained by the data center invoking a capability corresponding to each task in at least one task to process the corresponding task and integrating at least one processing result, wherein the at least one task is obtained by the data center analyzing and processing first data using a first model, and the first data includes at least one of the following: image data, text data, and voice data;

[0160] Step 202: Output and display information related to the execution result.

[0161] In this embodiment, the terminal has a display component, such as a display screen, a touch screen, etc. After receiving the first information sent by the data center, the terminal can display the first information through the display component. Exemplarily, the terminal can be a mobile phone, a computer, a wearable device (such as a smart watch, smart glasses, etc.), etc.

[0162] In some optional embodiments, the output and display of information related to the execution result includes: outputting first image data and / or first audio data, wherein the first image data and / or the first audio data correspond to the information related to the execution result, and the first image data is used to superimpose a virtual picture corresponding to the first image data in a real scene.

[0163] In this embodiment, the terminal may be an AR device, such as AR glasses. The display component may be used to superimpose a virtual image on a real scene. Figure 8 shown.

[0164] In some optional embodiments, the virtual screen may have operation information, and the operation information is used for the user to provide operation suggestions through gesture operations to facilitate remote operation.

[0165] The data center intelligent monitoring agent (i.e., the first model) of the embodiment of the present invention can effectively reflect the full equipment status and network status of the data center in the digital world. Through the agent (i.e., the first model), it can identify user intentions, task scheduling decomposition, extract time, location, model scheduling, and other tasks. Through model experiments and analysis and prediction in the digital world, it provides a decision-making basis for the further optimization of the data center network troubleshooting and formulation system, and integrates the training of multimodal large models to achieve visual presentation of abnormal information. The large model is used to analyze the audio, image, and video data of the data center, monitor equipment status, abnormal conditions, etc., and update the data to the platform in real time to keep the data center status synchronized. The agent screens the data analyzed by the multimodal model, and uses MR glasses, mobile phones, large screens, voice information, etc. to directly present the analysis results to the on-site operation and maintenance personnel.

[0166] The first model is the brain of the data center. By understanding the user's intention, it uses the language model, visual model, auditory model, and capability (or second model) for multimodal interaction, orchestrates the relevant capabilities that meet the data center operation scenarios, and calls various applications at the application layer to form the following applications:

[0167] 1. Based on massive network resources, alarms, performance and other network operation data combined with the communication knowledge question-and-answer database, the system can answer the service failure problems of the equipment room encountered by users, provide the best handling suggestions for network failures for operation and maintenance personnel when handling failures, and display the fault information of the equipment in real time in MR glasses in combination with AR technology. In actual operation, the virtual projection of the equipment can be seen to guide the operation and maintenance personnel to handle the failure. The interactive function of MR glasses can also be used to interact with the network management system in real time to query equipment information and record maintenance logs.

[0168] 2. Based on the data center resource data, the image data collected by on-site robots, dynamic environment cameras, mobile phone cameras, etc. are combined with the network management system data to establish equipment status classification models, anomaly detection models, resource allocation models, etc., and then combined with MR technology to build a virtual data center model. Combined with intelligent assistants, it is displayed to data center managers through MR glasses, mobile phones, etc. to achieve consistency of front-end and back-end resources. It can also display comprehensive information such as computer room space, power resources, network resources, etc. in a panoramic, visual, and 3D manner.

[0169] 3. Integrate the image data collected by on-site robots, dynamic environment cameras, mobile phone cameras, etc. and the data of the business work order system as the input of the visual big model. At the same time, use sensors and other equipment to collect real-time operation data of the equipment in the computer room, and transfer the multimodal data to the big model for analysis, monitor abnormal conditions in the computer room, and generate alarms such as infrastructure damage, personnel intrusion, equipment alarms, items left behind, and electronic fences. The abnormal alarm information detected by the visual big model can be sent to the monitoring big screen, fault center, and the operation and maintenance personnel's mobile phone in real time, reflecting the actual operation of the computer room in real time.

[0170] Based on the above embodiments, an embodiment of the present invention further provides an information processing device, which is applied to a data center. Fig. 9 The structure diagram of the information processing device according to the embodiment of the present invention is shown in FIG. Figure 1 ;like Fig. 9 As shown, the device comprises: a data acquisition unit 11, a processing unit 12 and a first output unit 13; wherein,

[0171] The data acquisition unit 11 is used to obtain first data, wherein the first data includes at least one of the following: image data, text data, and voice data;

[0172] The processing unit 12 is used to analyze and process the first data using the first model to obtain an analysis result; the analysis result includes at least one task; and is also used to call the capability corresponding to each task to process the corresponding task to obtain a processing result; and obtain an execution result according to at least one processing result;

[0173] The first output unit 13 is used to output first information, where the first information is information related to the execution result.

[0174] In some optional embodiments of the present invention, the processing unit 12 is used to call at least one second model to process the corresponding task, and each second model has the ability to process the corresponding task.

[0175] In some optional embodiments of the present invention, the processing unit 12 is used to use a first model to identify at least one second information of the first data and extract at least one third information from the first data; determine the third information corresponding to each second information, and obtain an analysis result containing the corresponding second information and third information; wherein the second information represents the processing intention of the first data; and the third information includes task-related parameters.

[0176] In some optional embodiments of the present invention, the processing unit 12 is used to process the first data using a third model to obtain initial target words before analyzing and processing the first data using the first model; and process the first data based on the initial target words.

[0177] In some optional embodiments of the present invention, the processing unit 12 is used to analyze and process the processed first data using a first model.

[0178] In some optional embodiments of the present invention, the processing unit 12 is used to perform the following operations on the first data using the third model: perform feature extraction on the first data to obtain a first feature; the first data includes image data and / or text data; the first feature includes a first image feature and / or a first text feature; perform low-rank decomposition processing on the first feature to obtain a second feature; the second feature includes a second image feature and / or a second text feature; obtain key information of the first data based on the second feature, and determine an initial target word corresponding to the key information based on a preset vocabulary library.

[0179] In some optional embodiments of the present invention, the processing unit 12 is used to fuse the second image feature and the second text feature to obtain a fused feature when the second feature includes a second image feature and a second text feature, and obtain key information of the first data based on the fused feature.

[0180] In some optional embodiments of the present invention, the processing unit 12 is used to process the first data based on the initial target word by at least one of the following methods: image cropping, image rotation, image reduction, image enlargement, and color jittering when the first data includes image data.

[0181] In some optional embodiments of the present invention, the processing unit 12 is used to use the first model to identify multiple first-category words in the first data that represent processing intentions, and to extract multiple second-category words that represent task-related parameters; respectively count the frequency of occurrence of each word in the first category of words and the second category of words, and use the first category words with a frequency higher than a first threshold as the at least one second information, and use the second category words with a frequency higher than a second threshold as the at least one third information.

[0182] In some optional embodiments of the present invention, the processing unit 12 is used to classify the at least one second information and the at least one third information according to a preset classification rule, and determine the third information corresponding to each second information according to the classification result.

[0183] In some optional embodiments of the present invention, the processing unit 12 is further used to generate training data according to the analysis result, and train the second model using the training data and the first data.

[0184] In some optional embodiments of the present invention, the first output unit 13 is used to directly output or display the execution result; and / or, query the knowledge base according to the execution result to obtain fourth information related to the execution result, output or display the fourth information, and the fourth information includes at least one of the following: fault information, root cause location information, and solution information.

[0185] In some optional embodiments of the present invention, the data acquisition unit 11 is used to obtain the first data in at least one of the following ways: obtaining image data and / or voice data collected by a camera in a computer room of a data center; obtaining image data and / or voice data collected by a camera set up by a patrol robot; obtaining image data and / or voice data collected by a terminal device; obtaining at least one of performance data, resource data, alarm data, and industrial parameter data during the operation of the equipment; and obtaining text data input by a user.

[0186] In some optional embodiments of the present invention, the first output unit 13 is used to output and display information related to the execution result through a display component; and / or send first information to a terminal so that the terminal can output and display information related to the execution result.

[0187] In an embodiment of the present invention, the processing unit 12 in the device can be implemented by a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU) or a programmable gate array (FPGA) in actual applications; the data acquisition unit 11 in the device can be implemented by a communication module (including: basic communication kit, operating system, communication module, standardized interface and protocol, etc.) and a transceiver antenna in actual applications; the first output unit 13 in the device can be implemented by a multimedia component in actual applications. For example, the multimedia component may include a display screen and / or a microphone.

[0188] An embodiment of the present invention further provides an information processing device, which is applied to a terminal. Fig.10 The structure diagram of the information processing device according to the embodiment of the present invention is shown in FIG. Figure 2 ;like Fig.10As shown, the device comprises: a communication unit 21 and a second output unit 22; wherein,

[0189] The communication unit 21 is used to receive first information sent by the data center, the first information is information related to an execution result obtained by the data center invoking a capability corresponding to each task in at least one task to process the corresponding task and integrating at least one processing result, the at least one task is obtained by the data center analyzing and processing first data using a first model, and the first data includes at least one of the following: image data, text data, and voice data;

[0190] The second output unit 22 is used to output and display information related to the execution result.

[0191] In some optional embodiments of the present invention, the second output unit 22 is used to output first image data and / or first audio data, wherein the first image data and / or the first audio data correspond to information related to the execution result, and the first image data is used to superimpose a virtual picture corresponding to the first image data in a real scene.

[0192] In an embodiment of the present invention, the communication unit 21 in the device can be implemented in actual applications through a communication module (including: a basic communication kit, an operating system, a communication module, a standardized interface and protocol, etc.) and a transceiver antenna; the second output unit 22 in the device can be implemented in actual applications by a multimedia component. Exemplarily, the multimedia component may include a display screen and / or a microphone.

[0193] It should be noted that: when the information processing device provided in the above embodiment performs information processing, only the division of the above program modules is used as an example. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the information processing device provided in the above embodiment and the information processing method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0194] An embodiment of the present invention further provides an electronic device, which is a data center or a terminal. Fig.11 FIG. 1 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Fig.11 As shown, the electronic device includes a memory 32, a processor 31, and a computer program stored in the memory 32 and executable on the processor 31. When the processor 31 executes the program, the steps of the information processing method applied to a data center or a terminal according to an embodiment of the present invention are implemented.

[0195] Optionally, the electronic device may further include at least one network interface 33. The components in the electronic device are coupled together via a bus system 34. It is understood that the bus system 34 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 34 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig.11 Various buses are labeled as bus system 34 .

[0196] It can be understood that the memory 32 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 32 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0197] The method disclosed in the above embodiment of the present invention can be applied to the processor 31, or implemented by the processor 31. The processor 31 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 31 or the instruction in the form of software. The above processor 31 can be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 31 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 32. The processor 31 reads the information in the memory 32 and completes the steps of the above method in combination with its hardware.

[0198] In an exemplary embodiment, the electronic device may be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general purpose processors, controllers, MCUs, microprocessors, or other electronic components to execute the aforementioned method.

[0199] In an exemplary embodiment, the present invention also provides a computer-readable storage medium, such as a memory 32 including a computer program, and the computer program can be executed by a processor 31 of an electronic device to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM; or it can be various devices including one or any combination of the above memories.

[0200] The computer-readable storage medium provided by the embodiment of the present invention stores a computer program, which, when executed by a processor, implements the steps of the information processing method of the embodiment of the present invention.

[0201] The embodiment of the present application also provides a computer program product, including a computer program, which can be executed by an electronic device (such as the processor 31 of the electronic device) to complete the steps of any of the aforementioned information processing methods.

[0202] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0203] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0204] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0205] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0206] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0207] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0208] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.

[0209] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0210] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An information processing method, characterized in that: The method is applied to a data center, and the method comprises: Obtaining first data, and analyzing and processing the first data using a first model to obtain an analysis result; the first data includes at least one of the following: image data, text data, and voice data; the analysis result includes at least one task; Call the capability corresponding to each task to process the corresponding task and obtain the processing result; An execution result is obtained according to at least one processing result, and first information is output, where the first information is information related to the execution result.

2. The method according to claim 1, characterized in that The calling of the capability corresponding to each task to process the corresponding task includes: At least one second model is called to process the corresponding task, and each second model has the capability to process the corresponding task.

3. The method according to claim 1, characterized in that The analyzing and processing the first data by using the first model to obtain an analysis result comprises: Use the first model to identify at least one second information of the first data and extract at least one third information from the first data; determine the third information corresponding to each second information, and obtain an analysis result containing the corresponding second information and third information; wherein the second information represents the processing intention of the first data; and the third information includes task-related parameters.

4. The method according to claim 1, characterized in that: Before analyzing and processing the first data using the first model, the method further includes: Processing the first data using a third model to obtain an initial target word; The first data is processed based on the initial target word.

5. The method according to claim 4, characterized in that The analyzing and processing the first data by using the first model includes: The first data after being processed is analyzed and processed using a first model.

6. The method according to claim 4, characterized in that The using the third model to process the first data to obtain an initial target word includes: The third model is used to perform the following operations on the first data: Performing feature extraction on the first data to obtain a first feature; the first data includes image data and / or text data; the first feature includes a first image feature and / or a first text feature; Performing low-rank decomposition processing on the first feature to obtain a second feature; the second feature includes a second image feature and / or a second text feature; Key information of the first data is obtained based on the second feature, and an initial target word corresponding to the key information is determined based on a preset vocabulary library.

7. The method according to claim 6, characterized in that In the case where the second feature includes a second image feature and a second text feature, obtaining key information of the first data based on the second feature includes: The second image feature and the second text feature are fused to obtain a fused feature, and key information of the first data is obtained based on the fused feature.

8. The method according to claim 4, characterized in that In the case where the first data includes image data, the processing of the first data based on the initial target word includes at least one of the following processing: Image cropping, image rotation, image reduction, image enlargement, color dithering.

9. The method according to claim 3, characterized in that: The step of using the first model to identify at least one second information of the first data and extracting at least one third information from the first data includes: Using the first model, identifying a plurality of first-category words in the first data that represent processing intentions, and extracting a plurality of second-category words that represent task-related parameters; The frequency of occurrence of each word in the first category of words and the second category of words is counted respectively, and the first category of words with a frequency higher than a first threshold is used as the at least one second information, and the second category of words with a frequency higher than a second threshold is used as the at least one third information.

10. The method according to claim 3, characterized in that: The determining of the third information corresponding to each second information includes: The at least one second information and the at least one third information are classified according to a preset classification rule, and the third information corresponding to each second information is determined according to the classification result.

11. The method according to any one of claims 2 to 10, characterized in that: The method further comprises: Generate training data according to the analysis result, and use the training data and the first data to train the second model.

12. The method according to claim 1, characterized in that The outputting of the first information includes: directly outputting or displaying the execution result; and / or, A knowledge base is queried according to the execution result to obtain fourth information related to the execution result, and the fourth information is output or displayed, where the fourth information includes at least one of the following: fault information, root cause location information, and solution information.

13. The method according to claim 1, characterized in that The obtaining of the first data includes at least one of the following: Obtain image data and / or voice data collected by cameras in the data center computer room; Obtain image data and / or voice data collected by a camera set up by the inspection robot; Obtaining image data and / or voice data collected by a terminal device; Obtain at least one of performance data, resource data, alarm data, and industrial parameter data during equipment operation; Get the text data entered by the user.

14. The method according to claim 1, characterized in that The outputting of the first information includes: Outputting and displaying information related to the execution result through a display component; and / or, The first information is sent to the terminal, so that the terminal can output and display information related to the execution result.

15. An information processing method, characterized in that: The method is applied to a terminal, and the method includes: receiving first information sent by a data center, wherein the first information is information related to an execution result obtained by the data center invoking a capability corresponding to each task in at least one task to process the corresponding task and integrating at least one processing result, wherein the at least one task is obtained by the data center analyzing and processing first data using a first model, and the first data includes at least one of the following: image data, text data, and voice data; Output and display information related to the execution result.

16. The method according to claim 15, characterized in that The output and display of information related to the execution result include: Output first image data and / or first audio data, wherein the first image data and / or the first audio data correspond to information related to the execution result, and the first image data is used to overlay and display a virtual screen corresponding to the first image data in a real scene.

17. An information processing device, characterized in that: The device is applied to a data center, and comprises: a data acquisition unit, a processing unit and a first output unit; wherein, The data acquisition unit is used to obtain first data, wherein the first data includes at least one of the following: image data, text data, and voice data; The processing unit is used to analyze and process the first data using the first model to obtain an analysis result; the analysis result includes at least one task; and is also used to call the capability corresponding to each task to process the corresponding task to obtain a processing result; and obtain an execution result according to at least one processing result; The first output unit is used to output first information, where the first information is information related to the execution result.

18. An information processing device, characterized in that: The device is applied to a terminal, and comprises: a communication unit and a second output unit; wherein, The communication unit is used to receive first information sent by a data center, the first information being information related to an execution result obtained by the data center invoking a capability corresponding to each task in at least one task to process the corresponding task and integrating at least one processing result, the at least one task being obtained by the data center analyzing and processing first data using a first model, the first data comprising at least one of the following: image data, text data, and voice data; The second output unit is used to output and display information related to the execution result.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 14 are implemented; or, when the program is executed by a processor, the steps of the method described in claim 15 or 16 are implemented.

20. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method described in any one of claims 1 to 14 are implemented; or, when the processor executes the program, the steps of the method described in claim 15 or 16 are implemented.

21. A computer program product, characterized in that The method comprises computer program instructions, which enable a computer to execute the steps of the method according to any one of claims 1 to 14; or, the computer program instructions enable a computer to execute the steps of the method according to claim 15 or 16.

Images