Data processing method and device, equipment, storage medium and program product

Abstract

The application provides a data processing method, device, equipment, medium and program product. The method comprises: obtaining preparation data of multiple modalities in the process of preparing a target material; for the preparation data of each modality, performing feature extraction on the preparation data by an agent corresponding to the modality to obtain data features of the preparation data, wherein the data features of the preparation data contain input material information of the target material; simulating generation of output material information based on the data features corresponding to the preparation data of the multiple modalities; generating an initial adjustment instruction based on the output material information and the input material information, wherein the initial adjustment instruction is used to adjust a preparation parameter after a target time in the process of preparing the target material; and determining a target adjustment instruction for adjusting the preparation parameter in the preparation process based on the data features corresponding to the preparation data of the multiple modalities and the initial adjustment instruction. In this way, the production efficiency, the preparation parameter approximation degree and the quality stability of the prepared material can be significantly improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] In the preparation of existing materials (such as non-ferrous metals), due to the complexity of material composition and the difficulty of process monitoring, traditional control systems typically rely on only single process time-series data, failing to effectively integrate multimodal production data. This results in existing systems being unable to fully cover complex operating conditions requiring manual intervention, exhibiting poor online performance, and failing to form an effective closed-loop control solution. Consequently, this leads to large fluctuations in intermediate product quality, frequent occurrences of abnormal operating conditions, and severely impacts production efficiency. Therefore, related technologies suffer from poor preparation efficiency and accuracy in preparing target materials. Summary of the Invention

[0003] This application provides a data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can significantly improve production efficiency, preparation parameter approximation, and the quality stability of the prepared materials.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a data processing method, including:

[0006] During the preparation of the target material, preparation data of multiple modes are acquired, and the preparation data is used to characterize the preparation state of the target material at the target time.

[0007] For the preparation data of each modality, the preparation data is feature extracted by the intelligent agent corresponding to the modality to obtain the data features of the preparation data, and the data features of the preparation data include the input material information of the target material;

[0008] Based on the data characteristics corresponding to the preparation data of the various modalities, output material information is simulated and generated;

[0009] Based on the output material information and the input material information, an initial adjustment command is generated, which is used to adjust the preparation parameters after the target time in the preparation process of the target material;

[0010] Based on the data characteristics corresponding to the preparation data of the multiple modalities and the initial adjustment instructions, a target adjustment instruction for adjusting the preparation parameters during the preparation process is determined.

[0011] This application provides a data processing apparatus, including:

[0012] The acquisition module is used to acquire preparation data of multiple modes during the preparation of the target material, and the preparation data is used to characterize the preparation state of the target material at the target time.

[0013] The feature extraction module is used to extract features from the preparation data for each modality by using an agent corresponding to the modality to obtain the data features of the preparation data. The data features of the preparation data include the input material information of the target material.

[0014] The first generation module is used to simulate and generate output material information based on the data characteristics corresponding to the preparation data of the multiple modes;

[0015] The second generation module is used to generate an initial adjustment instruction based on the output material information and the input material information. The initial adjustment instruction is used to adjust the preparation parameters after the target time in the preparation process of the target material.

[0016] The determination module is used to determine a target adjustment instruction for adjusting the preparation parameters during the preparation process based on the data characteristics corresponding to the preparation data of the multiple modalities and the initial adjustment instruction.

[0017] This application provides an electronic device, including:

[0018] Memory is used to store executable instructions or computer programs.

[0019] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data processing method provided in the embodiments of this application.

[0020] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed by a processor, implement the data processing method provided in this application.

[0021] This application provides a computer program product, which includes computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the data processing method provided in this application.

[0022] The embodiments of this application have the following beneficial effects:

[0023] By acquiring preparation data from multiple modalities and extracting features from the preparation data using agents corresponding to each modality, this approach breaks through the limitations of traditional systems that rely solely on single-time-series data. This method, utilizing proprietary agents to extract features from multimodal data, effectively addresses the challenges of complex material composition and monitoring difficulties, achieving comprehensive perception of complex preparation conditions. Furthermore, compared to fixed or single control methods, this application first simulates and generates output material information based on the data features corresponding to the preparation data from multiple modalities. Then, it combines the input and output information to generate initial adjustment commands. Finally, it again combines the global multimodal preparation data... The initial instructions are corrected or confirmed based on the corresponding data characteristics, and the target adjustment instructions are finally determined. This progressive instruction generation process constructs an effective and accurate closed-loop adjustment mechanism. In addition, through the dynamic simulation and instruction adjustment based on multimodal characteristics, precise intervention can be made on the preparation parameters after the target time. This not only covers the complex working conditions of traditional manual intervention, but also effectively suppresses the quality fluctuations of intermediate products. Thus, it overcomes the engineering obstacles of low preparation efficiency and large result deviation in related technologies, and achieves the technical effect of significantly improving overall production efficiency, preparation parameter approximation, and the quality stability of the prepared materials. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the architecture of the data processing system 100 provided in an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0026] Figure 3 This is a flowchart illustrating the data processing method provided in an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of the multimodal fabrication data provided in the embodiments of this application;

[0028] Figure 5 This is a schematic diagram of multiple storage modules provided in the embodiments of this application.

[0029] Explanation of reference numerals in the attached figures:

[0030] Figure 1 100 - Data processing system; 200 - Server; 300 - Network; 400 - Terminal;

[0031] Figure 220 - Electronic device; 410 - Processor; 420 - Network interface; 430 - User interface; 440 - Bus system; 450 - Memory; 431 - Output device; 432 - Input device; 451 - Operating system; 452 - Network communication module; 453 - Presentation module; 454 - Input processing module; 455 - Data processing device; 4551 - Acquisition module; 4552 - Feature extraction module; 4553 - First generation module; 4554 - Second generation module; 4555 - Determination module;

[0032] Figure 4 40-Data Acquisition Module; 401-Handwritten Data; 402-Image Data; 403-Time Series Data; 404-Relational Data; 405-Document Data; 406-Voice Data;

[0033] Figure 5 50 - Distributed heterogeneous storage architecture; 501 - Relational storage module; 502 - Document storage module; 503 - Key-value pair storage module; 504 - Time-series storage module. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0036] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0038] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0039] 1) Client, also known as user terminal, refers to the program that provides local services to users in contrast to the server. Except for some applications that can only run locally, it is generally installed on the terminal and needs to work with the server. That is, there needs to be a corresponding server and service program on the network to provide the corresponding services. Thus, a specific communication connection needs to be established between the client and the server to ensure the normal operation of the application.

[0040] 2) An Application Programming Interface (API) is a set of predefined functions or conventions for connecting different components of a software system, used to enable communication and data exchange between two separate software systems.

[0041] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the data processing system 100 provided in the embodiments of this application. The terminal (terminal 400 is shown as an example) is connected to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two, and data transmission is achieved using wireless or wired links.

[0042] In some embodiments, server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal 400 can be a smartphone, tablet, laptop, desktop computer, set-top box, smart voice interaction device, smart home appliance, virtual reality device, vehicle terminal, aircraft, portable music player, personal digital assistant, dedicated messaging device, portable gaming device, smart speaker, and smartwatch, but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0043] The electronic device implementing the data processing method provided in the embodiments of this application will now be described. See also Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 20 can be a server or a terminal. Taking the electronic device 20 as an example... Figure 1 Taking server 200 as an example, Figure 2The illustrated electronic device 20 includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 440.

[0044] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0045] User interface 430 includes one or more output devices 431 that enable the display of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0046] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0047] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0048] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0049] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0050] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0051] Presentation module 453 is configured to enable the display of information (e.g., user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with user interface 430.

[0052] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0053] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A data processing device 455 stored in memory 450 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: an acquisition module 4551, a feature extraction module 4552, a first generation module 4553, a second generation module 4554, and a determination module 4555. These modules are logically linked and can therefore be arbitrarily combined or further divided according to their implemented functions. The functions of each module will be described below.

[0054] In other embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the data processing apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the data processing method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0055] In some embodiments, the terminal or server can implement the data processing method or data processing mechanism provided in this application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a local client, i.e., a program that needs to be installed in the operating system to run, such as a web-based APP; it can also be a mini-program, i.e., a program that only needs to be downloaded to a browser environment to run; or it can be a mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of client, module, or plugin.

[0056] Based on the foregoing description of the data processing system and electronic device provided in the embodiments of this application, the data processing method provided in the embodiments of this application will be described below. In actual implementation, the data processing method provided in the embodiments of this application can be implemented by a terminal or a server alone, or by a terminal and a server working together, so that... Figure 1 The following description uses the example of server 200 executing the data processing method provided in this application embodiment alone. See also... Figure 3 , Figure 3 This is a schematic flowchart of the data processing method provided in the embodiments of this application. Next, we will discuss... Figure 3 The steps shown are explained.

[0057] Step S101: During the preparation of the target material, preparation data of multiple modes are acquired. The preparation data is used to characterize the preparation state of the target material at the target time.

[0058] It should be noted that the target materials encompass various basic or composite materials involved in material conversion and energy interaction processes in industrial production. Specifically, the target materials include, but are not limited to, metallic materials in the metallurgical field (e.g., iron, copper, lead, zinc and their alloys), inorganic non-metallic materials (e.g., glass), and chemical materials in the chemical industry. This application does not limit the specific chemical composition or final physical form of the target materials. Any material exhibiting continuous, semi-continuous, or batch preparation characteristics, and whose preparation process is constrained by the laws of material and energy balance, falls within the scope of application and protection of this application.

[0059] In practical implementation, multimodal preparation data refers to multi-source information with differentiated physical meanings and data structures acquired through different types of acquisition terminals during the material preparation process. Specifically, such as... Figure 4 As shown, Figure 4 This is a schematic diagram of the multimodal fabrication data provided in the embodiments of this application, based on Figure 4The process of acquiring preparation data of multiple modalities during the preparation of the target material can be as follows: during the preparation of the target material, at least two of the following preparation data are acquired through the data acquisition module 40: handwritten data 401, image data 402, time series data 403, relational data 404, document data 405, and voice data 406.

[0060] The data includes handwritten records by material preparation personnel regarding the preparation status of the target material during the preparation process; image data obtained by capturing images of the preparation status of the target material at the target time; time-series data including parameter values ​​of state parameters indicating the preparation status at different times; relational data including environmental state data associated with the preparation process and component data in the preparation results at the target time; document data used to record the constraints of the preparation process and the topological relationships between multiple devices used to prepare the target material; and voice data consisting of audio recordings of the material preparation personnel describing the preparation status.

[0061] It should be noted that the handwritten data includes handwritten records by material preparation personnel regarding the preparation status of the target material during the preparation process. Examples include operation records, material change records, and shift records filled out by on-site material preparation personnel, as well as handwritten chemical symbols, temperature indicator arrows, superscripts, or subscripts contained in the records. This application does not limit the scope of the handwritten data. Furthermore, the handwritten data can be collected using an external high-speed scanner or a mobile terminal camera. For example, the handwritten record data can be obtained by scanning and imaging the forms filled out by material preparation personnel regarding the preparation status using a high-resolution scanning device or a mobile terminal. This application also does not limit the scope of the handwritten data.

[0062] Image data is obtained by acquiring images of the preparation state of the target material at the target time. Examples include probe lifting images representing the melt level, 3D point cloud reconstruction images representing the slag bag under full load, images of oxygen-enriched blowing (such as underblowing / overblowing / normal state), and abnormal working conditions (such as flue gas, jamming, splashing) captured in real time by a high-temperature furnace camera. In this application embodiment, the specific representation content and acquisition object of the image data are not limited. At the same time, the image data can be acquired by analyzing video streams or capturing images of the preparation state at the target time through industrial camera equipment deployed on site. For example, it can be obtained by analyzing video streams captured by on-site cameras or image files directly uploaded by staff.

[0063] Time-series data is used to indicate state parameters that change continuously over time during the preparation process, including the parameter values ​​of the state parameters at different times, such as real-time changes in furnace temperature, pressure, and gas flow rate. This application does not limit the specific parameters in this embodiment. Meanwhile, the time-series data can be continuously received through communication with the underlying industrial control network.

[0064] The relational data is obtained by calling the application network interface to obtain the component data in the laboratory test report and the environmental status data of the associated sensors, such as fluorescence spectrometer data, X-ray fluorescence spectrometer measurement data, component analysis results on the laboratory test report for the target material being prepared, discrete temperature and humidity records of the preparation environment, etc. This application embodiment does not limit this.

[0065] The document data is used to record the constraints of the preparation process and the topological relationships between multiple devices used to prepare the target material. The constraints may be, for example, engineering drawings that record the preparation formula and tables containing existing metallurgical calculation formulas, while the topological relationships between multiple devices may be piping and instrumentation diagrams, etc.

[0066] The voice data consists of spoken audio recorded by the material preparation personnel. It can be recorded in real time using on-site audio pickup equipment, such as through on-site walkie-talkies.

[0067] Thus, by acquiring at least two types of preparation data from handwritten data, image data, time-series data, relational data, document data, and speech data for joint feature extraction and application, the information dimensions representing the preparation state are greatly enriched. This allows the final target adjustment instructions to be based on comprehensive and accurate multimodal data features, significantly improving the parameter adjustment accuracy under equipment topology network and complex process constraints, and enhancing the stability and accuracy of the target material preparation results.

[0068] In practical implementation, after acquiring preparation data for multiple modalities, the preparation data for multiple modalities can be stored. Specifically, multiple types of storage modules are acquired, and attribute analysis is performed on the preparation data for each modality. Based on the attribute analysis results, the preparation data for each modality is stored in the corresponding type of storage module.

[0069] It should be noted that, as Figure 5 As shown, Figure 5 This is a schematic diagram of multiple storage modules provided in the embodiments of this application, based on Figure 5The distributed heterogeneous storage architecture 50, which includes multiple types of storage modules, includes at least a relational storage module 501, a document storage module 502, a key-value pair storage module 503, and a time-series storage module 504. Among them, the time-series storage module is used to cache time-series data, the relational storage module is used to cache relational data and handwritten data, the document storage module is used to cache document, image, and audio data, and the key-value pair storage module is used as a real-time cache and message queue.

[0070] Step S102: For the preparation data of each mode, feature extraction is performed on the preparation data by the intelligent agent corresponding to the mode to obtain the data features of the preparation data. The data features of the preparation data include the input material information of the target material.

[0071] Here, the intelligent agent corresponding to a modality refers to an AI algorithm model specifically designed and trained for a particular data type (such as images, text, or continuous time-series signals), possessing independent perception, parsing, and abstract representation capabilities. The data characteristics of the preparation data include the input material information of the target material. This means that information directly or indirectly related to the material input can be extracted from various modal data. For example, the precise chemical composition and weight information of the input material can come from laboratory reports and weighing parameters recorded in relational data (e.g., collected by sensors), while the proportioning formula of the input material comes from document data. The physical form and instantaneous feeding rate of the input material when it is actually fed into the furnace can be implicitly contained in image data and time-series data. Therefore, the input material information of the target material can be determined from the data characteristics of the preparation data; that is, the data characteristics of the preparation data contain the input material information of the target material.

[0072] In some embodiments, the preparation data for each modality includes multiple sub-data. For each modality's preparation data, the process of extracting features from the preparation data using an agent corresponding to the modality to obtain the data features of the preparation data can be as follows: For each modality's preparation data, the agent corresponding to the modality extracts features from each sub-data to obtain initial features of the sub-data; at least one initial feature associated with a reference preparation state is selected as a benchmark feature; other initial features besides the benchmark feature are subjected to correlation analysis with the benchmark feature to obtain corresponding correlation analysis results; based on the correlation analysis results, multiple initial features are filtered to obtain multiple intermediate features; and multiple intermediate features are combined to obtain the data features of the preparation data.

[0073] It should be noted that sub-data refers to a single data sample or independent data instance that constitutes the preparation data of a specific modality. Here, if the preparation data of each modality is regarded as a data set, a sub-data refers to a single and complete set of data contained within that set. For example, when the preparation data includes time-series data, the sub-data is a parameter collected for a specific physical quantity, such as temperature changes inside the reactor or pressure changes in the oxygen supply pipeline; when the preparation data includes handwritten data, the sub-data is the amount of objective records corresponding to a single business field extracted from the handwritten form, such as the material replacement weight data or environmental humidity record data handwritten by the operator; when the preparation data includes document data, the sub-data is a single type of constraint parameter extracted from engineering data, such as the target feeding ratio data in the formula table or the theoretical withstand limit data of the equipment pipeline, etc. This application does not limit the specifics of the sub-data. "At least one initial feature associated with the reference preparation state" refers to the core feature that can most directly characterize, reflect, or determine the ideal preparation conditions of the target material among the many initial features extracted from the multimodal preparation data. In actual system execution, this selection process can be achieved through a preset feature mapping table or association rules. As an example, assuming that the reference preparation state includes the ideal operating condition of "the temperature of the copper matte melt is maintained between 1200°C and 1250°C", the system can automatically extract feature expressions from the sub-data directly related to this 'temperature condition' based on the preset mapping relationship, and use them as the reference features.

[0074] The process of selecting at least one initial feature associated with the reference preparation state as the benchmark feature, and performing correlation analysis on other initial features other than the benchmark feature with the benchmark feature to obtain the corresponding correlation analysis results, is used to quantitatively evaluate the degree of linear or nonlinear correlation between the benchmark feature and other initial features other than the benchmark feature. It is equivalent to indicating the correlation between two initial features, such as the Pearson correlation coefficient, Spearman rank correlation coefficient, or mutual information entropy value. This application does not limit the specifics of this process.

[0075] In practical implementation, for the preparation data of each modality, the corresponding agent extracts features from each sub-data to obtain the initial features of the sub-data. Then, the initial features other than the baseline features are compared with the baseline features to generate a Pearson correlation matrix or a Spearman correlation matrix as the correlation analysis result. If the correlation analysis result indicates that the correlation coefficient between two initial features is greater than a preset threshold, the two initial features are determined to be highly correlated, and a filtering operation is performed, that is, the initial features with lower information gain are filtered out. If the correlation analysis result indicates that the correlation coefficient between two initial features is not greater than the preset threshold, no filtering operation is performed on the corresponding initial features. After filtering, multiple intermediate features are obtained. Subsequently, these intermediate features are combined, specifically by automatically constructing paired interaction terms, three-term interaction terms, and higher-order interaction terms, thereby obtaining the data features of the preparation data.

[0076] For example, in a certain alloy smelting process, after a feature filtering step, three independent intermediate features are obtained: Intermediate Feature A: the real-time temperature value inside the reactor (variable set as T), Intermediate Feature B: the flow rate value of oxygen-enriched gas blown into the bottom of the reactor (variable set as V), and Intermediate Feature C: the weight value of the catalyst added into the reactor (variable set as M). Specifically, for the process of constructing paired interaction terms, intermediate features A and B are multiplied mathematically to construct paired interaction terms T×V. Thus, in actual metallurgical physicochemical reactions, the intensity of the oxidation reaction depends not only on temperature or gas flow rate alone, but is strongly correlated with the combined superposition of temperature and gas flow rate. Defining the interaction term T×V as a novel data feature provides a quantitative indicator characterizing the "synergistic effect strength of high temperature and high oxygen flux," avoiding prediction bias caused by isolated calculations of a single variable.

[0077] The process of constructing the three-term interaction term involves multiplying the three intermediate features together to create the interaction term T×V×M. This means that under specific reaction kinetics conditions, a particular chemical reaction may only experience a rate jump when temperature, oxygen concentration, and catalyst weight are all within specific ranges. Therefore, by constructing the three-term interaction term T×V×M, it is possible to directly obtain explicit composite data features representing "temperature-gas-catalyst" without needing extremely deep and complex neural network structures to fit implicit logic, thus significantly improving the convergence efficiency of control parameter optimization.

[0078] As for the process of constructing higher-order interaction items, specifically, four or more intermediate features (such as introducing additional parameters such as ambient relative humidity and furnace absolute pressure) can be cross-multiplied simultaneously. This will not be elaborated in the embodiments of this application.

[0079] Thus, by extracting initial features from each sub-data set and performing correlation analysis between the initial features other than the baseline features and the baseline features, and then filtering the initial features based on the correlation analysis results, noise interference terms with high collinearity or redundant overlap can be accurately identified and eliminated, overcoming the aforementioned problem of excessive consumption of electronic equipment's computing and storage resources due to redundant features. Furthermore, by combining multiple intermediate features after filtering and purification to generate the final data features of the prepared data, the input features are greatly enriched, significantly improving the accuracy of subsequent control command generation and the robustness of system operation.

[0080] In practical implementation, the process of extracting features from each sub-data and obtaining the initial features of the sub-data by using an agent corresponding to the modality can be as follows: filtering invalid data from multiple sub-data by using an agent corresponding to the modality to obtain filtered sub-data; and then extracting features from the filtered sub-data to obtain the initial features of the sub-data.

[0081] It should be noted that invalid data refers to interference data in multimodal information that is generated due to equipment failure, transmission abnormality or physical stagnation, and does not have effective information gain or violates the objective physical laws of material preparation. Examples include constant measurement point sequences (constant fields) whose values ​​remain absolutely unchanged over a long monitoring period, and completely missing fields (null value fields) caused by network packet loss. This application does not limit such data.

[0082] Thus, the underlying processing step of filtering invalid data before extracting the initial features of sub-data solves the problems of wasted computing resources and compromised control precision caused by receiving all underlying data in existing technologies. By removing interference terms before feature mapping, not only is the computing power pressure on downstream processing modules significantly reduced and the overall execution efficiency of human-computer interaction and data flow improved, but the quality of input data is also purified from the source, ensuring that the initial features of the final sub-data can accurately reflect the actual preparation conditions with a high signal-to-noise ratio.

[0083] In practice, for the preparation data of each modality, features are extracted from the preparation data through the agent corresponding to that modality. The feature extraction process for preparation data of different modalities also differs. Next, for the preparation data of each modality, the process of extracting features from the preparation data through the agent corresponding to that modality to obtain the data features of the preparation data will be explained.

[0084] In some embodiments, when the prepared data includes handwritten data, the process of extracting features from the prepared data using an agent corresponding to the modality for each modality of prepared data to obtain data features of the prepared data may be as follows: when the prepared data includes handwritten data, for the handwritten data, the corresponding agent uses multiple recognition methods to perform content recognition on the handwritten data respectively to obtain content recognition results; multiple content recognition results are grouped to obtain at least one recognition result group, and the recognition result group includes the same content recognition results; the recognition result group with the most content recognition results is taken as the target recognition result group, and features are extracted from the content recognition results in the target recognition result group to obtain data features of the handwritten data.

[0085] It should be noted that various recognition methods, such as convolutional recurrent neural networks, Transformer-based character recognition models, and PP-OCRv3 models, are not limited in the embodiments of this application.

[0086] In practical implementation, for handwritten data, a corresponding intelligent agent is used to perform content recognition on the handwritten data using various recognition methods, including convolutional recurrent neural networks, Transformer-based character recognition models, and PP-OCRv3 models, to obtain content recognition results. Then, multiple content recognition results are grouped to obtain at least one recognition result group, ensuring that the content recognition results in the recognition result group are completely identical. Next, the recognition result group containing the most content recognition results (e.g., the group that satisfies more than two-thirds of the voting results) is taken as the target recognition result group, and feature extraction is performed on the content recognition results in the target recognition result group to obtain the data features of the handwritten data.

[0087] For example, suppose a materials preparation worker fills in the current reactor temperature value on a paper material change record sheet, with the handwritten note indicating 1250℃. After acquiring an image slice containing this handwritten note, the agent corresponding to the handwritten data uses three heterogeneous optical character recognition engines (e.g., a convolutional recurrent neural network (CRNN) engine, a Transformer-based TroOCR engine, and a PP-OCRv3 engine) to independently recognize the same image slice. Each of the three recognition engines outputs an independent result string, i.e., the content recognition result, i.e., the output of engine A (CRNN). The content recognition result from Engine A is 1250℃. Engine B (TrOCR) outputs 1250℃. Engine C (PP-OCRv3), due to interference from cursive handwriting, outputs 1250C (omitting the degree symbol). Then, a string matching operation is performed, grouping completely identical strings into the same logical set to obtain the recognition result group. Specifically, it is determined that the 1250℃ output by Engine A is completely identical to the 1250℃ output by Engine B, so these two results are grouped into the first set, forming the first recognition result group. It is determined that the "1250 C" output by Engine C is inconsistent with both of the previous two, so it is grouped into the second set, forming the second recognition result group. Next, the first recognition result group contains two content recognition results, and the second recognition result group contains one content recognition result. By comparing the distribution of the number of results in each recognition result group, it is determined that the first recognition result group contains the most content recognition results (i.e., satisfying the two-thirds majority agreement principle), thus selecting the first recognition result group containing the most results as the target recognition result group. Finally, the correct content (i.e., 1250℃) is extracted from the target recognition result group, the incorrectly recognized content (i.e., 1250C) is removed, and the correct content is converted into structured numerical and unit features, thereby completing the high-accuracy handwritten data feature extraction.

[0088] In practical implementation, after obtaining the data features of handwritten data, it is also possible to obtain the timestamp information generated by the handwritten data; automatically align the timestamp information generated by the handwritten data with the obtained time-series data to obtain handwritten-time-series master data; monitor the recognition performance parameters of the content recognition process of handwritten data; when the recognition performance parameters meet the preset abnormal conditions, trigger the manual review process and record the abnormal recognition log for retraining the text recognition model.

[0089] It should be noted that the recognition performance parameters include single-page recognition time and recognition accuracy; the preset abnormal conditions include single-page recognition time exceeding a preset time threshold (e.g., 5 seconds) or recognition accuracy being less than a preset accuracy threshold (e.g., 98%).

[0090] In practical implementation, after obtaining the data features of the handwritten data, the timestamp information generated by the handwritten data is extracted. With an alignment delay of less than or equal to 1 second, the timestamp information generated by the handwritten data is automatically matched and aligned with the time-series data to generate structured handwritten-time-series master data. At the same time, point monitoring is performed, that is, continuously monitoring the single-page recognition time and the final output recognition accuracy during the content recognition process of the handwritten data. When it is determined that the single-page recognition time is greater than 5 seconds or the recognition accuracy is less than 98%, it is determined that there is a reliability risk in the recognition result of the handwritten data. The data processing process is automatically rolled back to the manual review channel for material preparation personnel to review and confirm. The handwritten data images with recognition difficulties and the final confirmed content results are merged and recorded as an abnormal recognition log to trigger the retraining of the text recognition model.

[0091] In practical implementation, the process of extracting features from the content recognition results in the target recognition result group to obtain the data features of the handwritten data can be as follows: when the handwritten data is presented in form, features are directly extracted from the content recognition results of the target recognition result group to obtain the data features of the handwritten data; when the handwritten data is presented in non-form form, a form template for the handwritten data is obtained, and based on the form template, the preparation meaning of the content recognition results in the target recognition result group during the preparation process is determined; preparation meaning is added to the content recognition results of the target recognition result group, and features are extracted from the content recognition results with added preparation meaning to obtain the data features of the handwritten data.

[0092] It should be noted that "form format" refers to a layout where the data carrier has pre-defined structured grid lines, fixed header text, and other corresponding relationships. For example, a standardized metallurgical workshop shift handover record sheet with fixed header text such as "reactor temperature" and "initial feed amount." "Non-form format" refers to a free layout where the data carrier has a blank background and lacks pre-defined structured guide lines or header text. For example, discrete handwritten notes such as "1250℃" or "ore feed increased by 20kg" randomly recorded by material preparation personnel on blank sticky notes or unlined notebooks. "Preparation meaning" refers to the specific physical, chemical, or operational adjustment action type represented by an isolated content recognition result within a specific material preparation process system. For example, assigning the simple numerical sequence "200" the preparation meaning of "oxygen-enriched air introduction rate (standard cubic meters / hour)."

[0093] In practice, connected component detection is performed on the overall pixel distribution of the handwritten data image. If closed grid connected components and standard row and column dividing lines are detected, the handwritten data is determined to be presented in form format. When the handwritten data is presented in form format, it is determined that the content recognition result of the target recognition result group has been automatically aligned with the inherent structured header of the form, thus allowing direct feature extraction from the content recognition result of the target recognition result group to obtain the data features of the handwritten data. If the handwritten data does not contain closed grid connected components, the handwritten data is determined to be presented in non-form format. When the handwritten data is presented in non-form format, a form template matching the handwritten data is retrieved from the local feature database. Then, based on the expert rule base preset within the form template, the preparation meaning of the content recognition result of the target recognition result group in the preparation process is determined. Subsequently, the preparation meaning is added to the content recognition result of the target recognition result group, and feature extraction is performed on the content recognition result with the added preparation meaning to obtain the data features of the handwritten data.

[0094] In some embodiments, when the preparation data includes image data, the process of extracting features from the preparation data by an agent corresponding to the modality for each modality of preparation data to obtain the data features of the preparation data may be as follows: when the preparation data includes image data, the image data is identified by a corresponding agent to obtain process parameters of the preparation process, which are used to indicate the preparation state; and features are extracted from the process parameters to obtain the data features of the preparation data.

[0095] In practical implementation, for image data, the corresponding intelligent agent calls the YOLOv8-seg segmentation network or ResNet-50 classification network to identify the image data, obtain the identification result representing the preparation state at the target time, and extract features from the identification result to obtain the data features of the image data; wherein, the preparation state at the target time is, for example, the melt level height value or oxidation state category, etc., which is not limited in this application embodiment.

[0096] It should be noted that when processing image data, the corresponding agent will invoke specific visual neural network algorithms to perform feature extraction operations based on the specific preparation conditions. For example, for monitoring the physical morphology inside the reactor, the agent invokes the YOLOv8-seg segmentation network model to extract geometric segmentation features that can strictly distinguish the pixel boundaries between the slag layer and the molten metal surface, and then calculates the numerical features of the liquid level height. For oxygen-enriched blowing conditions, the agent invokes the ResNet-50 classification network model to perform deep convolution calculations on the image data, extracts visual features characterizing the flame morphology or the degree of oxidation on the melt surface, and maps them into three-classification features indicating whether the current condition is under-blown, over-blown, or normal.

[0097] In actual implementation, after obtaining the data characteristics of the image data, it is also possible to obtain the original image data, labeled data, and model weight data corresponding to the image data processing process, and to distribute and store the original image data, labeled data, and model weight data according to the preset number of copies, and to configure the target retention time for the distributed storage data.

[0098] It should be noted that after obtaining the data features of the image data, the system automatically extracts the original image generated during the processing of the visual network, the JSON labeled data that records the feature results, and the model weight data of the currently used visual neural network (such as YOLOv8-seg or ResNet-50). Then, the original image, JSON labeled data, and model weight data are written together into a distributed document database (such as MongoDB GridFS) for storage, and the retention period of this batch of data in the database is set to be greater than or equal to a time threshold (such as 7 years).

[0099] It should be noted that storing the context of image processing and the original materials over a long period with high redundancy not only meets the needs of auditing and tracing historical abnormal working conditions in industrial sites, but also provides valuable data support for subsequent incremental training and offline fine-tuning of visual neural networks.

[0100] In some embodiments, when the preparation data includes time-series data, the process of extracting features from the preparation data for each modality using an agent corresponding to that modality to obtain the data features of the preparation data can be as follows: When the preparation data includes time-series data with different state parameters, for each state parameter, a statistical benchmark value of the time-series data is obtained, and based on the statistical benchmark value, the corresponding agent filters out abnormal data in the time-series data to obtain filtered time-series data; missing data in the filtered time-series data is numerically interpolated to obtain target time-series data; target time-series data with different state parameters are aggregated to obtain aggregated time-series data, and features are extracted from the aggregated time-series data to obtain the data features of the preparation data.

[0101] It should be noted that the statistical benchmark value refers to the reference benchmark value calculated by a specific mathematical statistical model for the continuously collected state parameter value distribution sequence. It is used as the evaluation benchmark for identifying abnormal data that deviates from the fluctuation range of normal preparation conditions. For example, it can be the median of the continuous reading sequence of temperature sensor or flow sensor within a specific time window, and the median can be used as the statistical benchmark value to determine whether the reading at the current moment is abnormal data caused by equipment failure.

[0102] In practical implementation, for the time series data of each state parameter, the median of the time series data within the target time window is obtained as a statistical benchmark value. Then, based on the statistical benchmark value, the corresponding intelligent agent filters out abnormal data in the time series data to obtain filtered time series data. Next, the missing data in the filtered time series data is numerically imputed to obtain the target time series data. Then, the target time series data of different state parameters are aggregated to obtain aggregated time series data, and feature extraction is performed on the aggregated time series data to obtain the data features of the prepared data.

[0103] For example, when processing high-frequency continuous time-series data, the corresponding agent first uses a set time window (i.e., a target time window, such as a Hampel window) to remove outliers and clean the data. Specifically, the agent automatically identifies and removes constant fields whose values ​​remain constant within the continuous time window, and centrally extracts dynamic indicator features that physically change over time. For instance, the agent calculates and extracts the rate of change or first-order time derivative features of state parameters such as temperature, pressure, or flow rate after cleaning, thereby characterizing the real-time dynamic fluctuation trend of the underlying operating parameters of the preparation system. The extracted data features are then used as the data features of the corresponding preparation data.

[0104] In some embodiments, when the prepared data includes relational data, the process of extracting features from the prepared data using an agent corresponding to the modality for each modality to obtain the data features of the prepared data may involve: when the prepared data includes relational data and the relational data is a low-frequency discrete distribution, obtaining the constraints corresponding to the relational data; aligning the low-frequency discrete distribution relational data using the corresponding agent under the constraints to obtain aligned relational data; and extracting features from the aligned relational data to obtain the data features of the prepared data.

[0105] In practical implementation, when processing relational data, the corresponding intelligent agent mainly processes low-frequency and discretely distributed laboratory test results or environmental sampling indicators. Specifically, the intelligent agent completes the upsampling alignment operation of the data time axis through a quadratic interpolation algorithm and boundary derivative constraint rules.

[0106] It should be noted that most relational data are low-frequency discrete distributions. Therefore, it is necessary to obtain the boundary derivative constraints corresponding to the relational data as constraints. Then, through the corresponding intelligent agent, under the constraints of the boundary derivative constraints, the quadratic interpolation algorithm is called to align the low-frequency discrete relational data (such as aligning it to a unified time grid) to obtain the aligned relational data. Then, feature extraction is performed on the aligned relational data to obtain the data features of the prepared data.

[0107] It should be noted that relational data (such as XRF fluorescence spectral composition data issued by the laboratory every few hours) are low-frequency, discrete point data. The underlying time-series data, such as temperature and pressure, are high-frequency and continuous. To integrate these two data points in subsequent processes, it is necessary to supplement the low-frequency test points (i.e., upsampling interpolation) and stretch them into a continuous curve with corresponding component values ​​every second. Without constraints, simply using quadratic interpolation or higher-order polynomials to smoothly connect a few sparse test points can easily lead to Runge's phenomenon, where the interpolation curve exhibits violent oscillations and physically impossible extrema between known data points. Therefore, boundary derivative constraints are needed. Here, boundary derivative constraints refer to a mandatory condition set when using a quadratic interpolation algorithm to fit unknown data between two points: it mandates that the slope (rate of change) of the fitted curve at the boundary points (test points) must meet specific numerical or physical limits. In this way, by using boundary derivative constraints as constraints, the virtual component data generated by interpolation for each second must satisfy both the real data points on the test report and ensure that its rate of change conforms to the objective laws of metallurgy, thus successfully achieving precise alignment of features.

[0108] In practical implementation, for relational data, a unique identifier can be assigned to each operating furnace or feeding batch by a corresponding intelligent agent. Then, using the unique identifier, the discrete test results are precisely bound to the corresponding material batch, and the material change records in the control system are synchronously linked, thereby realizing three-dimensional tracking of composition, time, and weight to obtain material tracking data. Next, an uncertainty band of plus or minus two standard deviations is calculated and generated, and the generated uncertainty band is input into the model predictive control module as a soft constraint condition, thereby completing the extraction and processing of relational data features.

[0109] It should be noted that the unique identifier refers to the exclusive data number assigned to each independently operating furnace or charging batch, which is used to accurately anchor and connect discrete attributes scattered across different system nodes; the uncertainty band refers to the boundary range (e.g., a fluctuation range of plus or minus two standard deviations) determined based on mathematical statistics principles, which characterizes the confidence interval of numerical fluctuations and is used to provide a reasonable tolerance boundary for subsequent control algorithms.

[0110] In some embodiments, when the preparation data includes document data, the process of extracting features from the preparation data using an agent corresponding to the modality for each modality to obtain data features of the preparation data may be as follows: When the preparation data includes document data, the corresponding agent parses the document data to obtain the image portion and the text portion of the document data; identifies the image portion to obtain the equipment image used to prepare the target material in the document data, and extracts features from the equipment image to obtain image features in the document data; extracts text parameters associated with the preparation process from the text portion, and extracts features from the text parameters to obtain text features in the document data; and determines the image features and text features as data features of the preparation data.

[0111] In practical implementation, a corresponding intelligent agent parses the document data in PDF or drawing format, separating the charts and text to obtain the image and text portions of the document data. Then, a visual Transformer model is invoked to recognize the image portion, obtaining the pipeline and instrumentation flow diagram used for preparing the target material as the equipment image, and feature extraction is performed on the equipment image to obtain the image features of the document data. Simultaneously, a BERT model is invoked to extract text parameters associated with the preparation process from the text portion, and feature extraction is performed on the text parameters to obtain the text features of the document data. Finally, the image features and text features are merged to determine the data features of the document data.

[0112] It should be noted that the text parameters associated with the preparation process include, for example, equipment and safety constraint parameters (such as the maximum safe temperature that the reactor can withstand, the maximum allowable pressure of the gas supply pipeline, etc.), process formula and material ratio parameters (such as the target weight ratio between various initial reaction raw materials, the theoretical consumption coefficient of a specific chemical catalyst, the standard acidity and alkalinity range that the target material needs to reach, etc.), control and process timing parameters (such as the forced holding time requirement for a specific metallurgical reaction stage, the maximum allowable heating rate limit of the heating device, etc.), etc. The embodiments of this application do not limit these parameters.

[0113] In practical implementation, after determining image features and text features as the data features for data preparation, version management of document data can also be performed. Specifically, the currently acquired document data is compared with historical versions of document data. When the differences obtained from the comparison meet the preset remodeling conditions, a difference log of the difference content is recorded, and remodeling is performed based on the currently acquired document data. After version management is completed, the extracted text features, image features, and difference logs are processed by data tracking, and the data generated by the data tracking is stored in a specific field of a preset database for full-text search.

[0114] It should be noted that the difference comparison adopts a Git-like difference comparison logic, which is used to compare the changes in document data character by character. The preset remodeling conditions include that the number of changed characters in the difference content is greater than ten characters. Specific fields include binary format fields based on JavaScript Object Notation (JSON) (such as PostgreSQL's JSONB fields). The binary format fields are used to support efficient retrieval of unstructured data, so that the time consumption of full-text search is less than or equal to one hundred milliseconds.

[0115] In actual implementation, after determining the data characteristics of the document data, the underlying version management module uses a Git-like difference comparison to calculate the number of character changes between the currently parsed document data and the previous version of the document data cached in the processing system. When it is determined that the number of changed characters is greater than 10 characters, the downstream model remodeling process is automatically triggered, and a difference log containing the specific modification location and content is recorded synchronously. Then, the data tracking operation is performed to format the parsed image features, text features, and difference logs and store them in the JSONB field of the PostgreSQL database. Since the JSONB field supports the creation of efficient inverted indexes, it is possible to complete the full-text search of the data stored in the specific field with a response time of less than or equal to 100 milliseconds when a retrieval request is subsequently called.

[0116] For example, suppose a materials preparation engineer uploads a newly updated "Copper Matte Preparation Process Specification" document to the processing system. After the corresponding agent completes image-text separation and feature extraction, the version management module uses a Git-like difference comparison logic to perform a character-level comparison between the newly updated "Copper Matte Preparation Process Specification" document and the old version document stored in the processing system. Suppose the comparison finds that "the oxygen-enriched air inlet rate has changed from 20,000 standard cubic meters per hour to 25,000 standard cubic meters per hour," and the total number of changed characters is clearly greater than 10 characters. When the total number of changed characters is greater than 10 characters, it is determined that the remodeling condition is met, and a difference log is automatically generated. The difference log records the content "the inlet rate has changed to 25,000." Then, the predictive model is immediately triggered to extract the data features of the newly updated "Copper Matte Preparation Process Specification" document for remodeling calculation. Subsequently, the underlying tracking program writes the difference log and the newly extracted feature data in batches into the JSONB field of a PostgreSQL relational database for persistent storage. Subsequently, when material preparation personnel enter the keyword "25,000" to query historical process change records, the underlying indexing mechanism of the JSONB field can return the matching difference logs and corresponding process parameters as full-text search results to the terminal display screen within 100 milliseconds, thereby ensuring the rigor of document data version traceability and the retrieval efficiency of underlying data interaction.

[0117] In some embodiments, when the preparation data includes speech data, the process of extracting features from the preparation data using an agent corresponding to the modality for each modality to obtain the data features of the preparation data can be as follows: When the preparation data includes speech data, the corresponding agent performs text conversion on the speech data to obtain the corresponding text content, and performs terminology correction on the text content to obtain the target text content; target words are extracted from the target text content, and the target words include process entities and the relationships between different process entities; based on the target words, a tuple corresponding to the speech data is constructed, and the tuple includes the corresponding preparation equipment; features are extracted from the tuple to obtain the data features of the preparation data.

[0118] In practical implementation, the speech data is converted into text by a speech recognition model such as the Whisper model deployed by the corresponding intelligent agent to obtain the corresponding text content. Then, a professional domain dictionary is called to perform term replacement and correction on the text content to obtain the target text content. Then, the entity relation extraction model is called to extract the target word units from the target text content. Based on the target word units, the processing system constructs a multi-tuple in the form of triples corresponding to the speech data. Finally, the multi-tuple is subjected to graph node feature extraction to obtain the data features of the prepared data.

[0119] For example, when processing voice data, the corresponding agent first converts the recorded audio waveform into text data using a speech recognition model (such as the Whisper network), and performs strict terminology standardization correction in conjunction with an industry-specific dictionary. Then, the agent calls a unified named entity recognition network to perform deep semantic parsing on the corrected target text, accurately extracting process entity identifiers (such as specific oxygen supply valves, specific reactor numbers) and the objectively existing action or physical connection relationships between different process entities. Finally, a machine-readable knowledge graph tuple (such as the "entity 1-relationship-entity 2" structure) is constructed, and graph node features are extracted from the tuple to obtain the data features of the prepared data.

[0120] Thus, by employing refined and customized analysis and feature extraction for handwritten data, image data, time-series data, relational data, document data, and voice data, the problem of low human-computer interaction efficiency in handling heterogeneous data in existing technologies is solved. Specifically, by combining multiple recognition methods with majority voting logic for handwritten data, and by correcting professional terminology in voice data and converting it into structured plural groups, the recognition error rate in the process of converting unstructured human experience information is significantly reduced, and the accuracy of information acquisition through human-computer interaction is significantly improved. By using statistical benchmark values ​​to filter out abnormal readings in time-series data and performing numerical interpolation, and by aligning the time axis of low-frequency discrete relational data based on constraints, data deviations caused by sensor noise and multi-source asynchronous sampling are eliminated, ensuring that the system can obtain a stable and reliable continuous input stream. For document data, a processing method of separating images and text and independently extracting image and text features is adopted, overcoming the problem that traditional electronic devices cannot deeply mine the inherent logic of engineering drawings due to computing power limitations, and providing more accurate and multi-dimensional deep features for the final generation of target adjustment instructions.

[0121] Step S103: Based on the data characteristics corresponding to the preparation data of multiple modes, simulate and generate output material information.

[0122] Here, output material information refers to the physical state, chemical composition, yield, and quality indicators of the intermediate or final products expected to be produced during the current cycle or a specific future cycle of the target material preparation process (for example, in the context of non-ferrous metal smelting, this information may specifically refer to matte grade, copper content in slag, or melt temperature, etc.).

[0123] In some embodiments, the process of simulating the generation of output material information based on the data features corresponding to the preparation data of multiple modalities may involve obtaining component data and weight data associated with the preparation process from the data features corresponding to the preparation data of multiple modalities as input material information; and based on the input material information, performing material and energy balance calculations by running executable code (i.e., computer-executable instructions) to obtain output material information, wherein the executable code is derived from the material and energy balance equations used to prepare the target material.

[0124] It should be noted that executable code refers to a sequence of computer instructions that can be directly interpreted and run or compiled and executed by the underlying computing engine. Here, it is used to transform complex metallurgical equations that are originally recorded in the form of static text or charts into underlying computing logic that the control system can call in real time and perform adaptive calculations. For example, by parsing the metallurgical formula formulas stored in offline spreadsheet files, Python script code that can be automatically generated and run directly in the background of the control system can be generated.

[0125] Material and energy balance calculation refers to the numerical extrapolation process that quantitatively analyzes the changes in input substances, output substances, and the internal thermodynamic state of the system within a specific production period based on the laws of conservation of mass and energy. Here, it is used to dynamically extrapolate and simulate the output material information after the preparation reaction is completed based on real-time input material information; for example, based on the weight data of the initial raw materials put into the reactor and the composition data of each chemical element contained in the blown gas, the expected grade of copper matte and the expected heating power required to maintain the physicochemical reaction are calculated.

[0126] In practical implementation, before simulating the generation of output material information, composition and weight data are accurately obtained from the data characteristics corresponding to the preparation data of multiple modalities, and these data are used as input material information. Then, a file containing the material and energy balance equations used to prepare the target material is automatically imported. A preset script conversion algorithm then automatically extracts and converts these equations into executable Python code. Subsequently, based on the acquired input material information, the executable code is run to perform material and energy balance calculations, completing forward logic deduction and obtaining output material information characterizing the product state.

[0127] For example, from the data features corresponding to the preparation data of multiple modes, the transient feeding rate of solid materials is extracted, such as: the current cycle copper concentrate feeding rate is 100 tons / hour, and the quartz slagging agent feeding rate is 15 tons / hour (i.e., important data). The chemical element percentage of the solid raw materials is also extracted, such as copper concentrate containing 25% copper, 30% iron, and 32% sulfur (composition data). The parameters of the oxygen-enriched air currently blown into the reactor are also extracted, such as the air supply rate being 25,000 standard cubic meters / hour and the oxygen concentration being 70% (composition data). The extracted data is used as input material information. Based on the input material information, material and energy balance calculations are performed by running executable code to obtain output material information. Specifically, the material and energy balance equations used to prepare the target material are obtained, such as... as well as The material and energy balance equations are converted into executable code. Based on the input material information, the executable code is used to calculate the material and energy balance and obtain the output material information. Specifically, the executable code automatically deduces the preparation process of the target material based on the chemical reaction. Specifically, the blown-in oxygen preferentially reacts with the iron and sulfur elements in the concentrate in an exothermic oxidation reaction. The generated iron oxide immediately combines with the added quartz slagging agent to generate liquid iron silicate (i.e., slag). The sulfur dioxide generated in the reaction is converted into industrial flue gas and escapes. The remaining unoxidized copper and sulfur in the concentrate melt and combine at high temperature to form a eutectic of cuprous sulfide and ferrous sulfide (i.e., the target intermediate product: matte). In this way, it can be accurately calculated that 65% of the iron element enters the slag and 35% is retained in the matte. At the same time as the material distribution is completed, the total chemical reaction heat value released by the above iron-sulfur oxidation reaction is calculated. Then, the sensible heat value required to heat the room temperature cold material to the melting temperature is subtracted, and the estimated heat dissipation loss value of the reactor water-cooled wall is deducted. Thus, based on the final net heat surplus, the expected temperature of the current thermodynamic equilibrium within the reactor system can be calculated.

[0128] After the above process, the final output is the core indicator representing the expected product after the reaction reaches physicochemical equilibrium under the current input conditions, namely the output material information. Specifically, this includes: expected matte grade: the calculated mass percentage of copper in the matte produced by the reaction (e.g., accurately predicting the matte grade under the current conditions to be 62.5%), expected slag yield and properties: the calculated slag formation rate and iron-silicon ratio, and expected melt endpoint temperature: the thermodynamic dynamic equilibrium temperature inside the molten pool calculated based on the energy surplus / deficit value (e.g., the predicted temperature is 1235℃).

[0129] In this way, the static material and energy balance equations are converted into executable code that can be directly run by the processing system, eliminating the problem of low system execution efficiency caused by relying on manual verification of formulas and manual input of derivation parameters. The fully automated material and energy balance calculation process enables the system to strictly follow the objective laws of physicochemical processes and perform a closed-loop scientific derivation of high-frequency dynamic input material information, greatly improving the accuracy and real-time response capability of the simulated output material information.

[0130] In practice, the material and energy balance equations are extracted from preparation data of multiple modes.

[0131] It should be noted that the material and energy balance equation refers to the mathematical analytical formula or logical rule used to quantitatively describe the conservation relationship of mass conversion and thermodynamic energy conversion of the target material during the preparation process. The material and energy balance equation does not rely on external artificial preset, but is directly obtained by dynamic analysis or calculation from preparation data of multiple modes. It is used to generate executable code to calculate and output material information, such as chemical equations extracted from document data containing process formulations.

[0132] In practical implementation, based on preparation data from multiple modalities, text parameters containing mathematical operators and chemical element variables are extracted. For example, an agent corresponding to the document data modality parses the document data to obtain the text and image parts. Then, a natural language processing model is called to extract text parameters containing mathematical operators and chemical element variables from the text part. The extracted text parameters are then logically reorganized and validated according to a preset metallurgical process abstract syntax tree, thereby converting unstructured text parameters into structured mathematical analytical expressions, which are then determined as material and energy balance equations.

[0133] Thus, by dynamically extracting material and energy balance equations directly from preparation data across multiple modalities, this method eliminates the outdated interactive approach of relying on manual review of engineering documents and input of calculation formulas, resolving the issues of low efficiency in human-computer interaction and the susceptibility to subjective transcription errors in industrial settings. Furthermore, the extracted material and energy balance equations closely reflect the actual metallurgical preparation state and the latest changes in process specifications at the target time, providing a highly accurate foundational model for subsequent conversion into executable code and significantly improving the accuracy of parameter simulation calculations.

[0134] In practical implementation, the process of calculating material and energy balance by running executable code based on input material information to obtain output material information can be as follows: for the preparation data of multiple modes, image data of the target material output container is collected, and three-dimensional point cloud reconstruction is performed to obtain the volume data of the material in the output container; based on the volume data, the material flow rate data is determined, and the material flow rate data is written back to the executable code to participate in the material and energy balance calculation to obtain the output material information.

[0135] It should be noted that 3D point cloud reconstruction is used to non-contactly acquire the physical and geometric features of high-temperature target materials in three-dimensional space. For example, by analyzing images of molten slag inside a metallurgical slag bag captured by an industrial camera, a 3D coordinate grid of the dynamic surface of the molten slag can be constructed. Material flow rate data refers to the absolute quantity of material flowing in or out per unit time (expressed as volumetric flow rate or mass flow rate), which is used here to indicate the actual material preparation output rate.

[0136] In practical implementation, for the preparation data of various modalities, 3D point cloud reconstruction is performed on the image data collected from the target material output container. The height coordinate grid data of the target material surface inside the output container is extracted. Then, the height coordinate grid data is used to fit the surface of the target material, and 3D spatial integration is performed in combination with the geometric model of the inner wall contour of the output container to obtain the volume data of the material inside the output container. Then, the time interval value corresponding to two consecutive acquisition cycles is obtained. The difference between the volume data corresponding to the two consecutive acquisition cycles is divided by the time interval value to obtain the material flow rate data. Then, the material flow rate data is written back to the executable code as a dynamic feedback variable, so that the material and energy balance calculation is performed again based on the updated executable code to obtain the output material information.

[0137] In this way, by non-contactly reconstructing 3D point clouds from image data collected from the target material output container to obtain volume data, and further calculating material flow data and writing it back into the executable code, the problem of existing control systems being unable to obtain real-time material output status under high-temperature and harsh operating conditions due to the limitations of the physical performance of contact sensors is solved. At the same time, the non-contact 3D vision measurement combined with the dynamic code write-back mechanism enables the calculated output material information to accurately fit the actual physicochemical changes in the industrial environment, significantly improving the prediction accuracy and adaptive correction capability of the data processing process.

[0138] Step S104: Based on the output material information and the input material information, an initial adjustment command is generated. The initial adjustment command is used to adjust the preparation parameters after the target time in the preparation process of the target material.

[0139] Here, the initial adjustment command refers to the basic control signal that the control system preliminarily calculates based on the difference between the expected current operating conditions and the set target, and is used to guide the underlying execution equipment to change the physical operating state, i.e., the preparation parameters (such as adjusting the valve opening, changing the feeding rate, or adjusting the heating power).

[0140] In some embodiments, the process of generating an initial adjustment command based on output material information and input material information may involve determining the feedforward compensation requirement in the target material preparation process based on the difference between the output material information and the target output material requirement; and generating the initial adjustment command by calculating the material and energy balance equation used to prepare the target material based on the feedforward compensation requirement and the input material information.

[0141] It should be noted that the target output material requirements refer to the range or benchmark values ​​of physicochemical indicators that the prepared product must meet in the pre-set process standard for the target material preparation. This is used to evaluate whether the simulated output material information meets the standard. For example, it may be the pre-set target percentage of matte grade in non-ferrous metal smelting process (e.g., set to 65%) or the safe upper limit threshold of copper content in slag.

[0142] Feedforward compensation requirements refer to the amount of control correction that needs to be performed on the material ratio or energy input in advance to offset the expected product quality deviation before the disturbance actually affects the target material. For example, to compensate for the difference in predicted low matte grade, it is necessary to determine the feedforward compensation requirement to increase the amount of oxygen-enriched air introduced by a certain volume or reduce the amount of quartz flux added by a certain weight.

[0143] In actual implementation, when generating adjustment commands, the output material information and target output material requirements are first extracted. Then, it is determined whether there is a difference between the output material information and the target output material requirements. If there is a difference between the output material information and the target output material requirements, the feedforward compensation requirement in the target material preparation process is determined according to the specific magnitude of the difference and the positive or negative offset direction. If there is no difference between the output material information and the target output material requirements, the feedforward compensation requirement is set to zero constant, and the current compensation action is terminated to avoid introducing unnecessary control oscillations.

[0144] Then, the feedforward compensation requirement and the input material information are synchronously input into the feedforward control module. The feedforward compensation requirement is used as the active compensation variable, and the input material information is used as the initial boundary condition. The feedforward control module directly calls the material and energy balance equations for preparing the target material, which are pre-converted into computer-executable code, to perform matrix calculations. The output is a precise increment for changing the preparation parameters (e.g., adjusting the opening of the gas supply valve), and the precise increment is generated as the initial adjustment command.

[0145] For example, the input material information is "the feeding rate of copper concentrate is 100 tons / hour, and the copper content of the copper concentrate is 25%. The current recorded basic oxygen-enriched air supply is 20,000 standard cubic meters / hour". The target output material requirement is "according to the process specifications of the production order, after the target material preparation is completed, the target grade of the produced matte (i.e., the mass percentage of copper in the matte) must reach 65%". In the process described above, based on the input material information, it is simulated that "if the basic gas supply is maintained at 20,000 standard cubic meters / hour, the final output material information (expected matte grade) at the target time will only be 60%". Therefore, the output material information (60%) is extracted and compared with the target output material requirement (65%). It is calculated that there is an absolute grade gap difference of 5% between the output material information and the target output material requirement. Based on the 5% difference between the output material information and the target output material requirement, the feedforward compensation requirement in the target material preparation process is determined, that is, "it is necessary to further deepen the target material preparation process". The oxidation reaction during the preparation of the target material is addressed by supplementing an additional amount of oxygen to oxidize more iron and sulfur, thereby compensating for the grade gap. The determined feedforward compensation requirement (i.e., the additional reaction cost required to increase the grade of copper matte by 5%) and the input material information (feeding rate 100 tons / hour, copper specific gravity 25%) are substituted into the material and energy balance equation used to prepare the target material for reverse quantitative solution. This yields the conclusion that "under the dynamic boundary condition of a feeding rate of 100 tons / hour, in order to achieve the feedforward compensation requirement, an additional 2500 standard cubic meters / hour of oxygen-enriched air must be added." Based on this, the additional oxygen demand (2500 standard cubic meters / hour) calculated from the feedforward compensation requirement is numerically superimposed with the initial air supply baseline (20000 standard cubic meters / hour) corresponding to the input material information. This generates an initial adjustment command directly sent to the underlying actuator, namely, "adjusting the oxygen-enriched air supply valve opening setting value after the target time to the corresponding control level of 22500 standard cubic meters / hour in advance."

[0146] In this way, before the physical properties of the actual metallurgical products deviate, the predicted differences can be converted into feedforward compensation requirements in advance, and quantitative forward calculations can be performed through the material and energy balance equations. This enables proactive offsetting and precise compensation for potential fluctuations in input materials in advance, greatly shortening the adjustment response time of the target material preparation parameters, fundamentally suppressing the violent fluctuations in production conditions, and significantly improving the accuracy of the target material preparation results and the stability of online operation.

[0147] Step S105: Based on the data characteristics corresponding to the preparation data of multiple modes and the initial adjustment instructions, determine the target adjustment instructions for adjusting the preparation parameters during the preparation process.

[0148] Here, the target adjustment command refers to the final control signal generated after secondary correction, verification or constraint optimization, which takes into account the initial adjustment command and multi-dimensional data characteristics. This command can be directly sent to the underlying controller (such as DCS or PLC) to drive the field actuators.

[0149] In some embodiments, the process of determining a target adjustment instruction for adjusting preparation parameters during the preparation process based on the data features corresponding to preparation data of multiple modalities and the initial adjustment instruction may involve fusing the data features of preparation data of each modality and using a target time-series prediction model to predict the preparation state within a target time period after the target time based on the first fused feature obtained from the fusion, as the predicted preparation state; when the predicted preparation state does not conform to the reference preparation state, the initial adjustment instruction is adjusted to obtain the target adjustment instruction, and after the target adjustment instruction is executed, the preparation process of the target material is in the reference preparation state.

[0150] It should be noted that the target time series prediction model refers to a model that can infer the future physical or chemical evolution trajectory according to the time evolution sequence based on the multidimensional input variables of the current and historical moments. Here, it is used to receive the multidimensional feature fusion results and forward calculate the expected state performance of the material preparation process in the future time period. For example, it is an LSTM model, TimesNet model, iTransformer model, ARIMA model, etc., pre-trained on historical metallurgical datasets, or a dynamic state inference engine built based on a hidden Markov model, etc. This application does not limit the specific implementation of the model.

[0151] The reference preparation state refers to the ideal preparation parameters that the target material should maintain at a specific preparation stage, as defined by standard metallurgical process specifications, safety production rules, or specific material order requirements. For example, the ideal operating condition that is clearly specified in the preparation process guide is that "the temperature of the copper matte melt is maintained between 1200 degrees Celsius and 1250 degrees Celsius".

[0152] In actual implementation, after generating the initial adjustment command, forward prediction and command verification are performed. Specifically, the data features of the preparation data of each modality are spliced ​​and fused to obtain the first fused feature. Then, using the internally loaded target time series prediction model, forward inference calculation is performed based on the first fused feature to predict the preparation state within the target time period after the target time, and the inference result is used as the predicted preparation state. Then, the reference preparation state pre-stored in the database is extracted to determine whether the predicted preparation state conforms to the reference preparation state.

[0153] If the predicted preparation state matches the reference preparation state, the initial adjustment command is deemed safe and effective, and therefore no adjustment is made to the initial adjustment command; instead, the initial adjustment command is directly confirmed as the target adjustment command. If the predicted preparation state does not match the reference preparation state, the initial adjustment command is adjusted in the form of numerical restrictions or incremental superposition to obtain the adjusted target adjustment command, thereby ensuring that the preparation process of the target material can be strictly in the reference preparation state after the target adjustment command is executed.

[0154] For example, for the preparation data of each modality, the data features containing input material information are obtained by the intelligent agent corresponding to the modality. The cross-modal attention network is invoked to perform weight allocation and joint representation fusion of the data features of the preparation data of each modality to obtain the first fused feature.

[0155] Then, the first fused feature is input into the target time-series prediction model to calculate and output the melt temperature evolution curve and the reactor pressure change curve within the target time period after the target time (for example, for smelting processes with large time lag and high thermal inertia, the target time period is set to the next few hours). These are used as the predicted preparation state. The extreme values ​​of the predicted preparation state curves are then compared with the safe range of the reference preparation state defined by the standard process specification to determine whether the extreme values ​​of the predicted preparation state curves fall within the safe range of the reference preparation state. If the extreme values ​​of the predicted preparation state curves do not fall within the safe range of the reference preparation state (i.e., the predicted preparation state does not conform to the reference preparation state), the expert rule engine is called to calculate the deviation penalty term. The deviation penalty term is used to make reverse fine adjustments to the valve opening setting value in the initial adjustment command to obtain the corrected target adjustment command. This forces the target material preparation process to converge to the reference preparation state after the target adjustment command is executed by the underlying network.

[0156] Thus, before officially issuing control parameters, it is determined whether the predicted preparation state conforms to the reference preparation state, and the initial adjustment instructions that do not conform to the reference preparation state are forcibly adjusted. This not only effectively avoids over-adjustment or adjustment distortion caused by local characteristic fluctuations, eliminating potential process oscillation hazards from the root, but also ensures that the final issued target adjustment instructions can accurately guide the preparation equipment to operate smoothly, greatly improving the accuracy of the target material preparation results and the safety and stability of the production process.

[0157] In practical implementation, before predicting the preparation state within the target time period after the target time based on the first fusion feature obtained by the target time series prediction model, multiple candidate time series prediction models and labeled preparation sample data can be obtained. The preparation sample dataset includes preparation sample data for training and preparation sample data for validation. The labels are used to indicate the standard adjustment instructions corresponding to the preparation sample data. Using the preparation sample data for training, multiple candidate time series prediction models are trained respectively to obtain multiple trained candidate time series prediction models. The preparation sample data for validation is input into multiple trained candidate time series prediction models respectively to obtain the predicted preparation state corresponding to each model. Each predicted preparation state is compared with the label to determine the prediction error of each trained candidate time series prediction model and obtain the parameter complexity of each trained candidate time series prediction model. Based on the prediction error and parameter complexity, the model evaluation cost of each trained candidate time series prediction model is determined, and the model with the minimum model evaluation cost is selected from all trained candidate time series prediction models as the target time series prediction model.

[0158] It should be noted that model evaluation cost refers to a mathematical evaluation index that quantifies the feasibility and computational economy of deploying a specific algorithm model in the preparation equipment of the target material. Here, model evaluation cost is obtained by quantifying and fusing the prediction error and parameter complexity of the model through a specific mathematical formula, and is used as the sole criterion for finalizing the target time series prediction model among multiple candidate time series prediction models.

[0159] In practice, before using the target time-series prediction model to predict the preparation state within the target time period after the target time based on the first fusion feature obtained by fusion, the target time-series prediction model is first selected. Specifically, multiple candidate time-series prediction models and preparation sample data with labels are obtained. The preparation sample dataset includes preparation sample data for training and preparation sample data for validation. The labels are used to indicate the standard adjustment instructions corresponding to the preparation sample data.

[0160] Then, using the preparation sample data for training, backpropagation gradient descent is performed to train multiple candidate time series prediction models, resulting in multiple trained candidate time series prediction models. Next, the preparation sample data for validation is input into each of the trained candidate time series prediction models to obtain the predicted preparation state output by each model. Subsequently, each predicted preparation state is compared with the label to determine the prediction error of each trained candidate time series prediction model. The parameter complexity of each model is obtained by parsing the network architecture file. Based on the prediction error and parameter complexity, the model evaluation cost of each model is calculated, such as by weighted summation of the prediction error and parameter complexity, to obtain the model evaluation cost of the corresponding candidate time series prediction model.

[0161] Next, it is determined whether the model evaluation cost of the currently traversed trained candidate time series prediction model is the minimum value among all model evaluation costs. If the model evaluation cost of the currently traversed trained candidate time series prediction model is the minimum value among all model evaluation costs, then this candidate time series prediction model is taken as the final target time series prediction model; if the model evaluation cost of the currently traversed trained candidate time series prediction model is not the minimum value, then the corresponding trained candidate time series prediction model is marked as rejected and completely released from the runtime memory.

[0162] It should be noted that when determining the model evaluation cost, the mean squared absolute error algorithm can be used to calculate the spatial Euclidean distance difference between each predicted preparation state and the label, and this spatial Euclidean distance difference can be used as the prediction error. Simultaneously, the parameter complexity is calculated by statically scanning the total number of fully connected layers and the total number of bytes occupied by the weight matrix nodes in each candidate time-series prediction model. Then, a linear normalization function is called to perform a linear weighted sum of the standardized values ​​of the prediction error and the standardized values ​​of the parameter complexity to obtain the model evaluation cost.

[0163] In this way, by obtaining the parameter complexity and combining it with the prediction error, the model evaluation cost of each candidate time series prediction model after training is determined. Then, the model with the minimum model evaluation cost is selected as the final target time series prediction model. While ensuring high-precision prediction capability for the preparation state of metallurgical materials, the memory usage and computing power of the model are strictly controlled from the underlying algorithm architecture. This effectively avoids the lag in the underlying computing network inference caused by the bloated model structure, and greatly improves the execution efficiency of the control instruction generation cycle and the smoothness of human-computer interaction in harsh industrial environments.

[0164] In practical implementation, after determining the target adjustment command for adjusting the preparation parameters during the preparation process based on the data characteristics corresponding to the preparation data of multiple modes and the initial adjustment command, the target adjustment command can be sent to the preparation equipment used to prepare the target material, so that the preparation equipment can drive and adjust the preparation parameters during the preparation process of the target material based on the received target adjustment command.

[0165] In practical implementation, based on the data characteristics corresponding to the preparation data of multiple modalities and the initial adjustment instructions, after determining the target adjustment instructions for adjusting the preparation parameters during the preparation process, the target adjustment instructions can also be sent to the preparation equipment used to prepare the target material, and the current preparation state of the target material can be obtained. The current preparation state is obtained by adjusting the preparation parameters of the target material preparation process based on the target adjustment instructions. Reference data for the target material is obtained to indicate the preparation experience rules, and the model parameters of the target time series prediction model are updated based on the reference data and the current preparation state.

[0166] It should be noted that the current preparation state refers to the objective preparation state of the target material and the preparation equipment after the preparation equipment receives and executes the issued target adjustment command. For example, after the preparation equipment adjusts the opening of a specific fuel valve according to the target adjustment command, the actual temperature or pressure value inside the reaction pool measured by the bottom temperature sensor.

[0167] Reference data refers to a set of prior knowledge summarized from long-term industrial production practice, which can characterize the constraints or operational safety limits in the preparation process. It is used to indicate the preparation experience rules during the preparation process. Here, the reference data is integrated with the actual production feedback results, jointly constraining and guiding the target time series prediction model to make inverse parameter corrections. For example, the reference data can be the expert experience summarized and recorded by senior metallurgical engineering experts, such as "when the iron-silicon ratio of the reactor slag is greater than the set extreme value, it is very easy to cause violent splashing of the internal melt."

[0168] In practical implementation, after determining the target adjustment command for adjusting the preparation parameters during the preparation process based on the data characteristics corresponding to the preparation data of multiple modalities and the initial adjustment command, the target adjustment command is sent to the preparation equipment used to prepare the target material via the underlying communication bus. Then, the preparation equipment drives the adjustment of the preparation parameters during the preparation process of the target material based on the received target adjustment command. After the adjustment is completed, the current preparation status of the target material fed back by the preparation equipment is obtained through the underlying network. At the same time, reference data for the target material used to indicate preparation experience rules is obtained from the knowledge graph database. Based on the reference data and the current preparation status, the model parameters of the target time series prediction model are updated.

[0169] It should be noted that if there is a physical or logical conflict between the current preparation state and the preparation experience rules (for example, the current preparation state indicates drastic temperature fluctuations, violating the preparation experience rule in the reference data that requires a steady temperature rise), then the current preparation state is extracted as the true supervision label feature, and the reference data is transformed into a regularization penalty term constraint in the cost function. Then, the error gradient generated by the cost function is calculated using the backpropagation algorithm, and the error gradient is used to perform fine-tuning calculations on the network weight nodes. Finally, based on the reference data and the current preparation state, the model parameters of the time series prediction model for the data processing target are updated. If there is no physical or logical conflict between the current preparation state and the preparation experience rules, then the current prediction mechanism is determined to be completely accurate and effective, thereby maintaining the model parameters of the time series prediction model for the data processing target unchanged.

[0170] It should be noted that the process of extracting the current preparation state as the true supervisory label feature, transforming the reference data into a regularization penalty term constraint in the cost function, and then using the backpropagation algorithm to calculate the error gradient generated by the cost function is as follows: First, the forward predicted numerical sequence of the target time series prediction model is extracted as the prediction vector, and the current preparation state fed back by the real sensor is extracted as the true label vector. Then, the mean square error algorithm is called to calculate the Euclidean distance difference between the two, forming a data error term. At the same time, the preparation experience rule indicated in the reference data is extracted. Assuming the preparation experience rule is "the rate of change of reactor temperature must not exceed the limit extreme value", the first time derivative of the prediction vector is calculated, and the modified linear unit activation function is called to transform the violation part exceeding the limit extreme value into a regularization penalty term. Then, the data error term and the regularization penalty term are linearly weighted and summed to obtain the final total cost function used to guide the model update. Finally, the backpropagation algorithm is used to calculate the error gradient of the total cost function with respect to the parameter matrix of each layer inside the target time series prediction model. The process of updating the model parameters of the target time series prediction model is a zero-downtime update.

[0171] Thus, by obtaining the current preparation state after adjusting the preparation parameters after generating the adjustment command, and updating the model parameters of the data processing target time-series prediction model online in conjunction with the reference data of the preparation experience rules, the actual physical execution feedback results of the preparation equipment are combined with the high-value preparation experience rules to form a dynamic data feedback closed loop. This endows the data processing system with the dynamic learning ability to adapt to the evolution of the field working environment, prevents inference distortion caused by the aging of the preparation equipment or external environmental interference, and significantly improves the accuracy of command calculation in subsequent control cycles and the long-term operational reliability of the material preparation process.

[0172] In some embodiments, after determining the target adjustment instructions for adjusting the preparation parameters during the preparation process based on the data characteristics corresponding to the preparation data of multiple modalities and the initial adjustment instructions, it is also possible to obtain the adjustment text in natural language form input by the material preparation personnel. The adjustment text is used to indicate the adjustment method for the preparation state. Based on the adjustment text, operation and maintenance instructions are generated and the operation and maintenance instructions are simulated to obtain the simulation results of the operation and maintenance instructions. Based on the operation and maintenance instructions, an adjustment scheme for the preparation state is generated, and the adjustment scheme and simulation results are output. The simulation results are used for the material preparation personnel to evaluate the adjustment scheme.

[0173] It should be noted that adjustment text refers to the character sequence entered by material preparation personnel in the form of everyday natural communication to express their intention to change a specific preparation state. For example, it is the natural language text "increase the oxygen-to-material ratio by 2 points" typed by material preparation personnel in the control terminal chat window.

[0174] In practical implementation, after determining the target adjustment instructions for regulating the preparation parameters during the preparation process, a human-computer interaction intervention channel is activated. Specifically, the adjustment text in natural language form input by the materials preparation personnel is acquired. Then, a pre-trained large-scale language model is invoked to perform contextual semantic recognition on the adjustment text, obtaining the recognition results. Based on the recognition results, machine-readable operation and maintenance instructions are generated. Next, the physical actions represented by the operation and maintenance instructions are simulated in real time to simulate the execution of the operation and maintenance instructions. After the simulation is completed, the output virtual thermodynamic state change curves and virtual product composition data are extracted as the simulation results of the operation and maintenance instructions. Subsequently, based on the extracted intent and the equipment network topology, the operation and maintenance instructions are refined and decomposed to generate an adjustment scheme for the preparation state, including specific valve switching sequences and execution timestamps. Finally, the adjustment scheme and simulation results are synchronously output and rendered on the display screen in the control room. The simulation results are used for materials preparation personnel to intuitively evaluate the safety and effectiveness of the adjustment scheme.

[0175] In this way, by acquiring the adjustment text in natural language form input by the material preparation personnel and generating operation and maintenance instructions based on it, and then simulating the execution of the operation and maintenance instructions to obtain the simulation results of the operation and maintenance instructions, and at the same time outputting the adjustment plan and simulation results for the material preparation personnel to conduct a comprehensive evaluation, it is ensured that any intervention action caused by human experience can be rigorously verified in the virtual computing environment. While greatly improving the convenience of on-site operation, it fundamentally guarantees the absolute security of the online continuous operation of the target material physical preparation system.

[0176] As an example, in a specific application scenario, the data processing method provided in this application is applied to a copper smelting production line with an annual output of 400,000 tons. Specifically, in step 1, preparation data of multiple modalities are acquired. For time-series data, 2000 measurement points are acquired via the OPC-UA protocol at a sampling period of 1 second; for relational data, 200 measurement points (e.g., laboratory reports and energy consumption invoices) are acquired via a Web-API interface. Simultaneously, handwritten data, image data, document data, and voice data are mounted and collected as needed according to the on-site operating conditions. Furthermore, for all modalities of acquired data, a unified Coordinated Universal Time (UTC) timestamp alignment operation is performed, with the timestamp alignment error strictly controlled within 1 millisecond.

[0177] Step 2 involves analyzing handwritten data, including a year's worth of operation records, material change records, and shift records (totaling 4300 pages). The image generation and recognition time per page is less than or equal to 1 second, and the field-level content extraction accuracy reaches 99.3%, reducing the monthly manual anomaly verification time to less than 30 minutes. For time-series and relational data, outlier removal is performed on 2000 collected time-series measurement points using a Hampel filter with a window length of 11 combined with the 3σ rule. Subsequently, the cleaned time-series data is aligned with 200 relational data points using 1-second grid-level interpolation. The resulting standard master data (Gold-master table) has an error rate of only 0.2%, providing reliable data for all control nodes across the production line. For image data, 600 on-site images are used to train a molding state model, enabling automatic three-classification of reactor under-blowing, over-blowing, and normal operating states, with a classification accuracy of 99.1%. In the measurement phase, a visual segmentation algorithm enables fully automated measurement of the molten liquid level (measurement error less than or equal to 1 mm) and real-time calculation of the slag ladle volume (volume error less than or equal to 2%), meeting the requirements for unmanned operation on-site. For document and audio data, it parses 2GB Word and Excel technical documents with a text extraction accuracy greater than 96%, automatically imports and corrects existing Excel metallurgical formulas, and automatically generates runnable Python script code within 10 seconds. For 20 hours of on-site duty audio recordings, the speech-to-text error rate is controlled at 2.8%, automatically extracting and generating 847 knowledge triples and pushing them into the background graph database, reducing the incremental update time of the knowledge base to within 30 seconds.

[0178] Step 3: Through the above steps, a 100GB multimodal feature dataset is finally constructed. In the model prediction stage, the matching rate between the imported metallurgical calculation formula and the actual field data is greater than 90%. The prediction accuracy for the grade of matte produced in the next 4 hours reaches 70%, and the prediction accuracy for the sulfur content of slag reaches 68%. When dealing with fluctuations in operating conditions, if the production load changes abruptly, it can automatically recalculate and output a new control target within 1 minute. By implementing this control scheme, the grade fluctuation of intermediate product (matte) has been significantly reduced by 15% (grade error range converged from 0.15% to 0.15%). In terms of human resource allocation, the number of operators in the field control room has been reduced from 7 to 6, directly saving approximately 200,000 yuan in labor costs annually. Thus, with automatic iteration of the underlying algorithm on a monthly cycle, it has been running continuously and stably in the field for more than a year, maintaining an overall service availability of over 98%.

[0179] In another specific application scenario, the data processing method provided in this application is applied to a single aluminum smelting furnace with an annual output of 40,000 tons. Specifically, in step 1, for time-series data, 80 underlying sensor measurement points are accessed via the OPC-UA protocol, with a data transmission latency of less than or equal to 200 milliseconds. For relational data, 20 business data points are accessed via the Web-API protocol; simultaneously, historical handwritten operation logs and real-time high-temperature furnace camera image data are also accessed. For handwritten data, approximately one month's worth of historical operation logs are parsed in batches, with a single page processing time of less than or equal to 1 second, and the field recognition accuracy remains at 95%. For image data, inference calculations are performed on images input from the infrared imager, with the time consumed in a single image processing cycle controlled within 5 seconds.

[0180] Step 2: Through the above steps, a 5GB multimodal dataset is formed. Based on this dataset, fully automated reasoning and identification of the physicochemical state inside the aluminum smelting furnace is achieved, without the need for human experience in the entire reasoning process. Through closed-loop optimization control, the absolute value of the overall smelting temperature stability of the individual aluminum smelting furnace has been improved to a certain extent, effectively improving the thermodynamic equilibrium state of the smelting process.

[0181] It should be noted that, as mentioned above, the target materials include metal materials in the metallurgical field, such as iron and copper, and also include chemical materials in the chemical field, such as glass. Next, the technical solutions of this application will be described using metal materials and chemical materials as examples respectively.

[0182] In some embodiments, when the target material is copper matte in the copper smelting process, preparation data of multiple modes are acquired during the preparation of copper matte. The preparation data is used to characterize the preparation state of copper matte at a target time. The preparation state characterizes the objective physical and chemical environment of the copper flash smelting furnace at a specific target time. For example, the preparation state is specifically reflected in the real-time melt temperature inside the copper flash smelting furnace being a first preset temperature and the gas pressure inside the copper flash smelting furnace being a first preset pressure, etc. This application does not limit the specifics of this embodiment.

[0183] For the preparation data of each mode, the preparation data is feature extracted by the intelligent agent corresponding to the mode to obtain the data features of the preparation data. The data features of the preparation data include the input material information of copper matte. The input material information is the initial raw material attributes that participate in the metallurgical chemical reaction. For example, the input material information is specifically reflected in the initial feeding rate of copper concentrate, the initial mass ratio of copper element in copper concentrate, and the initial injection rate of oxygen-enriched air blown into the copper flash smelting furnace. This application does not limit the specifics of these aspects.

[0184] Based on the data characteristics corresponding to the preparation data of multiple modes, the output material information is simulated and generated. The output material information is the expected index of the target product calculated by simulation through the material and energy balance equation based on the input material information. For example, the output material information is specifically reflected in the expected copper grade (copper content by mass percentage) of the simulated expected copper matte as the target copper grade, and the expected iron-silicon ratio of the generated slag as the target iron-silicon ratio, etc. This application does not limit the specifics of this embodiment.

[0185] Based on the output material information and the input material information, an initial adjustment command is generated. The initial adjustment command is used to adjust the preparation parameters after the target time in the copper matte preparation process. The initial adjustment command is a preliminary control parameter calculated by the model in reverse based on the difference between the output material information and the target process objective. For example, in order to improve the expected copper grade of the copper matte, the initial adjustment command is an operation command that instructs the bottom control equipment to increase the oxygen-enriched air introduction rate from the initial introduction rate to the first adjustment rate (the first adjustment rate is greater than the initial introduction rate) in one go.

[0186] Based on the data characteristics corresponding to the preparation data of multiple modes and the initial adjustment command, the target adjustment command for adjusting the preparation parameters in the preparation process is determined. The target adjustment command is a safety control parameter that is finally issued to the underlying control network for execution after predictive verification or logical evaluation of the initial adjustment command. For example, the forward extrapolation of the target time series prediction model found that directly increasing the oxygen-enriched air introduction rate to the first adjustment rate would cause a serious overheating phenomenon in the copper flash smelting furnace in the future period. Therefore, the target adjustment command is a control command that has been adaptively corrected by the system, instructing the underlying control equipment to smoothly increase the oxygen-enriched air introduction rate to the second adjustment rate (the second adjustment rate is between the initial introduction rate and the first adjustment rate), and simultaneously issue a control command to increase the amount of quartz flux by a first compensation amount to absorb excess heat.

[0187] In other embodiments, when the target material is molten glass in the float glass melting process, preparation data of multiple modes are acquired during the preparation of the molten glass. This preparation data is used to characterize the preparation state of the molten glass at a target time. The preparation state characterizes the objective operating conditions of the glass melting furnace at a specific target time. For example, the preparation state may include the real-time molten glass temperature in the melting zone of the glass melting furnace being a first preset temperature, and the liquid level height characteristic value inside the glass melting furnace being a first target liquid level height, etc. This application does not limit the specifics of these embodiments.

[0188] For the preparation data of each mode, features are extracted from the preparation data by an intelligent agent corresponding to the mode, resulting in data features of the preparation data. These data features include input material information for the molten glass, specifically the mixing ratio parameters and heat source energy parameters of the mixture participating in the physicochemical process of glass melting. For example, the input material information may specifically include a first preset feed rate for silica sand raw material, a second preset feed rate for soda ash raw material, and an instantaneous supply flow rate of natural gas fuel providing heat energy, which is the initial supply flow rate. This application does not limit the scope of these features.

[0189] Based on the data characteristics corresponding to the preparation data of multiple modes, output material information is simulated and generated. This output material information consists of the expected physicochemical properties of the glass melt derived from the material and energy balance equation. For example, the output material information includes the predicted dynamic viscosity of the simulated glass melt (a first dynamic viscosity value) and the predicted final mass percentage of silica in the finished glass product (a target mass percentage threshold). This application does not limit the scope of the embodiments in this regard.

[0190] Based on the output and input material information, an initial adjustment command is generated. This initial adjustment command is used to adjust the preparation parameters after a target time during the glass melt preparation process. The initial adjustment command consists of preliminary thermal control parameters calculated based on the need to adjust the dynamic viscosity of the glass melt. For example, to increase the predicted dynamic viscosity of the glass melt, the initial adjustment command might instruct the underlying execution equipment to reduce the instantaneous supply flow rate of natural gas fuel from the initial supply flow rate to a first adjusted flow rate (the first adjusted flow rate is less than the initial supply flow rate).

[0191] Based on the data characteristics corresponding to the preparation data of multiple modes and the initial adjustment instructions, target adjustment instructions are determined for adjusting the preparation parameters during the preparation process. These target adjustment instructions are the final defect prevention control parameters determined after forward state prediction and constraint verification. For example, verification by the target time-series prediction model reveals that an excessively rapid decrease in the instantaneous supply flow rate of natural gas fuel can lead to crystallization defects in the molten glass during preparation. Therefore, the target adjustment instruction is a control instruction that, after adjusting the decrease range, instructs the underlying execution equipment to slowly reduce the instantaneous supply flow rate of natural gas fuel to a second regulated flow rate (the second regulated flow rate is between the first regulated flow rate and the initial supply flow rate), and simultaneously instructs the auxiliary electric heating electrode to increase its output power to the target compensation power.

[0192] By applying the above embodiments of this application, preparation data of multiple modalities is acquired, and features of the preparation data are extracted by intelligent agents corresponding to the modalities. This breaks the limitation of traditional systems that rely solely on single time-series data. In this way, by using proprietary intelligent agents to extract features from multimodal data, the problems of complex material composition and monitoring difficulties are effectively addressed, and comprehensive perception of complex preparation conditions is achieved. At the same time, compared with a fixed or single control method, this application first simulates and generates output material information based on the data features corresponding to the preparation data of multiple modalities. Then, it generates initial adjustment commands by combining input and output information. Finally, it corrects or confirms the initial commands by combining the data features corresponding to the global multimodal preparation data, and finally determines the target adjustment command. This progressive command generation process constructs an effective and accurate closed-loop adjustment mechanism. In addition, through the above-mentioned dynamic simulation and command adjustment based on multimodal features, precise intervention can be made for preparation parameters after the target time. This not only covers complex conditions that require traditional manual intervention, but also effectively suppresses the quality fluctuations of intermediate products, thereby achieving the technical effect of significantly improving overall production efficiency, preparation parameter approximation, and the quality stability of the prepared materials.

[0193] The following description continues to illustrate the exemplary structure of the data processing apparatus 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2As shown, the software modules stored in the data processing device 455 of the memory 450 may include:

[0194] The acquisition module 4551 is used to acquire preparation data of multiple modes during the preparation of the target material, and the preparation data is used to characterize the preparation state of the target material at the target time.

[0195] The feature extraction module 4552 is used to extract features from the preparation data for each modality by using an intelligent agent corresponding to the modality to obtain the data features of the preparation data, wherein the data features of the preparation data include the input material information of the target material;

[0196] The first generation module 4553 is used to simulate and generate output material information based on the data characteristics corresponding to the preparation data of the multiple modes.

[0197] The second generation module 4554 is used to generate an initial adjustment instruction based on the output material information and the input material information. The initial adjustment instruction is used to adjust the preparation parameters after the target time in the preparation process of the target material.

[0198] The determination module 4555 is used to determine a target adjustment instruction for adjusting the preparation parameters during the preparation process based on the data characteristics corresponding to the preparation data of the multiple modalities and the initial adjustment instruction.

[0199] In some embodiments, the second generation module 4554 is further configured to determine the feedforward compensation requirement in the preparation process of the target material based on the difference between the output material information and the target output material requirements; and to generate the initial adjustment command based on the feedforward compensation requirement and the input material information by performing calculations using the material and energy balance equations used to prepare the target material.

[0200] In some embodiments, the determining module 4555 is further configured to fuse the data features of the preparation data of each modality, and use the target time series prediction model to predict the preparation state within the target time period after the target time based on the first fused feature, as the predicted preparation state;

[0201] When the predicted preparation state does not conform to the reference preparation state, the initial adjustment command is adjusted to obtain the target adjustment command. After the target adjustment command is executed, the preparation process of the target material is in the reference preparation state.

[0202] In some embodiments, the determining module 4555 is further configured to acquire multiple candidate time series prediction models and preparation sample data carrying labels, wherein the preparation sample dataset includes preparation sample data for training and preparation sample data for validation, and the labels are used to indicate the standard adjustment instructions corresponding to the preparation sample data; using the preparation sample data for training, the multiple candidate time series prediction models are trained respectively to obtain multiple trained candidate time series prediction models; the preparation sample data for validation is input into the multiple trained candidate time series prediction models respectively to obtain the prediction preparation state corresponding to each model; each prediction preparation state is compared with the label to determine the prediction error of each trained candidate time series prediction model and obtain the parameter complexity of each trained candidate time series prediction model; based on the prediction error and the parameter complexity, the model evaluation cost of each trained candidate time series prediction model is determined, and the model with the smallest model evaluation cost is selected from the trained candidate time series prediction models as the target time series prediction model.

[0203] In some embodiments, the determining module 4555 is further configured to send the target adjustment instruction to the preparation device for preparing the target material, and obtain the current preparation state of the target material; wherein the current preparation state is obtained by the preparation device adjusting the preparation parameters during the preparation process of the target material based on the target adjustment instruction; obtain reference data for the target material to indicate preparation experience rules, and update the model parameters of the target time series prediction model based on the reference data and the current preparation state.

[0204] In some embodiments, the first generation module 4553 is further configured to obtain component data and weight data associated with the preparation process from the data features corresponding to the preparation data of the multiple modalities, as the input material information;

[0205] Based on the input material information, the material and energy balance is calculated by running executable code to obtain the output material information. The executable code is obtained by converting the material and energy balance equation used to prepare the target material.

[0206] In some embodiments, the material and energy balance equations are extracted from the preparation data of the multiple modes.

[0207] In some embodiments, the first generation module 4553 is further configured to perform three-dimensional point cloud reconstruction on the image data collected from the target material output container in the preparation data of the multiple modalities to obtain the volume data of the material in the output container; determine the material flow data based on the volume data; and write the material flow data back to the executable code to participate in the material and energy balance calculation to obtain the output material information.

[0208] In some embodiments, the acquisition module 4551 is further configured to acquire at least two of the following preparation data during the preparation of the target material: handwritten data, image data, time-series data, relational data, document data, and voice data; the handwritten data includes handwritten records by the material preparation personnel regarding the preparation status of the target material during the preparation process; the image data is obtained by acquiring images of the preparation status of the target material at the target time; the time-series data includes parameter values ​​of state parameters indicating the preparation status at different times; the relational data includes environmental state data associated with the preparation process and component data in the preparation results at the target time; the document data is used to record the constraints of the preparation process and the topological relationships between multiple devices used to prepare the target material; the voice data is oral audio recorded by the material preparation personnel, which is used to describe the preparation status.

[0209] In some embodiments, the feature extraction module 4552 is further configured to: when the preparation data includes the handwritten data, perform content recognition on the handwritten data using a corresponding intelligent agent and multiple recognition methods to obtain content recognition results; group multiple content recognition results to obtain at least one recognition result group, wherein the recognition result group includes the same content recognition results; take the recognition result group with the most content recognition results as the target recognition result group, and perform feature extraction on the content recognition results in the target recognition result group to obtain the data features of the handwritten data; when the preparation data includes the image data, perform feature extraction on the image data using a relative intelligent agent and multiple recognition methods ... The corresponding intelligent agent identifies the image data to obtain process parameters of the preparation process, which are used to indicate the preparation state; features are extracted from the process parameters to obtain data features of the preparation data; when the preparation data includes time-series data with different state parameters, a statistical benchmark value is obtained for the time-series data with each state parameter, and the corresponding intelligent agent filters out abnormal data in the time-series data based on the statistical benchmark value to obtain filtered time-series data; missing data in the filtered time-series data is numerically interpolated to obtain target time-series data; and the target time-series data with different state parameters are clustered. The data is aggregated to obtain aggregated time-series data, and features are extracted from the aggregated time-series data to obtain the data features of the prepared data. When the prepared data includes the relational data and the relational data is a low-frequency discrete distribution, the constraints corresponding to the relational data are obtained. Under the constraints of the constraints, the low-frequency discrete distribution relational data is aligned by a corresponding intelligent agent to obtain aligned relational data. Features are extracted from the aligned relational data to obtain the data features of the prepared data. When the prepared data includes document data, the document data is parsed by a corresponding intelligent agent to obtain the image portion of the document data and the document data itself. The document data includes: a text portion; an image portion; a document ...Target words are extracted from the target text content. These target words include process entities and the relationships between different process entities. Based on the target words, a multi-tuple corresponding to the speech data is constructed. Each multi-tuple includes a corresponding preparation device. Feature extraction is performed on the multi-tuple to obtain the data features of the preparation data.

[0210] In some embodiments, the preparation data for each modality includes multiple sub-data. The feature extraction module 4552 is further configured to, for each modality's preparation data, extract features from each sub-data using an agent corresponding to the modality to obtain initial features of the sub-data; select at least one initial feature associated with the reference preparation state as a benchmark feature; perform correlation analysis between the other initial features and the benchmark feature to obtain corresponding correlation analysis results; and filter multiple initial features based on the correlation analysis results to obtain multiple intermediate features; and combine the multiple intermediate features to obtain the data features of the preparation data.

[0211] In some embodiments, the feature extraction module 4552 is further configured to filter invalid data in the plurality of sub-data by an agent corresponding to the modality to obtain the filtered sub-data; and to extract features from the filtered sub-data to obtain the initial features of the sub-data.

[0212] In some embodiments, the acquisition module 4551 is further configured to acquire adjustment text in natural language form input by the material preparation personnel, the adjustment text being used to indicate the adjustment method for the preparation state; based on the adjustment text, generate operation and maintenance instructions, and simulate the execution of the operation and maintenance instructions to obtain simulation results of the operation and maintenance instructions; based on the operation and maintenance instructions, generate an adjustment scheme for the preparation state, and output the adjustment scheme and the simulation results, the simulation results being used by the material preparation personnel to evaluate the adjustment scheme.

[0213] In some embodiments, the target material is a metallic material from the metallurgical field.

[0214] In some embodiments, the modules in the aforementioned data processing device employ a two-layer collaborative architecture: a modal layer and a functional layer. Specifically, the feature extraction module and the intelligent agents it calls, corresponding to each modality, constitute the modal layer, primarily used for specialized parsing, alignment, and feature extraction of heterogeneous data (such as images, speech, time series, handwriting, etc.) with different physical manifestations. The acquisition module, the first generation module, the second generation module, and the determination module together constitute the functional layer, primarily used for unified scheduling of global data, logical deduction of material and energy balance, and generation and issuance of closed-loop control commands. Through this architectural division of the modal layer and functional layer, deep decoupling between the underlying heterogeneous data processing and the upper-layer core control logic is achieved. On the one hand, the loosely coupled design of the modal layer supports the on-demand mounting of intelligent agents based on on-site hardware conditions, improving the system's compatibility and scalability. On the other hand, the functional layer shields the differences in underlying data and protocols, focusing solely on global logical deduction based on standardized features. This "high cohesion, low coupling" architecture effectively overcomes the challenge of aligning multi-source data, significantly improving computational efficiency and the accuracy and reliability of adjustment commands.

[0215] This application provides a computer program product, which includes computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the data processing method provided in this application.

[0216] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the data processing method provided in this application, for example, such as... Figure 3 The data processing method is shown.

[0217] In some embodiments, the computer-readable storage medium may be a read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disk, or CD-ROM, etc.; or it may be a device that includes one or any combination of the above-mentioned memories.

[0218] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0219] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hypertext Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0220] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0221] It should be noted that in the embodiments of this application, data related to the acquisition of preparation data and user-inputted adjustment text are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0222] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A data processing method, characterized in that, The method includes: During the preparation of the target material, preparation data of multiple modes are acquired, and the preparation data is used to characterize the preparation state of the target material at the target time. For the preparation data of each modality, the preparation data is feature extracted by the intelligent agent corresponding to the modality to obtain the data features of the preparation data, and the data features of the preparation data include the input material information of the target material; Based on the data characteristics corresponding to the preparation data of the various modalities, output material information is simulated and generated; Based on the output material information and the input material information, an initial adjustment command is generated, which is used to adjust the preparation parameters after the target time in the preparation process of the target material; Based on the data characteristics corresponding to the preparation data of the multiple modalities and the initial adjustment instructions, a target adjustment instruction for adjusting the preparation parameters during the preparation process is determined.

2. The method according to claim 1, characterized in that, The step of generating an initial adjustment command based on the output material information and the input material information includes: Based on the difference between the output material information and the target output material requirements, the feedforward compensation requirements in the target material preparation process are determined; Based on the feedforward compensation requirements and the input material information, the initial adjustment command is generated by calculating using the material and energy balance equations used to prepare the target material.

3. The method according to claim 1, characterized in that, Based on the data characteristics corresponding to the preparation data of the multiple modalities and the initial adjustment command, a target adjustment command for adjusting the preparation parameters during the preparation process is determined, including: The data features of the preparation data of each modality are fused, and the preparation state within the target time period after the target time is predicted by the target time prediction model based on the first fused feature. When the predicted preparation state does not conform to the reference preparation state, the initial adjustment command is adjusted to obtain the target adjustment command. After the target adjustment command is executed, the preparation process of the target material is in the reference preparation state.

4. The method according to claim 3, characterized in that, Before predicting the preparation state within the target time period after the target time using the target time series prediction model based on the first fusion feature obtained by fusion, the method further includes: Multiple candidate time series prediction models and labeled preparation sample data are obtained. The preparation sample dataset includes preparation sample data for training and preparation sample data for validation. The labels are used to indicate the standard adjustment instructions corresponding to the preparation sample data. Using the prepared sample data for training, the multiple candidate time series prediction models are trained respectively to obtain multiple trained candidate time series prediction models; The preparation sample data used for verification is input into the multiple trained candidate time series prediction models respectively, and the prediction preparation state corresponding to each model is obtained respectively; The prediction preparation states are compared with the labels to determine the prediction error of each trained candidate time series prediction model and to obtain the parameter complexity of each trained candidate time series prediction model. Based on the prediction error and the parameter complexity, the model evaluation cost of each of the trained candidate time series prediction models is determined, and the model with the lowest model evaluation cost is selected from the trained candidate time series prediction models as the target time series prediction model.

5. The method according to claim 4, characterized in that, After determining the target adjustment command for adjusting the preparation parameters during the preparation process based on the data characteristics corresponding to the preparation data of the multiple modalities and the initial adjustment command, the method further includes: The target adjustment command is sent to the preparation equipment used to prepare the target material, and the current preparation status of the target material is obtained; The current preparation state is obtained by adjusting the preparation parameters during the preparation process of the target material based on the target adjustment command using the preparation equipment. Obtain reference data for the target material to indicate empirical rules for preparation, and update the model parameters of the target time series prediction model based on the reference data and the current preparation state.

6. The method according to claim 1, characterized in that, The data features corresponding to the preparation data of the multiple modalities are used to simulate and generate output material information, including: From the data features corresponding to the preparation data of the multiple modalities, the component data and weight data associated with the preparation process are obtained as the input material information; Based on the input material information, the material and energy balance is calculated by running executable code to obtain the output material information. The executable code is obtained by converting the material and energy balance equation used to prepare the target material.

7. The method according to claim 2 or 6, characterized in that, The material and energy balance equations are extracted from the preparation data of the various modes.

8. The method according to claim 6, characterized in that, The step of performing material and energy balance calculations based on the input material information, and obtaining the output material information by running executable code, includes: For the image data of the target material output container collected from the preparation data of the various modalities, three-dimensional point cloud reconstruction is performed to obtain the volume data of the material inside the output container; Based on the volume data, the material flow rate data is determined, and the material flow rate data is written back to the executable code to participate in the material and energy balance calculation, thereby obtaining the output material information.

9. The method according to claim 1, characterized in that, During the preparation of the target material, preparation data in multiple modes are acquired, including: During the preparation of the target material, at least two of the following preparation data are acquired: handwritten data, image data, time-series data, relational data, document data, and voice data. The handwritten data includes handwritten records by material preparation personnel regarding the preparation status of the target material during the preparation process; The image data is obtained by acquiring images of the preparation state of the target material at the target time; The time-series data includes parameter values ​​of state parameters used to indicate the preparation state at different times; The relational data includes environmental state data associated with the preparation process and component data in the preparation results at the target time. The document data is used to record the constraints of the preparation process and the topological relationships between multiple devices used to prepare the target material; The voice data is audio recordings of the material preparation personnel, which are used to describe the preparation status.

10. The method according to claim 9, characterized in that, For the preparation data of each modality, feature extraction is performed on the preparation data by an agent corresponding to the modality to obtain the data features of the preparation data, including: When the prepared data includes the handwritten data, for the handwritten data, a corresponding intelligent agent uses multiple recognition methods to perform content recognition on the handwritten data to obtain content recognition results; multiple content recognition results are grouped to obtain at least one recognition result group, and the recognition result group includes the same content recognition results; the recognition result group with the most content recognition results is taken as the target recognition result group, and feature extraction is performed on the content recognition results in the target recognition result group to obtain the data features of the handwritten data; When the preparation data includes the image data, the corresponding intelligent agent identifies the image data to obtain the process parameters of the preparation process, which are used to indicate the preparation state; feature extraction is performed on the process parameters to obtain the data features of the preparation data. When the prepared data includes time-series data with different state parameters, for each type of time-series data with different state parameters, a statistical benchmark value is obtained, and an abnormal data in the time-series data is filtered based on the statistical benchmark value by a corresponding intelligent agent to obtain filtered time-series data; missing data in the filtered time-series data is numerically interpolated to obtain target time-series data; target time-series data with different state parameters are aggregated to obtain aggregated time-series data, and feature extraction is performed on the aggregated time-series data to obtain the data features of the prepared data; When the prepared data includes the relational data and the sampling frequency of the relational data is lower than a preset frequency threshold, the constraint conditions corresponding to the relational data are obtained; through the corresponding intelligent agent, under the constraint conditions, the relational data with a sampling frequency lower than the preset frequency threshold is aligned to obtain the aligned relational data; feature extraction is performed on the aligned relational data to obtain the data features of the prepared data; When the preparation data includes the document data, the document data is parsed by a corresponding intelligent agent to obtain the image portion and the text portion of the document data; the image portion is identified to obtain the equipment image used to prepare the target material in the document data, and feature extraction is performed on the equipment image to obtain the image features in the document data; text parameters associated with the preparation process are extracted from the text portion, and feature extraction is performed on the text parameters to obtain the text features in the document data; the image features and the text features are determined as the data features of the preparation data; When the preparation data includes the speech data, the speech data is converted into text by a corresponding intelligent agent to obtain the text content corresponding to the speech data. Terminology correction is then performed on the text content to obtain the target text content. Target words are extracted from the target text content. These target words include process entities and the relationships between different process entities. Based on the target words, a tuple corresponding to the speech data is constructed. This tuple includes the corresponding preparation equipment. Feature extraction is performed on the tuple to obtain the data features of the preparation data.

11. The method according to claim 3, characterized in that, The preparation data for each modality includes multiple sub-data. For each modality's preparation data, feature extraction is performed on the preparation data by an agent corresponding to that modality to obtain the data features of the preparation data, including: For the preparation data of each modality, the initial features of each sub-data are obtained by extracting features from each sub-data through the intelligent agent corresponding to the modality. At least one of the initial features associated with the reference preparation state is selected as a reference feature. Correlation analysis is performed on the other initial features other than the reference feature with the reference feature to obtain the corresponding correlation analysis results. Based on the correlation analysis results, multiple initial features are filtered to obtain multiple intermediate features. The data features of the prepared data are obtained by combining the multiple intermediate features.

12. The method according to claim 11, characterized in that, The step of extracting features from each of the sub-data using an agent corresponding to the modality to obtain the initial features of the sub-data includes: The agent corresponding to the modality filters out invalid data from the multiple sub-data to obtain the filtered sub-data. Feature extraction is performed on the filtered sub-data to obtain the initial features of the sub-data.

13. The method according to claim 1, characterized in that, After determining the target adjustment command for adjusting the preparation parameters during the preparation process based on the data characteristics corresponding to the preparation data of the multiple modalities and the initial adjustment command, the method further includes: The adjustment text, in natural language form, is obtained from the material preparation personnel and is used to indicate the adjustment method for the preparation state. Based on the adjustment text, operation and maintenance instructions are generated, and the operation and maintenance instructions are simulated and executed to obtain the simulation results of the operation and maintenance instructions; Based on the operation and maintenance instructions, an adjustment plan for the preparation state is generated, and the adjustment plan and the simulation results are output. The simulation results are used by the material preparation personnel to evaluate the adjustment plan.

14. The method according to claim 1, characterized in that, The target material is a metallic material from the metallurgical field.

15. A data processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire preparation data of multiple modes during the preparation of the target material, and the preparation data is used to characterize the preparation state of the target material at the target time. The feature extraction module is used to extract features from the preparation data for each modality by using an agent corresponding to the modality to obtain the data features of the preparation data. The data features of the preparation data include the input material information of the target material. The first generation module is used to simulate and generate output material information based on the data characteristics corresponding to the preparation data of the multiple modes; The second generation module is used to generate an initial adjustment instruction based on the output material information and the input material information. The initial adjustment instruction is used to adjust the preparation parameters after the target time in the preparation process of the target material. The determination module is used to determine a target adjustment instruction for adjusting the preparation parameters during the preparation process based on the data characteristics corresponding to the preparation data of the multiple modalities and the initial adjustment instruction.

16. An electronic device, characterized in that, include: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data processing method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that, It stores computer-executable instructions or computer programs for inducing a processor to execute and implement the data processing method according to any one of claims 1 to 14.

18. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the data processing method according to any one of claims 1 to 14 is implemented.

Images