A mining equipment state prediction method based on digital twinning and multi-modal fusion

Abstract

The application discloses a kind of based on digital twin and multi-modal fusion's mining equipment state prediction method, it is related to intelligent operation and maintenance technical field, including, multi-modal alignment working condition window sequence is input to digital twin mechanism link, carry out the same window ideal response deduction, output ideal multi-modal response, and construct twin dynamic residual error, simultaneously split residual error structure event, generate residual error event token set;Based on residual error event token set positioning multi-modal data segment, and execute cross-modal attention alignment, simultaneously combine mechanical topological observation mapping table, carry out topological propagation aggregation, generate topological constraint fusion representation set;Topological constraint fusion representation set is input to graph neural network, combine mechanical topological graph and carry out message passing, output component state and whole machine state, and encapsulate as mining equipment state prediction set.The application realizes the structural analysis of twin dynamic residual error, act on accurate positioning abnormal event, improve state prediction sensitivity and timeliness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent operation and maintenance, and particularly relates to a mining and excavation equipment state prediction method based on digital twinning and multi-modal fusion. BACKGROUND

[0002] Under the background of intelligent mines and digital transformation of heavy equipment, mining and excavation equipment state prediction technology has become a key support for ensuring operation safety and improving operation and maintenance efficiency. In recent years, with the rapid development of industrial internet of things, digital twinning and artificial intelligence technology, equipment state monitoring has gradually evolved from single sensor threshold alarm to multi-source information fusion and mechanism-data dual driving. Existing researches are mostly focused on fault diagnosis models based on single modal signals such as vibration, temperature or current, and some schemes try to introduce shallow multi-modal fusion strategies such as feature splicing or weighted average to enhance state representation capability. At the same time, digital twinning technology shows potential in simulation deduction and anomaly detection by constructing a virtual mapping of physical equipment.

[0003] The existing technology has two deficiencies: first, the multi-modal data acquisition and fusion process generally ignores the internal mechanical topological structure of the equipment and its dynamic transmission path, resulting in rough correlation modeling between different modalities and inability to effectively capture the fault propagation mechanism across components and modalities; second, traditional digital twinning models mostly use static or quasi-static response mechanisms, lack of structured analysis capability for dynamic residual errors between “actual observation-ideal deduction”, and cause fuzzy positioning and lagging prediction of abnormal events. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a mining and excavation equipment state prediction method based on digital twinning and multi-modal fusion to solve the problems of lack of topological constraints in multi-modal fusion and inability to structure the dynamic residual errors of digital twinning.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] The application provides a mining equipment state prediction method based on digital twinning and multi-modal fusion, which comprises the following steps: binding multi-modal collection points to topological nodes and topological edges, constructing a mechanical topological graph, registering component coupling transmission relationship to obtain a digital twinning mechanism link, generating a mechanical topological observation mapping table by topological edge weight labeling; collecting multi-modal data according to the mechanical topological observation mapping table, and performing sampling rate alignment and clock drift correction according to a unified time criterion to generate a multi-modal alignment working condition window sequence; inputting the multi-modal alignment working condition window sequence into the digital twinning mechanism link to perform window ideal response deduction, outputting ideal multi-modal responses, and constructing a twinning dynamic residual error, and simultaneously cutting residual error structure events to generate a residual error event token set; positioning multi-modal data segments based on the residual error event token set, performing cross-modal attention alignment, and combining the mechanical topological observation mapping table to perform topological propagation aggregation to generate a topological constraint fusion representation set; inputting the topological constraint fusion representation set into a graph neural network, combining the mechanical topological graph to perform message passing, outputting component states and whole machine states, and packaging into a mining equipment state prediction set.

[0008] As a preferred scheme of the mining equipment state prediction method based on digital twinning and multi-modal fusion, the mechanical topological graph is constructed by the following steps,

[0009] Read the mining equipment component list and the multi-modal collection point list, organize them into a bindable item set, and perform binding field alignment and installation pose analysis to generate a point binding preparation list;

[0010] Based on the point binding preparation list, perform port alignment to form edges, establish topological nodes and topological edges, and bind collection point identifiers to generate a mechanical topological graph.

[0011] As a preferred scheme of the mining equipment state prediction method based on digital twinning and multi-modal fusion, the mechanical topological observation mapping table is generated by the following steps,

[0012] Sort the edge types of the mechanical topological graph and solidify the transmission direction to obtain the component coupling transmission relationship, and concatenate it into a callable chain item to generate a digital twinning mechanism link;

[0013] Convert the digital twinning mechanism link into edge weight labeling values, perform edge weight labeling normalization, and merge and hang the collection point identifiers to generate a mechanical topological observation mapping table.

[0014] As a preferred scheme of the mining equipment state prediction method based on digital twinning and multi-modal fusion, the multi-modal data is collected according to the mechanical topological observation mapping table by the following steps,

[0015] Based on the mechanical topology observation mapping table, the collection point position identification is grouped, and the sampling rate configuration and channel clock source description are arranged to obtain a multi-modal collection arrangement list;

[0016] According to the multi-modal collection arrangement list, multi-modal data is collected, time markers and frame sequence numbers are encapsulated, abnormal frames are removed, and a multi-modal original collection stream set is generated.

[0017] As a preferred scheme of the mining equipment state prediction method based on digital twin and multi-modal fusion, the generation of the multi-modal alignment working condition window sequence is as follows,

[0018] Based on the multi-modal original collection stream set, a unified time scale sequence is established, time scale aggregation slicing and time scale point interpolation are performed, and a multi-modal time alignment segment set is generated.

[0019] From the multi-modal time alignment segment set, an adjacent time marker difference sequence is extracted, clock drift rollback correction is performed, and multi-modal alignment segments are encapsulated to generate a multi-modal alignment working condition window sequence.

[0020] As a preferred scheme of the mining equipment state prediction method based on digital twin and multi-modal fusion, the generation of the multi-modal alignment working condition window sequence is as follows,

[0021] From the multi-modal alignment working condition window sequence, time window identifiers and unified time scale index ranges are extracted, and collection point position identification mapping is performed in combination with the mechanical topology observation mapping table to generate a same window deduction input package;

[0022] According to the same window deduction input package, a digital twin mechanism link is driven to perform chain item scheduling deduction to generate an ideal multi-modal response;

[0023] The ideal multi-modal response and the multi-modal time alignment segment set are correspondingly related, and same window cutting alignment is performed to generate a same window ideal response reference set.

[0024] As a preferred scheme of the mining equipment state prediction method based on digital twin and multi-modal fusion, the generation of the residual event token set is as follows,

[0025] Based on the same window ideal response reference set, residual multi-modal paragraph extraction is performed, twin dynamic residuals are constructed, residual structure events are divided along the unified time scale index range, and a residual event item set is generated.

[0026] The residual event item set is encapsulated as a token load field and a topology priori field according to the time window identifier and the topology node identifier, and written into an event sequence index to generate a residual event token set.

[0027] As a preferred scheme of the mining equipment state prediction method based on digital twinning and multi-modal fusion, the cross-modal attention alignment is performed as follows,

[0028] According to the residual event token set, the segment playback range is extracted, the collection point position identifier is mapped, and the multi-modal alignment segment is cut in the multi-modal alignment working condition window sequence to generate an event segment positioning list;

[0029] The event anchor point trajectory is extracted from the multi-modal alignment segment of the event segment positioning list, and the cross-modal attention alignment is performed to generate a cross-modal alignment event representation set.

[0030] As a preferred scheme of the mining equipment state prediction method based on digital twinning and multi-modal fusion, the cross-modal attention alignment is performed as follows,

[0031] The cross-modal alignment event representation set is hung to the mechanical topology observation mapping table according to the topology node identifier and the topology edge identifier, and the propagation message backfill update is performed to generate a topology propagation aggregated representation set;

[0032] The node representation entries and edge representation entries of the topology propagation aggregated representation set are arranged, and the time window identifier and the event sequence index are attached to generate a topology constraint fusion representation set.

[0033] As a preferred scheme of the mining equipment state prediction method based on digital twinning and multi-modal fusion, the cross-modal attention alignment is performed as follows,

[0034] The topology constraint fusion representation set is arranged in batches according to the graph structure, and is arranged into node input sequence and edge input sequence, and a topology index table is established, and a cross-modal alignment weight summary is merged to generate a topology input arrangement package;

[0035] The topology input arrangement package is input into the graph neural network, and the connection relationship of the mechanical topology graph is aligned, and the directional message passing is initiated, and the strength modulation and type gating are performed to output a topology reasoning update representation set;

[0036] Based on the topology reasoning update representation set, the component state is output according to the mapping relationship between the component identifier and the topology node identifier, and the whole machine state is output according to the topology hierarchical relationship of the mechanical topology graph to generate a mining equipment state prediction set.

[0037] The application has the beneficial effects that: by constructing a mechanical topology observation mapping table with a weight label, the multi-modal data and the device mechanism are structurally bound, which provides a topological constraint for fusion and improves the cross-modal alignment physical consistency; by generating a residual event token set, the structural analysis of the twin dynamic residual is realized, which accurately locates the abnormal event and improves the state prediction sensitivity and timeliness. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0039] Figure 1 The flowchart of the mining equipment state prediction method based on digital twin and multi-modal fusion.

[0040] Figure 2 The twin dynamic residual energy evolution with working condition and residual event token triggering schematic diagram.

[0041] Figure 3 The cross-modal event anchor point consistency deviation changes with clock drift comparison diagram.

[0042] Figure 4 The twin dynamic residual event boundary alignment effect comparison schematic diagram. DETAILED DESCRIPTION

[0043] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0044] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0045] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. In this specification, "in one embodiment" appearing in different places does not mean the same embodiment, nor is it an independent or selective embodiment that excludes other embodiments.

[0046] REFERENCE Figures 1-4For an embodiment of the present application, the embodiment provides a mining equipment state prediction method based on digital twinning and multi-modal fusion, comprising the following steps:

[0047] S1: bind the multi-modal acquisition point to the topological node and the topological edge, construct a mechanical topology graph, and register the component coupling transmission relationship to obtain a digital twinning mechanism link, and at the same time, perform topological edge weight labeling to generate a mechanical topology observation mapping table;

[0048] S1.1: read the mining equipment component list and the multi-modal acquisition point list, sort into a bindable item set, and perform bind field alignment and installation pose analysis to generate a point binding preparation list;

[0049] Further, read the mining equipment component list and de-duplicate the component list items according to the component identifier, check the completeness of the pair of connection port identifiers, and mark the incomplete items as to-be-completed items, read the multi-modal acquisition point list and merge the acquisition channel configuration field set and the installation location description according to the acquisition point identifier, extract the coordinate reference and orientation reference in the installation location description, and perform installation pose analysis to obtain a pose expression item, perform bind field alignment on the component identifier of the mining equipment component list, the connection port identifier, and the acquisition point identifier of the multi-modal acquisition point list, the installation pose analysis record, establish a bindable mapping relationship of the acquisition point identifier to the candidate component identifier and the candidate connection port identifier, and register the alignment state, and gather the bindable mapping relationship, the alignment state and the pose expression item, and sort and arrange them according to the acquisition point identifier, to generate a point binding preparation list.

[0050] It should be noted that the coordinate reference and the orientation reference are extracted and solidified from the installation face reference and the port pointing reference bound to the component identifier in the installation location description, and the coordinate reference determines the origin and the axial direction by the installation face reference, and the orientation reference determines the positive direction and the rotational direction by the port pointing reference.

[0051] S1.2: based on the point binding preparation list, perform port alignment to form an edge, establish a topological node and a topological edge, and bind the acquisition point identifier, to generate a mechanical topology graph;

[0052] Further, the extractable mapping relationship of the binding preparation list is extracted according to the point position binding, candidate component identifier and candidate connection port identifier are registered as topological node candidate entries, the topological node entries are generated by merging the candidate component identifiers, and the node index is backfilled, the starting topological node identifier and the terminal topological node identifier are determined by traversing the candidate connection port identifier pair, the edge alignment is performed, the edge is aligned, the starting topological node identifier and the terminal topological node identifier are determined, and the topological edge entries are generated by registering the transfer direction description, the priority decision is performed on the port alignment conflict entries according to the alignment state and the pose expression entries, and the closable connection relationship is retained, the point position binding preparation list is traversed according to the collection point position identifier, the collection point position identifier is hung to the corresponding topological node entry or the corresponding topological edge entry according to the extractable mapping relationship, the pose expression entry is retained as the point position space positioning information, the collection point position identifier of the missed connection relationship is registered as a hanging point position entry, and the candidate component identifier is retained for subsequent completion, and a mechanical topology graph is generated.

[0053] S1.3: The edge type sorting and transfer direction solidification arrangement are performed on the mechanical topology graph, the component coupling transfer relationship is obtained, the chain entries are concatenated, the digital twin mechanism link is generated;

[0054] Further, the topological edge type entries, the starting topological node identifier and the terminal topological node identifier are extracted along the mechanical topology graph, the edge type sorting is performed according to the topological edge type entries to form a sorting edge list, and the transfer direction description is solidified to complete the transfer direction solidification arrangement; the topological node identifier and the coupling constraint point set corresponding to the topological edge type entry at both ends of the topological edge are arranged for each entry in the sorting edge list, the adjacent topological edges are concatenated into a transfer link according to the transfer direction description, and the component coupling transfer relationship is merged; the chain entry identifier, the chain entry input field set and the chain entry output field set are encapsulated for each entry in the component coupling transfer relationship, the chain entry call order mark is registered, and the chain entry is arranged into the digital twin mechanism link according to the call order mark.

[0055] It should be noted that the coupling constraint point set is a summary of the transfer constraints and boundary constraints that must be consistent in the component coupling transfer relationship corresponding to the topological edge type entry, which is used to limit the input-output matching range and directional transfer conditions when the transfer link is concatenated.

[0056] S1.4: The digital twin mechanism link is converted into an edge weight label value, the edge weight label is normalized, the range is unified, and the collection point position identifier is merged and hung, and a mechanical topology observation mapping table is generated;

[0057] Further, chain entries of the digital twin mechanism link are traversed, and topological edge identifiers and transmission intensity descriptions associated with the chain entries are extracted. The transmission intensity descriptions are converted into edge weight scale values according to a uniform weight scale, and the edge weight scale values are registered with source tags. The edge weight scale values of different edge type sets are subjected to edge weight scale normalization to eliminate dimensional and order-of-magnitude differences, and a comparable ordering relationship is preserved. The edge weight scale values are backfilled into corresponding topological edge identifier entries, and form a topological constraint field set together with the transmission direction descriptions. According to the observation anchor entries of the digital twin mechanism link, collection point identifiers are extracted and topological node identifiers or topological edge identifiers are attached. The collection point identifiers are merged and attached to the topological constraint field set, and a traceable mapping key is registered. Topological node identifiers, topological edge identifiers, collection point identifiers, and edge weight scale values are aggregated and sorted according to the topological node identifiers and the topological edge identifiers, and a mechanical topological observation mapping table is generated.

[0058] It should be noted that the edge weight scale values are obtained by interval mapping of the transmission intensity descriptions according to the transmission intensity indicators corresponding to the edge types, and the edge weight scale normalization is completed by using quantile scaling within the same edge type set. The dimensionalities of the transmission intensity indicators are given by the stiffness indicators, the damping indicators, and the energy transmission coefficient indicators in the coupling constraint point set, and are written into the edge weight scale value source tags.

[0059] S2: According to the mechanical topological observation mapping table, multi-modal data are collected, and sampling rate alignment and clock drift correction are performed according to a uniform time criterion, to generate a multi-modal alignment working condition window sequence.

[0060] S2.1: Based on the mechanical topological observation mapping table, collection point identifier grouping is performed, and a multi-modal collection scheduling list is obtained by arranging sampling rate configurations and channel clock source descriptions.

[0061] Further, according to the mechanical topological observation mapping table, collection point identifiers are traversed, and topological node identifiers and topological edge identifiers associated with the collection point identifiers are extracted. The collection point identifiers are grouped according to the topological node identifiers and the topological edge identifiers, and a grouping key is registered, so that the collection point identifiers corresponding to the same topological node identifier or the same topological edge identifier are grouped into the same group. For each group, sampling rate configurations and channel clock source descriptions are aggregated and subjected to consistency checking. For groups with inconsistent sampling rate configurations, sampling rate alignment prompt entries are registered. For groups with inconsistent channel clock source descriptions, clock anchoring candidate entries are registered. The grouping key, the collection point identifier, the sampling rate configuration, and the channel clock source description are combined into a collection scheduling entry, and the collection scheduling order is determined according to the transmission direction descriptions of the topological node identifiers and the topological edge identifiers. The collection scheduling entries and the prompt entries are aggregated and arranged into an ordered list, and a multi-modal collection scheduling list is generated.

[0062] S2.2: According to the multi-modal acquisition arrangement list, acquire multi-modal data, and encapsulate the time marker and frame sequence number field while rejecting abnormal frames to generate a multi-modal original acquisition stream set;

[0063] Further, according to the acquisition arrangement sequence of the multi-modal acquisition arrangement list, access the acquisition channel corresponding to the acquisition point position identifier and start the acquisition session in sequence, configure the pull payload field set according to the sampling rate for each acquisition channel, and generate the time marker from the channel clock source description value, at the same time, establish the incremental frame sequence number field for each acquisition channel and encapsulate the acquisition point position identifier, the time marker, the frame sequence number field and the payload field set as an acquisition entry; perform frame format constraint item checking on the acquisition entry and mark the format mismatch frame, perform time marker continuity checking on the acquisition entry and mark the time jump frame, perform frame sequence number continuity checking on the acquisition entry and mark the repeated frame and gap frame, merge the format mismatch frame, the time jump frame, the repeated frame and the gap frame into the abnormal frame and remove it from the acquisition channel output sequence, at the same time, register the continuous playable frame sequence number interval to maintain the traceable playback capability, converge the acquisition entry sequence of each acquisition channel and archive according to the acquisition point position identifier, and generate the multi-modal original acquisition stream set.

[0064] It should be noted that the channel clock source description generates the time marker according to the time source priority (for example, PTP is preferred, GPS is second, and NTP is third), performs low-pass anti-aliasing filtering on the acquisition channel with reduced sampling rate before resampling, and performs linear interpolation point filling on the acquisition channel with increased sampling rate according to the uniform time scale sequence.

[0065] S2.3: Based on the multi-modal original acquisition stream set, establish a uniform time scale sequence, and perform time scale aggregation slicing and time scale point filling interpolation to generate a multi-modal time alignment segment set;

[0066] Further, traverse the acquisition point position identifier and extract the time marker sequence and frame number field according to the multi-modal original acquisition stream set, filter the uniform time caliber anchoring channel according to the time marker integrity and continuous playback frame number interval stability, extract the time marker of the uniform time caliber anchoring channel as a reference scale, and generate a uniform time scale sequence covering the acquisition session range according to the reference scale; map the acquisition entry sequence corresponding to each acquisition point position identifier in the multi-modal original acquisition stream set to the uniform time scale sequence, perform time scale aggregation slicing on the acquisition entry sequence with a sampling rate higher than the uniform time scale sequence according to the scale boundary, and gather the payload field set for each scale, perform time scale point interpolation on the acquisition entry sequence with a sampling rate lower than the uniform time scale sequence according to the scale gap, register the interpolation source segment boundary, and uniformly backfill the time marker and scale index to form a one-to-one entry for each acquisition point position identifier under the uniform time scale sequence, cut the continuous alignment section according to the acquisition point position identifier and scale index range, and register the section boundary to generate a multi-modal time alignment segment set.

[0067] S2.4: Extract the adjacent time marker difference sequence from the multi-modal time alignment segment set, and perform clock drift rollback correction, while packaging the multi-modal alignment segment to generate a multi-modal alignment working condition window sequence;

[0068] Further, traverse the multi-modal time alignment segment set according to the acquisition point position identifier and extract the time marker sequence, calculate the adjacent time marker difference based on the time marker sequence, and register the difference sequence statistical summary, generate the drift rollback amount according to the linear offset trend of the difference sequence, and perform clock drift rollback correction on the time marker sequence, realign the corrected time marker sequence to the uniform time scale sequence and update the scale index mapping relationship, while registering the drift anomaly prompt entry for the abnormal jump section of the difference sequence before and after correction; generate the time window identifier and uniform time scale index range on the uniform time scale sequence according to the predetermined working condition window length and sliding step, cut the multi-modal alignment segment covered by each uniform time scale index range into multi-modal alignment segments and merge them according to the acquisition point position identifier, while attaching the working condition summary field set to the multi-modal alignment segment and registering the working condition window coverage range and playback index, so that the same time window identifier forms a working condition window entry that can be called in parallel, arrange the working condition window entries in order of time window identifier to generate a multi-modal alignment working condition window sequence.

[0069] It should be noted that the multi-modal alignment working condition window sequence is a sequential sequence formed by fixing the uniform time scale index range for each time window identifier on the uniform time scale sequence, and merging the multi-modal alignment segments covered by the index range into working condition window entries according to the acquisition point position identifier. The role is to provide parallel callable playback and alignment caliber for multi-modal alignment segments within the same time window, and serve as a time organization carrier for subsequent same window deduction input packages and event segment positioning lists.

[0070] Based on the statistical summary of the difference sequence, a robust linear fit is performed on the difference sequence according to the uniform time scale index to obtain the linear offset slope and linear offset intercept and convert them into drift back amount; quantile limit screening is performed on the residual sequence of the difference sequence and the linear fit result and consecutive out-of-bounds residual segments are registered as abnormal jump segments (e.g., three consecutive scales exceeding the 95% quantile limit of the residual).

[0071] The formula for calculating the difference between adjacent time stamps is:

[0072] ;

[0073] in, Indicates the first Article and No. Difference between adjacent time markers between bars Represents the first in the time-stamped sequence Each time stamp value can be obtained. Represents the first in the time-stamped sequence Each time stamp value can be selected. Indicates and The corresponding frame sequence number field value is used to represent the frame interval spanned between two adjacent acquisition entries. Indicates and The corresponding frame sequence number field value is obtained. This represents the frame interval normalization factor. When there are duplicate frame numbers or the frame number has not advanced, it is treated as 1 to avoid division by zero and to keep the difference calculable.

[0074] It should be noted that the operating condition window length (example range: 1s-20s) is directly used as the length of the uniform time scale index range covered by each time window identifier when dividing the uniform time scale sequence according to the time window identifier.

[0075] Sliding step size (example range: 0.1s-5s): When generating a uniform time scale sequence by sliding according to the time window identifier, the index increment of the starting point of the adjacent time window identifier on the uniform time scale sequence is directly used as the sliding step size.

[0076] S3: Input the multimodal aligned working condition window sequence into the digital twin mechanism link, perform the ideal response deduction of the same window, output the ideal multimodal response, construct the twin dynamic residual, and at the same time segment the residual structure event to generate the residual event token set;

[0077] S3.1: Extract the time window identifier and unified time scale index range from the multimodal aligned working condition window sequence, and combine it with the mechanical topology observation mapping table to map the acquisition point identifiers and generate the same window inference input package;

[0078] Further, the multi-modal alignment working condition window sequence is traversed in time window identifier sequence, and a time window identifier and a unified time scale index range are extracted. The multi-modal alignment segment corresponding to the unified time scale index range is merged according to a collection point position identifier and is registered with a segment playback index. According to the mechanical topology observation mapping table, mapping retrieval is performed on the collection point position identifier, and a topology node identifier, a topology edge identifier, an edge weight identifier value and a transmission direction description associated with the collection point position identifier are obtained. The topology node identifier, the topology edge identifier, the edge weight identifier value and the transmission direction description are merged with the time window identifier and the unified time scale index range to form a deduction positioning item. The deduction positioning item is sorted according to the topology node identifier and the topology edge identifier, and a chain item retrieval key is generated according to the transmission direction description, so as to form a topology position list required for chain item scheduling. At the same time, the working condition abstract field set is arranged into a driving field set, and the driving field set is hung and packaged with the topology position list to generate a same window deduction input package.

[0079] S3.2: According to the same window deduction input package, the digital twin mechanism link is driven to perform chain item scheduling deduction, and an ideal multi-modal response is generated.

[0080] Further, the chain items of the digital twin mechanism link are retrieved according to the topology position list in the same window deduction input package, and the chain item scheduling order is determined according to the transmission direction description. The driving field set is matched and arranged according to the input field required by the chain item, and the chain item driving load is generated. The chain items are triggered in turn according to the chain item scheduling order, the intermediate response output by the upstream chain item is backfilled into the input of the downstream chain item according to the topology node identifier and the topology edge identifier, and the time window identifier and the unified time scale index range are kept consistent, until the chain item sequence is completed. The ideal response output by each chain item is mapped into an ideal current response, an ideal vibration response and an ideal acoustic emission response according to the collection point position identifier, and is cut to the unified time scale index range. At the same time, a chain item gap prompt item is registered for a missing link segment. The ideal response segments corresponding to each collection point position identifier are gathered and archived according to the time window identifier, and an ideal multi-modal response is generated.

[0081] It should be noted that the ideal multi-modal response refers to a set of ideal current response, ideal vibration response and ideal acoustic emission response segments output by the digital twin mechanism link in the same window deduction input package along the chain item scheduling order and cut to the unified time scale index range. The role is to use the ideal side reference in the same window ideal response as a contrast set to construct a twin dynamic residual by comparing the measured segments in the multi-modal time alignment segment set, and trigger a residual structure event segmentation.

[0082] The chain entry of the digital twin mechanism link performs linear transmission processing or look-up table interpolation processing on the chain entry type marked transmission intensity description constraint. The linear transmission processing performs gain mapping and frequency band filtering on the chain entry input field set to obtain the chain entry output field set. The look-up table interpolation processing maps the transmission intensity description index table and interpolates the chain entry input field set to obtain the chain entry output field set. The chain entry output field set is mapped to the ideal current response, the ideal vibration response and the ideal acoustic emission response according to the acquisition point position identifier to form the ideal multi-modal response.

[0083] S3.3: Correspondence between the ideal multi-modal response and the multi-modal time alignment segment set is established, and windowed cutting alignment is performed to generate a windowed ideal response comparison set;

[0084] Further, the ideal multi-modal response is traversed according to the time window identifier, and the acquisition point position identifier, the time window identifier and the uniform time scale index range are extracted. The corresponding measured segment is retrieved according to the acquisition point position identifier, and the uniform time scale index range consistency is checked. The ideal response segment and the measured segment with the same acquisition point position identifier and index range matching are established in one-to-one correspondence, and a comparison hanging entry is generated. The comparison hanging entry with index range boundary offset is executed for windowed cutting alignment, and the ideal response segment and the measured segment are cut according to the intersection of the uniform time scale index range and registered cutting offset summary. The comparison hanging entry with a gap is registered for a gap prompt entry and the alignable segment section is retained. The ideal response segment after cutting alignment and the measured segment after cutting alignment are merged and packaged with the acquisition point position identifier, the time window identifier and the topology position field set as a comparison entry and sorted and gathered according to the time window identifier to generate a windowed ideal response comparison set.

[0085] S3.4: Based on the windowed ideal response comparison set, residual multi-modal paragraph extraction is performed, twin dynamic residual is constructed, and residual structure events are divided along the uniform time scale index range to generate a residual event entry set;

[0086] Further, the same window ideal response set is traversed according to the time window identifier, and the collection point identifier, the unified time scale index range, the ideal response segment and the measured segment are extracted, the ideal response segment and the measured segment are aligned in the unified time scale index range, and the difference sequence is generated. At the same time, the difference sequence is bound and registered as a twin dynamic residual with the collection point identifier, the topological position field set and the time window identifier; the residual multi-form paragraph extraction is performed on the twin dynamic residual along the unified time scale index range, and the paragraph boundary is positioned and the event start time scale and the event end time scale are registered according to the burst jump segment, the continuous sideband enhancement segment, the envelope pulse intensive segment and the acoustic emission cluster segment, and the residual intensity abstract is extracted for each paragraph; the event segment obtained by cutting the twin dynamic residual according to the paragraph boundary is combined and encapsulated as a residual event item with the event type marker, the event boundary, the collection point identifier, the topological position field set and the time window identifier, and the residual event item set is generated by sorting and converging according to the time window identifier and the topological node identifier.

[0087] It should be pointed out that the twin dynamic residual is defined as the difference sequence after the ideal response segment and the measured segment are aligned in the unified time scale index range, and the trigger condition of the residual structure event is limited to the residual energy abstract or the residual change rate abstract of the difference sequence continuously exceeding the quantile limit section based on the residual statistical abstract in the same time window identifier, that is, the event paragraph boundary is registered.

[0088] S3.5: The residual event item set is encapsulated as a token load field and a topological prior field according to the time window identifier and the topological node identifier, and written into an event sequence index to generate a residual event token set;

[0089] Further, the residual event item set is grouped and sorted according to the time window identifier and the topological node identifier, and the event type marker, the event boundary and the event segment of each residual event item are extracted, the token load compression is performed on the event segment, and the time structure and the amplitude structure abstract are reserved and converged as a token load field; the topological node identifier, the topological edge identifier, the edge weight identifier value and the transmission direction description of the same group of residual event items are extracted and arranged as a topological prior field, and the token load field and the topological prior field are combined and encapsulated as a residual event token item; the residual event token item is sorted according to the event boundary start time scale in the time window identifier, and an event sequence index is generated. The event sequence index is backfilled to the residual event token item to maintain the sequence position consistency across the collection point identifier, and the residual event token item with the event sequence index is output by converging according to the time window identifier to generate a residual event token set.

[0090] It should be pointed out that, Figure 2The time sequence evolution process of the twin dynamic residual energy and the triggering position of the residual event token under the continuous working condition window sequence number are shown. The horizontal axis is the working condition window sequence number, and the vertical axis is the twin dynamic residual energy, which is used to depict the dynamic deviation degree between the actual multi-modal response and the digital twin mechanism deduction result. It can be seen from the figure that when the equipment is in a stable running stage, the twin dynamic residual energy is maintained in a low and fluctuation controlled interval; when the running state gradually deviates from the mechanism deduction trend, the residual energy appears continuous lifting and forms a peak section. By marking the residual event token trigger point on the residual energy curve, the continuous residual change can be structured into a locatable abnormal event starting point, so that the abnormality is converted from "continuous numerical deviation" to "discrete event token". Compared with the method of determining only according to the residual amplitude change, this mechanism can complete the event starting point marking in advance in the residual lifting stage, providing a more sufficient time response window for subsequent state prediction and disposal.

[0091] S4: Positioning multi-modal data segments based on the residual event token set, performing cross-modal attention alignment, and combining the mechanical topology observation mapping table to perform topology propagation aggregation and generate a topology constraint fusion representation set;

[0092] S4.1: Extracting a segment playback range according to the residual event token set, mapping a collection point location identifier, and cutting multi-modal alignment segments in a multi-modal alignment working condition window sequence to generate an event segment positioning list;

[0093] Further, the time window identifier, collection point location identifier, event boundary and event sequence index are extracted according to the residual event token set, the event boundary is converted into a unified time scale index range and registered as the segment playback range; the collection point location identifier is mapped to the topology node identifier, topology edge identifier and channel positioning information according to the mechanical topology observation mapping table, and the matching working condition window entry is searched in the multi-modal alignment working condition window sequence according to the time window identifier; the current alignment segment, vibration alignment segment, acoustic emission alignment segment and acoustic emission alignment segment are indexed and cut in the matching working condition window entry according to the segment playback range, the cut segment is bound and packaged with the collection point location identifier, topology node identifier, topology edge identifier, event sequence index and event boundary, and written into the playback range consistency check record to generate the event segment positioning list.

[0094] S4.2: Extracting an event anchor point trajectory from the multi-modal alignment segment of the event segment positioning list, and performing cross-modal attention alignment to generate a cross-modal alignment event representation set;

[0095] Further, the event segment positioning list is used to locate the current alignment segment, vibration alignment segment, acoustic emission alignment segment, and acoustic emission alignment segment one by one, and the segment playback range and event boundary are extracted; the event anchor point track is extracted around the event boundary in the segment playback range, the event anchor point track includes the amplitude mutation track of the current alignment segment, the energy rise track of the vibration alignment segment, the pulse cluster track of the acoustic emission alignment segment, and the impact envelope track of the acoustic emission alignment segment, and the event anchor point track is aligned and arranged into an event anchor point track group according to a unified time scale index range; the time domain statistical features and the frequency domain statistical features of the current alignment segment, the vibration alignment segment, and the acoustic emission alignment segment are extracted and aggregated into an event segment representation vector, the cross-modal alignment weight is obtained by normalizing and mapping according to the similarity score of the event segment representation vector, and the edge weight label value and the transmission direction description in the topological prior field set are superimposed into a weight bias constraint; the cross-modal alignment weight summary and the event segment representation are packaged into an entry set according to the collection point label, the topological node label, the topological edge label, the event sequence index, and the event type mark, and a cross-modal alignment event representation set is generated.

[0096] It should be noted that the event anchor point track refers to a boundary response track set extracted from the current alignment segment, the vibration alignment segment, and the acoustic emission alignment segment around the event boundary in the segment playback range and aligned and arranged according to a unified time scale index range, which is used to provide alignment guidance and anchor point reference for cross-modal attention alignment.

[0097] S4.3: The cross-modal alignment event representation set is hung to the mechanical topological observation mapping table according to the topological node label and the topological edge label, and a propagation message backfill update is performed to generate a topological propagation aggregated representation set;

[0098] Further, the cross-modal alignment event representation set is extracted to obtain the topological node label, the topological edge label, the cross-modal alignment weight summary, and the event segment representation one by one, and the topological node label and the topological edge label are hung to establish a hanging index; according to the hanging index, the event segment representation is merged and hung to the corresponding topological node label entry and topological edge label entry of the mechanical topological observation mapping table, and the edge weight label value and the transmission direction description are synchronously hung to form a topological position event representation entry; according to the transmission direction description, the topological edge label entries of the mechanical topological observation mapping table are traversed, the adjacent topological position event representations are aggregated based on the topological node labels at both ends of the topological edge label, and the propagation message is generated by weighted aggregation according to the edge weight label value, while the cross-modal alignment weight summary is used for gating and screening to eliminate the cross-modal inconsistent propagation contribution; the propagation message aggregated by the topological edge label is written back to the node representation corresponding to the terminal topological node label and the node representation update summary is registered, and the edge representation summary corresponding to the adjacent topological edge label is synchronously updated according to the node representation update summary; the multi-round propagation message backfill update is repeatedly performed according to the time window label, and the update records of each round are summarized to generate a topological propagation aggregated representation set.

[0099] It should be noted that the topological propagation aggregation representation set refers to the aggregation of the node representation entries and edge representation entries obtained by propagating and converging the adjacent topological positions according to the transmission direction description and edge weight label value after the cross-modal alignment event representation set is hung to the mechanical topological observation mapping table according to the topological node identifier and the topological edge identifier, and the updated node representation entries and edge representation entries are written back, which is used to expand the local event fragment representation under the topological constraint to a global consistent representation input for subsequent topological constraint fusion representation set arrangement and graph neural network inference.

[0100] S4.4: Arrange the node representation entries and edge representation entries of the topological propagation aggregation representation set, and add time window identifier and event sequence index to generate a topological constraint fusion representation set;

[0101] Further, the topological propagation aggregation representation set is traversed according to the time window identifier, and the updated topological position event representation corresponding to the topological node identifier and the propagation constraint information corresponding to the topological edge identifier are extracted, the updated topological position event representation is arranged into a node representation entry according to the topological node identifier and the topological node identifier is registered, the propagation constraint information is arranged into an edge representation entry according to the topological edge identifier, and the topological edge identifier, edge weight label value and transmission direction description are registered; the node representation entries and edge representation entries are additionally added with the time window identifier, and the index alignment relationship is established according to the event sequence index of the residual event token set, and the event sequence index is written into the node representation entries and edge representation entries to keep the cross-step retrieval caliber consistent; the node representation entries and edge representation entries are subjected to entry-level integrity verification, and abnormal entries lacking time window identifier or lacking topological identifier are removed, the remaining entries are sorted according to the time window identifier and de-duplicated and merged according to the topological node identifier and the topological edge identifier to generate a topological constraint fusion representation set.

[0102] It should be noted that the propagation constraint information is a summary of the edge weight label value and the transmission direction description of the topological edge identifier in the propagation message aggregation process, which is used to limit the weighted convergence strength and propagation direction caliber of the node representation entries between adjacent topological positions.

[0103] Figure 3The change of cross-modal event anchor consistency deviation under different clock drift amplitude conditions is given, and the key difference interval is highlighted through local magnification. The horizontal axis is the clock drift amplitude, and the vertical axis is the cross-modal event anchor consistency deviation, which reflects the alignment degree between the multi-modal collected data and the mechanism event anchor in the time dimension. The overall trend shows that as the clock drift amplitude increases, the consistency deviation of each scheme increases; among them, the scheme combined with the mechanical topology observation mapping table and the introduction of residual event tokens has a lower growth rate, and can still maintain a relatively stable alignment level in the medium drift interval. The local magnified area further shows the separation degree of different schemes in this interval, and the difference value is marked at the largest drift amplitude to intuitively explain the supporting role of residual event tokens for cross-modal anchor alignment. It reflects that residual event tokens provide stable time anchor basis for cross-modal alignment.

[0104] Figure 4 The positioning results of twin dynamic residual event boundaries under different schemes are compared and displayed, which are used to reflect the role of residual event tokens in abnormal starting time judgment. The horizontal axis is the working condition window number, and the vertical axis is the twin dynamic residual event score. The rising section of the event score is used to depict the boundary starting point of the abnormal event. The boundary starting point positions under the baseline scheme, the mechanical topology observation mapping table scheme, the residual event token generation scheme and the mapping table and token superposition scheme are marked in the figure. Comparison shows that when residual event tokens are not introduced, the boundary starting point is later, usually falling after the residual energy peak section; after generating residual event tokens, the boundary starting point moves forward, and after superimposing the topology constraint, a more consistent positioning result is maintained. By comparing the boundary starting point advance amount of different schemes, the effect of the present application on early positioning of abnormal events can be intuitively presented, thereby providing more sufficient time margin for state prediction.

[0105] S5: input the topology constraint fusion feature set into the graph neural network, combine the mechanical topology graph for message passing, output the component state and the whole machine state, and package as a mining equipment state prediction set;

[0106] S5.1: arrange the topology constraint fusion feature set in batches according to the graph structure, organize into node input sequence and edge input sequence, establish a topology index table, combine the cross-modal alignment weight summary, and generate a topology input arrangement package;

[0107] Further, traverse the topology constraint fusion representation set by time window identifier, extract node representation entries and edge representation entries, de-duplicate and merge node representation entries according to topology node identifier, and sort by time window identifier and event sequence index to form node input sequence, de-duplicate and merge edge representation entries according to topology edge identifier, and sort by time window identifier and event sequence index to form edge input sequence; locate corresponding sequence positions in the node input sequence according to the start topology node identifier and the end topology node identifier in the edge representation entries, register the association pairs of the edge input sequence position and the node input sequence position, and form a topology index table; align the index caliber of the cross-modal alignment weight summary according to the event sequence index, merge the cross-modal alignment weight summary into the corresponding entries of the node input sequence and the edge input sequence, and perform missing item filling and conflict item elimination to generate a topology input arrangement package.

[0108] S5.2: input the topology input arrangement package into the graph neural network, align with the connection relationship of the mechanical topology graph, and initiate directional message passing, while performing intensity modulation and type gating, output the topology reasoning update representation set;

[0109] Further, input the topology input arrangement package into the graph neural network, perform consistency alignment on the topology index table according to the connection relationship of the mechanical topology graph and form a rejection list, initiate directional message passing from the start topology node identifier to the end topology node identifier for each topology edge identifier according to the transmission direction description, combine the node input sequence representation and the edge input sequence representation into propagation load and perform intensity modulation according to the edge weight label value, while performing type gating and gating screening according to the topology edge type entry and the cross-modal alignment weight summary, update the node representation according to the end topology node identifier and return the update summary to update the edge representation, repeat the directional message passing according to the predetermined propagation round, and the propagation round is selected as the minimum coverage round according to the topology diameter hop number of the mechanical topology graph and is set as an upper limit (for example, 2 rounds to 5 rounds) to control the reasoning delay, output the topology reasoning update representation set.

[0110] It should be noted that the topology reasoning update representation set refers to the set of updated node representations and updated edge representations obtained by the graph neural network after performing directional message passing, intensity modulation and type gating on the topology input arrangement package under the constraint of the mechanical topology graph, which is used as a direct input for aggregating and outputting component states and further aggregating and outputting whole machine states according to the mapping relationship between component identifiers and topology node identifiers.

[0111] S5.3: based on the topology reasoning update representation set, aggregate and output component states according to the mapping relationship between component identifiers and topology node identifiers, and aggregate and output whole machine states according to the topology level relationship of the mechanical topology graph, generate a mining equipment state prediction set;

[0112] Further, the topology reasoning update set is traversed according to the time window identifier, the updated node representation corresponding to the topology node identifier is extracted, the updated node representation is merged into a component-level representation entry according to the mapping relationship between the component identifier and the topology node identifier, and state decoding is performed on the component-level representation entry to output a component state. The component state is output in the form of a health level set (for example, normal, early warning, and failure) and can be output in parallel in the form of a health score (for example, example 0 to example 1). At the same time, a confidence label consistent with the time window identifier is added to the component state. The confidence label adopts a normalized confidence distribution of the health level set or an uncertainty interval of the health score and records a confidence threshold label. All component-level representation entries are hierarchically aggregated according to the topology level relationship of the mechanical topology graph to form a whole-machine-level representation entry, and state decoding is performed on the whole-machine-level representation entry to output a whole-machine state. The whole-machine state is output in the form of a whole-machine health level set and can be output in parallel in the form of a whole-machine health score. The whole-machine health level is obtained by performing consistency aggregation on the component state according to the topology level relationship of the mechanical topology graph. The component state and the whole-machine state are aligned according to the time window identifier, and a traceable index field set of the component identifier, the time window identifier, and the topology node identifier is packaged to generate a mining equipment state prediction set.

[0113] It should be noted that the mining equipment state prediction set refers to a result set obtained by uniformly packaging the component state and the whole-machine state, the corresponding confidence label, and the traceable index field set according to the time window identifier. The result set provides a positionable and traceable state output carrier for subsequent operation and maintenance, alarm linkage, and state playback.

[0114] In summary, the present application achieves the structured binding of multi-modal data and equipment mechanism by constructing a mechanical topology observation mapping table with a weight label, provides topology constraints for fusion, and improves the physical consistency of cross-modal alignment. The structured analysis of twin dynamic residuals is achieved by generating a residual event token set, which accurately locates abnormal events and improves the sensitivity and timeliness of state prediction.

[0115] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and all should be covered in the scope of the claims of the present application.

Claims

1. A mining equipment state prediction method based on digital twinning and multi-modal fusion, characterized by: The application relates to a method for predicting the state of a mining equipment, and belongs to the technical field of mining equipment state prediction. According to the mechanical topological observation mapping table, multi-modal data is collected, and a sampling rate is aligned and clock drift is corrected according to a unified time caliber, so that a multi-modal alignment working condition window sequence is generated. According to the mechanical topological observation mapping table, multi-modal data is collected, and a sampling rate is aligned and clock drift is corrected according to a unified time caliber, so that a multi-modal alignment working condition window sequence is generated. The multi-modal alignment working condition window sequence is input into the digital twin mechanism link, ideal multi-modal response is output through window ideal response deduction, and a twin dynamic residual error is constructed, residual error token sets are generated by cutting residual error structure events, and the residual error token sets are generated. The multi-modal data segment is positioned based on the residual error event token set, cross-modal attention alignment is performed, and topological propagation aggregation is performed by combining the mechanical topological observation mapping table, so that a topological constraint fusion representation set is generated. The topological constraint fusion representation set is input into a graph neural network, message transmission is performed by combining the mechanical topological graph, and component state and whole machine state are output, and the component state and whole machine state are packaged into a mining equipment state prediction set.

2. The method of claim 1, wherein the method is based on a digital twin and multi-modal fusion of a mining equipment state. The mechanical topological graph is constructed in the following steps. A mining equipment component list and a multi-modal acquisition point list are read, and are arranged into a bindable item set, and a binding field is aligned and an installation pose is analyzed, so that a point binding preparation list is generated. Based on the point binding preparation list, a port is aligned into an edge, a topological node and a topological edge are established, and an acquisition point identifier is bound, so that the mechanical topological graph is generated.

3. The method of claim 2, wherein the method is based on a digital twin and multi-modal fusion of a mining equipment state. The mechanical topological observation mapping table is generated in the following steps. The mechanical topological graph is subjected to edge type sorting and transfer direction solidification arrangement, the component coupling transfer relationship is obtained, and the component coupling transfer relationship is connected into a chainable item, so that the digital twin mechanism link is generated. The digital twin mechanism link is converted into an edge weight scale value, the edge weight scale value is subjected to edge weight scale normalization, and the edge weight scale normalization is unified, and the acquisition point identifier is merged and hung, so that the mechanical topological observation mapping table is generated.

4. The method of claim 3, wherein the method is based on a digital twin and multi-modal fusion of a mining equipment state. According to the mechanical topological observation mapping table, multi-modal data is collected, and a sampling rate is aligned and clock drift is corrected according to a unified time caliber, so that a multi-modal alignment working condition window sequence is generated. Based on the mechanical topological observation mapping table, acquisition point identifier grouping is performed, and a multi-modal acquisition arrangement list is obtained through arrangement of a sampling rate configuration and a channel clock source description. According to the multi-modal acquisition arrangement list, multi-modal data is collected, and a time mark and a frame serial number field are packaged, and abnormal frames are removed, so that a multi-modal original acquisition stream set is generated.

5. The method of claim 4, wherein the method is based on a digital twin and multi-modal fusion of a mining equipment state prediction. The multi-modal alignment working condition window sequence is generated in the following steps. Based on the multi-modal original acquisition stream set, a unified time scale sequence is established, and time scale aggregation slicing and time scale point interpolation are performed, so that a multi-modal time alignment segment set is generated. A difference sequence of adjacent time marks is extracted from the multi-modal time alignment segment set, clock drift is corrected, and multi-modal alignment segments are packaged, so that the multi-modal alignment working condition window sequence is generated.

6. The method of claim 5, wherein the method is based on a digital twin and multi-modal fusion of a mining equipment state prediction. The ideal multi-modal response is generated in the following steps. A time window identifier and a unified time scale index range are extracted from the multi-modal alignment working condition window sequence, and acquisition point identifier mapping is performed by combining the mechanical topological observation mapping table, so that a window deduction input package is generated. According to the window deduction input package, the digital twin mechanism link is driven to perform chain item scheduling deduction, and the ideal multi-modal response is generated. Corresponding relationship is established between the ideal multi-modal response and the multi-modal time alignment segment set, and the same window cutting alignment is performed to generate a same window ideal response comparison set.

7. The method of claim 6, wherein the method is based on a digital twin and multi-modal fusion of a mining equipment state prediction. The residual event token set is generated by the following steps, Based on the same window ideal response comparison set, residual multi-modal paragraph extraction is performed, twin dynamic residual is constructed, residual structure event is cut along the unified time scale index range, and a residual event item set is generated; The residual event item set is encapsulated into token load field and topology prior field according to time window identifier and topology node identifier, and written into event sequence index to generate residual event token set.

8. The method of claim 7, wherein the method is based on a digital twin and multi-modal fusion of a mining equipment state prediction. The cross-modal attention alignment is performed by the following steps, According to the residual event token set, the segment playback range is extracted, and the collection point location identifier is mapped, and the multi-modal alignment segment is cut in the multi-modal alignment working condition window sequence to generate an event segment positioning list; The event anchor point trajectory is extracted from the multi-modal alignment segment of the event segment positioning list, and the cross-modal attention alignment is performed to generate a cross-modal alignment event representation set.

9. The method of claim 8, wherein the method is based on a digital twin and multi-modal fusion of a mining equipment state prediction. The topology constraint fusion representation set is generated by the following steps, The cross-modal alignment event representation set is hung to the mechanical topology observation mapping table according to the topology node identifier and the topology edge identifier, and the propagation message backfilling update is performed to generate a topology propagation aggregation representation set; The node representation item and the edge representation item of the topology propagation aggregation representation set are arranged, and the time window identifier and the event sequence index are attached to generate the topology constraint fusion representation set.

10. The method of claim 9, wherein the method is based on a digital twin and multi-modal fusion of a mining equipment state prediction. The mining equipment state prediction set is packaged by the following steps, The topology constraint fusion representation set is arranged in batches according to the graph structure, and is arranged into node input sequence and edge input sequence, and a topology index table is established, and a cross-modal alignment weight summary is combined to generate a topology input arrangement package; The topology input arrangement package is input into the graph neural network, and the connection relationship of the mechanical topology graph is aligned, and the directional message passing is initiated, and the strength modulation and the type gating are performed to output the topology reasoning update representation set; Based on the topology reasoning update representation set, the component state is output according to the mapping relationship of the component identifier and the topology node identifier, and the whole machine state is output according to the topology hierarchical relationship of the mechanical topology graph to generate the mining equipment state prediction set.

Images