End-edge-cloud collaborative scheduling method and system based on multi-modal and anti-lock heterogeneous graph

Abstract

The application relates to the technical field of robot cooperative scheduling, and discloses an end-edge-cloud cooperative scheduling method and system based on a multi-modal and anti-lock heterogeneous graph, which comprises the following steps: using orthogonal projection and a second-order distance calculation technology of transmission cost to calculate an environment context representation vector corresponding to a feature tensor and an environment adaptive speed regulation coefficient; calculating an expected value of a mobile candidate action based on a heterogeneous scheduling graph, and combining the expected value with the environment context representation vector and the environment adaptive speed regulation coefficient to determine a physical driving speed instruction of an automatic guided vehicle; after the physical driving speed instruction is executed, implementing edge event matrix activation matching processing, intercepting a space-time context slice window to generate a space-time tracing sequence after the edge event matrix activation matching; and when a product quality defect alarm signal is received, calculating a real-time energy scalar of a time stamp corresponding to the space-time tracing sequence, and outputting a product defect tracing result. The application can realize high coverage of workshop monitoring.

Description

Technical Field

[0001] This invention relates to the field of robot collaborative scheduling technology, and more specifically, to an edge-cloud collaborative scheduling method and system based on multimodal and deadlock-preventing heterogeneous graphs. Background Technology

[0002] In the wave of intelligent manufacturing, production workshops are undergoing a profound transformation from automation to flexibility and intelligence. Intelligent robots, as core carriers (such as autonomous mobile robots (AMRs) and collaborative robots), have evolved from simple handlers or operators into intelligent agents that need to interact complexly with the environment, tasks, and other equipment. However, the current mainstream underlying technology system supporting its operation suffers from three major defects: blind spots in perception, slow collaborative response, and a lack of system flexibility. These defects manifest in the following three aspects:

[0003] 1. Fixed sensor layouts cannot achieve dynamic, seamless coverage and accurate defect tracing across the entire workshop: Existing workshop environmental monitoring systems generally use fixed point-based sensors (such as cantilever particle counters, machine-side vibration probes, and wall-mounted temperature and humidity sensors). Their monitoring range is highly limited to the physical location of the equipment, resulting in significant physical monitoring blind spots (such as equipment tops, pipe cavities, narrow corridors, and AGV dynamic transport paths). Simultaneously, traditional low-frequency polling sampling mechanisms are prone to creating temporal monitoring blind spots, failing to effectively capture high-frequency transient disturbances caused by equipment start-ups, shutdowns, or anomalies. When subsequent quality inspections discover product defects, existing systems cannot establish a complete spatiotemporal correlation map, making it impossible to accurately define the specific physical coordinates and time period of the defect's introduction. Root cause analysis of defects heavily relies on manual experience, resulting in low efficiency and a lack of data support. Furthermore, adding new monitoring points requires disruptive workshop modifications (such as drilling, wiring, and shutdown calibration), leading to high hardware deployment and implementation costs. Moreover, rapid reconstruction based on dynamic adjustments to the process flow is difficult, severely restricting the workshop's flexible manufacturing capabilities.

[0004] 2. The centralized scheduling architecture suffers from high response latency, failing to meet the requirements of real-time collaboration and scalable expansion: Existing workshop AGV scheduling generally adopts a center-radial control architecture, where the central server needs to centrally handle the state synchronization and global optimization calculations of all AGVs. Under this architecture, the combined overhead of communication networks and centralized computing pressure results in a significant latency bottleneck in end-to-end response. At normal AGV operating speeds, this level of control latency implies significant braking blind spots, making it difficult to cope with sudden obstacles and failing to meet the stringent functional safety requirements for real-time collision avoidance in precision manufacturing scenarios. Furthermore, the complexity of traditional centralized global planning algorithms typically increases exponentially or by O(n^2) with the number of AGVs, causing the system's computing power to easily reach its limit, making smooth expansion to large-scale clusters difficult. More critically, this architecture suffers from a severe single point of failure risk: once the central server crashes or the workshop's local network is interrupted, all AGVs will instantly lose scheduling instructions. Due to the lack of autonomous decision-making and degradation capabilities at the edge, this can easily lead to large-scale shutdowns or even physical collisions.

[0005] 3. Heavy reliance on manual maintenance and lack of adaptive evolution capabilities and optimization experience sharing mechanisms: Existing technologies have significant economic and efficiency deficiencies throughout their entire lifecycle. On the one hand, the business logic of centralized scheduling heavily relies on massive rule bases that are manually written and maintained. As process complexity increases, the rule base becomes increasingly complex and prone to logical conflicts, with daily rule updates and handling of abnormal deadlocks consuming a large amount of professional maintenance manpower. On the other hand, the system model is fixed, making it difficult to quickly adapt and evolve to new defect patterns or process changes once deployed; if optimization is needed, it usually involves a long cycle of re-collecting data, retraining the model, and re-deploying after downtime, often missing the best process optimization window. In addition, the systems in each workshop and factory generally operate in isolation, forming data silos. Excellent scheduling strategies and abnormal handling experience that work locally cannot be shared globally, resulting in huge redundant investments and low efficiency in overall intelligent upgrades.

[0006] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0007] To address the problems in related technologies, this invention proposes an edge-cloud collaborative scheduling method and system based on multimodal and deadlock-preventing heterogeneous graphs, in order to overcome the aforementioned technical problems existing in the existing related technologies.

[0008] Therefore, the specific technical solution adopted by the present invention is as follows:

[0009] In a first aspect, the present invention provides an edge-cloud collaborative scheduling method based on multimodal and deadlock-preventing heterogeneous graphs, the method comprising:

[0010] Based on the multimodal raw data processing results, a feature tensor is generated, and the environmental context representation vector and environmental adaptive speed adjustment coefficient corresponding to the feature tensor are calculated using orthogonal projection and second-order distance calculation techniques for transmission cost.

[0011] Construct a heterogeneous scheduling graph, calculate the expected value of candidate movement actions based on the heterogeneous scheduling graph, and combine it with the environmental context representation vector and the environmental adaptive speed adjustment coefficient to determine the physical drive speed command of the automated guided vehicle.

[0012] After the physical drive speed command is issued and executed, edge event matrix activation matching is performed, and after edge event matrix activation matching, spatiotemporal context slice window is captured to generate spatiotemporal tracing sequence;

[0013] An energy manifold model is constructed to define the total scalar energy. When a product quality defect alarm signal is received, the real-time energy scalar corresponding to the timestamp of the spatiotemporal tracing sequence is calculated, compared with the total scalar energy, and the product defect tracing result is output.

[0014] Furthermore, feature tensors are generated based on the multimodal raw data processing results, and the environmental context representation vector and environmental adaptive speed adjustment coefficients corresponding to the feature tensors are calculated using orthogonal projection and second-order distance calculation techniques based on transmission cost.

[0015] Vibration, particulate matter, and visual image data collected by the automated guided vehicle and fixed sensing devices are acquired to obtain multimodal raw data. The multimodal raw data is then time-stamp aligned, and multimodal data is obtained based on the processing results.

[0016] After orthogonally projecting the multimodal data, a feature tensor is generated. The feature tensor is then differentiated and combined with the channel dimension to construct a structural response matrix to determine the environmental context representation vector.

[0017] Based on the multivariate Gaussian distribution parameters of the feature tensor output by the pre-encoding layer in the latent space, the mean vector and diagonal covariance matrix are determined, and the feature distribution of the feature tensor is output according to the mean vector and diagonal covariance matrix.

[0018] The optimal transmission theory is used to calculate the transmission cost required to transform the feature distribution of any feature tensor into another distribution form, the second-order distance is determined, and the feature tensor with the largest second-order distance is selected as the global modal conflict metric factor. The global modal conflict metric factor is introduced into the servo control logic to calculate the environmental adaptive speed adjustment coefficient.

[0019] Furthermore, after orthogonally projecting the multimodal data, a feature tensor is generated. This feature tensor is then differentiated and combined with the channel dimension to construct a structural response matrix, which determines the environmental context representation vector, including:

[0020] The high-dimensional feature vectors contained in the multimodal data are extracted based on the pre-coding layer, and the high-dimensional feature vectors are input into the orthogonal projection layer. After stripping the redundant shared information between the modalities, they are mapped to a shared latent space with a fixed dimension to generate a feature tensor of a unified dimension. The feature tensor includes visual features, physical perception features and anomaly type encoding.

[0021] The second-order differential operation of the feature tensor is performed using the Laplacian operator to obtain the gradient feature map that characterizes the degree of local high-frequency abrupt changes, and the structural response matrix representing spatial topological changes is generated by combining the channel dimension.

[0022] Based on the normalization of the structural response matrix into a structural pseudo-probability matrix that satisfies the probability distribution characteristics, the relative information weights contained at spatial coordinates are obtained, and the spatial local structural entropy of the feature tensor is calculated through the structural pseudo-probability matrix.

[0023] The spatial local structural entropy is used as an exponential decay term to generate dynamic confidence gating coefficients. The environmental context representation vector is then determined by weighted fusion of the feature tensor and the dynamic confidence gating coefficients.

[0024] Furthermore, a heterogeneous scheduling graph is constructed, and the expected value of candidate movement actions is calculated based on the heterogeneous scheduling graph. This value is then combined with the environmental context representation vector and the environmental adaptive speed adjustment coefficient to determine the physical drive speed command of the automated guided vehicle, including:

[0025] Based on the workshop status and the coordinates of the automated guided vehicles (AGVs), a heterogeneous scheduling diagram covering AGVs, process equipment, and task nodes is constructed using physical fingerprint anchoring and forward projection technology.

[0026] The dynamic gravity coefficient and exponential repulsion force are determined based on the autonomous vehicle node and the task node, and the deadlock penalty factor is generated by constructing a label multiset through the joint feature sequence of the local topology in the local heterogeneous scheduling graph.

[0027] The dynamic gravity coefficient, exponential repulsion force, and deadlock penalty factor are introduced into the attention aggregation mechanism to output the final representation of the spatial structure after deep aggregation, and determine the spatial topological constraint results to eliminate the deadlock situation of the automated guided vehicle cluster.

[0028] The evolutionary features of the final representation of the spatial structure are extracted by using the reset gate, and the spatiotemporal hidden state representation is generated. The spatiotemporal hidden state representation, environmental context representation vector and action embedding vector are concatenated and input into the value network to decode and calculate the expected value of each movement candidate action of the automated guided vehicle.

[0029] The optimal action is selected from among all candidate actions, the one whose expected value is greater than the expected value of the current action of the automated guided vehicle. The optimal action is then generated by combining the environmental adaptive speed adjustment coefficient and the physical drive speed command is sent to the automated guided vehicle master controller.

[0030] Furthermore, based on the workshop status and the coordinates of the automated guided vehicles (AGVs), a heterogeneous scheduling diagram covering AGVs, process equipment, and task nodes is constructed using physical fingerprint anchoring and forward projection technology, including:

[0031] The automated guided vehicle nodes are determined based on global coordinates and load rate, the process equipment nodes are determined based on equipment coordinates and working status, and the task nodes are determined based on task priority and time tolerance.

[0032] The heterogeneous nodes are obtained by combining the automated guided vehicle nodes, process equipment nodes and task nodes. Then, a nonlinear hash function is used to concatenate the heterogeneous nodes with timestamps and node physical medium access control addresses at the feature level to generate tamper-resistant spatiotemporal physical fingerprints.

[0033] Heterogeneous nodes are projected onto a feature space of the same dimension using a linear transformation matrix to obtain an isomorphic feature tensor. Based on the isomorphic feature tensor, a forward transformation matrix is ​​used to generate a reconstructed original feature set. The L2-norm projection reconstruction residual between the heterogeneous nodes and the reconstructed original feature set is calculated to verify the node retention during the mapping process.

[0034] A dynamic projection fault tolerance benchmark threshold is constructed based on the condition number of the forward transformation matrix and the electromagnetic environment noise intensity, and the logic judgment is performed based on the dynamic projection fault tolerance benchmark threshold, node retention degree and spatiotemporal physical fingerprint.

[0035] Based on the logical judgment results, polluted nodes in the heterogeneous nodes are removed, and the Automated Guided Vehicle (AGV) is used as the center to perform topology connection processing based on the remaining heterogeneous nodes. A local heterogeneous scheduling graph is generated based on the topology connection results.

[0036] Furthermore, based on the dynamic gravity coefficient and exponential repulsion force determined by the automated guided vehicle nodes and task nodes, and by constructing a label multiset through the joint feature sequence of the local topology in the local heterogeneous scheduling graph to generate a deadlock penalty factor, including:

[0037] Extract the semantic attributes of all heterogeneous nodes in the local heterogeneous scheduling graph, assign initial labels to each heterogeneous node, and perform aggregation iteration processing based on the initial labels to obtain the first-order neighbor node set of each heterogeneous node;

[0038] Extract the label set of the first-order neighbor node set from the previous aggregation iteration, sort the label set in lexicographical order to obtain the neighbor label set, and concatenate the current label of each heterogeneous node with the neighbor label set to generate a joint feature sequence representing the local topology of each heterogeneous node.

[0039] By using a one-way hash function to map the joint feature sequence to fixed-length discrete labels, the fixed-length discrete labels of all heterogeneous nodes in the local heterogeneous scheduling graph are collected to obtain a label multiset;

[0040] The frequency of occurrence of various labels in the label multiset is counted, and the occurrence frequency is mapped to a fixed-length global topological feature vector as the topological fingerprint of the local heterogeneous scheduling graph. The cosine similarity between the topological fingerprint and the pre-set high-risk motif fingerprint is calculated to generate a deadlock penalty factor.

[0041] Furthermore, after the physical drive speed command is issued and executed, edge event matrix activation matching processing is implemented, and after edge event matrix activation matching, a spatiotemporal context slice window is captured to generate a spatiotemporal tracing sequence, including:

[0042] After the physical drive speed command is issued and executed, the global modal conflict measurement factor is extracted. When the global modal conflict measurement factor exceeds the safety tolerance threshold, or the dynamic confidence gating coefficient of any feature tensor decays, a perception confidence change event is triggered, indicating that the physical state of the environment has changed abruptly.

[0043] The cosine similarity between the topological fingerprint and the pre-set high-risk phantom fingerprint is extracted in real time. When the cosine similarity exceeds the similarity threshold and triggers an elastic repulsion field, it indicates that the local spatial topology has entered a congested state.

[0044] Real-time monitoring of the environmental adaptive speed adjustment coefficient; when the environmental adaptive speed adjustment coefficient triggers the underlying master controller to execute deceleration or emergency braking commands, it indicates that an extreme physical chassis intervention event has occurred.

[0045] When any of the following events occurs: a sudden change in the physical state of the environment, a congestion of the local spatial topology, or an extreme intervention event in the physical chassis, the circular buffer is used to backtrack to the historical state and extend to the future state, capturing a spatiotemporal context slice window that includes the preceding causes and the subsequent results.

[0046] The physical trajectory coordinates of the automated guided vehicle and the environmental context representation vector in the spatiotemporal context slice window are structured and packaged to generate a spatiotemporal tracing sequence. Trigger event type labels are added to the spatiotemporal tracing sequence and stored in the cloud spatiotemporal graph database.

[0047] Furthermore, an energy manifold model is constructed to define the total scalar energy. Upon receiving a product quality defect alarm signal, the real-time energy scalar of the spatiotemporal tracing sequence corresponding to the timestamp is calculated and compared with the total scalar energy. The product defect tracing results are then output, including:

[0048] An energy manifold model is constructed that includes an encoder network, a decoder network, and a latent space energy assessment network. The defect-free generated normal feature tensor is projected onto a low-dimensional Riemannian manifold through the encoder network.

[0049] Based on the decoder network, the low-dimensional Riemannian manifold is reconstructed into the original high-dimensional space to generate reconstructed features. The Euclidean distance between the normal feature tensor and the reconstructed features is calculated as the orthogonal projection error of the normal feature tensor on the surface of the normal Riemannian manifold.

[0050] The total scalar energy is defined based on the weighted result of orthogonal projection error and manifold surface latent energy, and the formula for calculating the total scalar energy is as follows:

[0051] ;

[0052] In the formula, Represents total scalar energy. Represents the balance coefficient. Represents the environmental context representation vector. Indicates the decoder network, Indicates the encoder network. Represents a latent space energy assessment network;

[0053] The system receives product quality defect alarm signals, retrieves the physical trajectory of the product during its flow in the workshop from the spatiotemporal tracing sequence, calculates the real-time energy scalar, compares it with the total scalar energy, determines the disaster contribution, and outputs the product defect tracing results.

[0054] Furthermore, upon receiving product quality defect alarm signals, the system retrieves the physical trajectory of the product during its flow in the workshop from the spatiotemporal tracing sequence, calculates the real-time energy scalar, compares it with the total scalar energy, determines the disaster contribution, and outputs the product defect tracing results, including:

[0055] Based on the alarm batch identifier, the physical trajectory of the corresponding batch during the workshop flow is retrieved in reverse from the spatiotemporal tracing sequence of the cloud spatiotemporal graph database, and the real-time energy scalar corresponding to each timestamp in the spatiotemporal tracing sequence is calculated.

[0056] When the real-time energy scalar is greater than the total scalar energy, the location of the anomaly is locked, and the partial derivative of the anomaly location with respect to the feature tensor is determined. The partial derivative is then multiplied by the sensitivity weight to obtain the process-aware disaster contribution.

[0057] The feature tensor corresponding to the maximum contribution of process perception to disaster is used as the disaster-causing factor. A defect report containing accurate coordinates, time period, process status and weighted causes is generated, and the defect source tracing results are output.

[0058] Secondly, the present invention also provides an edge-cloud collaborative scheduling system based on multimodal and deadlock-preventing heterogeneous graphs, the system comprising:

[0059] The modal adaptive fusion module is used to generate feature tensors based on the processing results of multimodal raw data, and to calculate the environmental context representation vector and environmental adaptive speed adjustment coefficient corresponding to the feature tensors using orthogonal projection and second-order distance calculation techniques based on transmission cost.

[0060] The anti-deadlock heterogeneous modeling module is used to construct a heterogeneous scheduling graph, calculate the expected value of the candidate movement based on the heterogeneous scheduling graph, and combine it with the environmental context representation vector and the environmental adaptive speed adjustment coefficient to determine the physical drive speed command of the automated guided vehicle.

[0061] The event-driven interception module is used to perform edge event matrix activation matching processing after the physical drive speed command is issued and executed, and to capture the spatiotemporal context slice window to generate a spatiotemporal tracing sequence after the edge event matrix activation matching.

[0062] The defect tracing module is used to construct an energy manifold model to define the total scalar energy, and when a product quality defect alarm signal is received, it calculates the real-time energy scalar corresponding to the timestamp of the spatiotemporal tracing sequence, compares it with the total scalar energy, and outputs the product defect tracing result.

[0063] The beneficial effects of this invention are as follows:

[0064] 1. This invention uses an AGV equipped with a multimodal sensor array as a mobile sensing node and introduces an adaptive fusion mechanism based on dynamic confidence and conflict resolution. The mobile AGV autonomously navigates the entire workshop area, effectively eliminating physical blind spots such as the top of traditional equipment and pipe interlayers, achieving high coverage of workshop monitoring. At the same time, the on-board high-frequency sensor supports continuous monitoring and can promptly capture transient anomalies such as equipment start-up and shutdown impacts or brief pollution outbreaks. By calculating spatial information entropy, the confidence of each modality is evaluated in real time. When a sensor is interfered with (such as when the lens is blocked), its weight is automatically reduced exponentially. At the same time, the modal conflict measurement factor is extracted. When serious disagreements occur between sensors, the underlying physical control module is forced to output deceleration or emergency braking commands, achieving absolute physical safety in complex environments. Compared with the traditional requirement of drilling and wiring, it can reconstruct the monitoring layout in a very short time according to changes in workshop processes, significantly reducing hardware deployment and modification costs.

[0065] 2. This invention adopts a cloud-edge-device collaborative architecture, combining a deadlock-preventing heterogeneous graph network with a jitter-preventing hysteresis control mechanism. By sinking the key control links, it achieves rapid emergency braking at the vehicle end and efficient real-time inference at the edge nodes, while the cloud focuses on global overall optimization, completely breaking the response latency bottleneck of the centralized architecture. Furthermore, it introduces deadlock phantom recognition and continuous elastic constraint fields into the heterogeneous graph network, and forcibly blocks the information transmission of high-risk deadlock paths through the underlying exponential repulsion force and topology penalty factor, fundamentally eliminating the paralysis problem during large-scale cluster scheduling, greatly improving scalability. At the time-series decision level, a cascaded anti-jitter threshold judgment based on a sliding observation window effectively filters invalid instruction switching caused by minor environmental disturbances, significantly reducing the computing power consumption of edge nodes and the mechanical wear of the AGV chassis.

[0066] 3. This invention achieves precise source tracing by establishing a multimodal spatiotemporal knowledge graph. It synchronously records product batches, AGV trajectories, and environmental multimodal characteristics in the cloud. When a quality warning occurs, the root cause tracing accuracy is narrowed down to precise physical coordinates and the exact time period of the anomaly, and a defect factor report is automatically generated. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 This is a flowchart of an edge-cloud collaborative scheduling method based on multimodal and deadlock-preventing heterogeneous graphs according to an embodiment of the present invention;

[0069] Figure 2 This is a multimodal fusion framework diagram in the edge-cloud collaborative scheduling method based on multimodal and deadlock-preventing heterogeneous graphs according to an embodiment of the present invention;

[0070] Figure 3 This is a flowchart of the scheduling decision process in the edge-cloud collaborative scheduling method based on multimodal and deadlock-preventing heterogeneous graphs according to an embodiment of the present invention. Detailed Implementation

[0071] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention.

[0072] According to embodiments of the present invention, an edge-cloud collaborative scheduling method and system based on multimodal and deadlock-preventing heterogeneous graphs are provided.

[0073] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figures 1 to 3 As shown, according to an embodiment of the present invention, an edge-cloud collaborative scheduling method based on multimodal and deadlock-prevention heterogeneous graphs includes:

[0074] Step S1: Generate a feature tensor based on the multimodal raw data processing results, and use orthogonal projection and second-order distance calculation techniques of transmission cost to calculate the environmental context representation vector and environmental adaptive speed adjustment coefficient corresponding to the feature tensor.

[0075] In one embodiment, generating a feature tensor based on the multimodal raw data processing results, and calculating the environmental context representation vector and environmental adaptive speed adjustment coefficient corresponding to the feature tensor using orthogonal projection and second-order distance calculation techniques for transmission cost, includes: acquiring vibration, particulate matter, and visual image data collected by the automated guided vehicle and fixed sensing devices to obtain multimodal raw data; performing timestamp alignment processing on the multimodal raw data; generating a feature tensor after orthogonal projection processing on the multimodal data; performing differentiation processing on the feature tensor; and combining it with the channel dimension to construct... The structural response matrix is ​​constructed to determine the environmental context representation vector. Based on the multivariate Gaussian distribution parameters of the feature tensor output by the pre-coding layer in the latent space, the mean vector and diagonal covariance matrix are determined, and the feature distribution of the feature tensor is output according to the mean vector and diagonal covariance matrix. The transmission cost required to convert the feature distribution of any feature tensor into another distribution form is calculated using optimal transmission theory, the second-order distance is determined, and the feature tensor with the largest second-order distance is selected as the global modal conflict metric factor. The global modal conflict metric factor is introduced into the servo control logic to calculate the environmental adaptive speed adjustment coefficient.

[0076] In one embodiment, after orthogonally projecting multimodal data to generate a feature tensor, and performing differentiation on the feature tensor, the environmental context representation vector is determined by constructing a structural response matrix in conjunction with the channel dimension. This includes: extracting high-dimensional feature vectors from the multimodal data based on the pre-coding layer, inputting the high-dimensional feature vectors into the orthogonal projection layer, stripping redundant shared information between modalities, mapping them to a shared latent space with a fixed dimension, and generating a feature tensor of uniform dimension. The feature tensor includes visual features, physical perception features, and anomaly type encoding. The feature tensor is then differentiated using the Laplacian operator to obtain a gradient feature map representing the degree of local high-frequency mutations, and a structural response matrix representing spatial topological changes is generated in conjunction with the channel dimension. The structural response matrix is ​​normalized to a structural pseudo-probability matrix that satisfies probability distribution characteristics to obtain the relative information weights contained at spatial coordinates, and the spatial local structural entropy of the feature tensor is calculated using the structural pseudo-probability matrix. The spatial local structural entropy is used as an exponential decay term to generate dynamic confidence gating coefficients, and the environmental context representation vector is determined by weighted fusion of the feature tensor and the dynamic confidence gating coefficients.

[0077] Step S2: Construct a heterogeneous scheduling graph, calculate the expected value of the candidate movement based on the heterogeneous scheduling graph, and combine it with the environmental context representation vector and the environmental adaptive speed adjustment coefficient to determine the physical drive speed command of the automated guided vehicle.

[0078] In one embodiment, the construction of a heterogeneous scheduling graph, the calculation of the expected value of candidate movement actions based on the heterogeneous scheduling graph, and the determination of the physical drive speed command of the automated guided vehicle (AGV) in combination with the environmental context representation vector and the environmental adaptive speed adjustment coefficient include: constructing a heterogeneous scheduling graph covering the AGV, process equipment, and task nodes based on the workshop state and AGV coordinates using physical fingerprint anchoring and forward projection techniques; determining the dynamic gravity coefficient and exponential repulsion force based on the AGV nodes and task nodes, and generating a deadlock penalty factor by constructing a label multiset through the joint feature sequence of the local topology in the local heterogeneous scheduling graph; and combining the dynamic gravity coefficient, exponential repulsion force, and deadlock penalty... Factors are introduced into the attention aggregation mechanism to output the final representation of the spatial structure after deep aggregation, and to determine the spatial topological constraint results in order to eliminate the deadlock situation of the automated guided vehicle cluster. The evolutionary features of the final representation of the spatial structure are extracted using the reset gate to generate the spatiotemporal hidden state representation. The spatiotemporal hidden state representation, the environmental context representation vector, and the action embedding vector are concatenated and input into the value network to decode and calculate the expected value of each candidate movement action of the automated guided vehicle. The movement candidate action with the expected value greater than the expected value of the current movement of the automated guided vehicle is selected as the optimal action. The physical driving speed command is generated by combining the environmental adaptive speed adjustment coefficient and sent to the automated guided vehicle master controller.

[0079] In one embodiment, based on workshop status and automated guided vehicle (AGV) coordinates, a heterogeneous scheduling graph encompassing AGVs, process equipment, and task nodes is constructed using physical fingerprint anchoring and forward projection techniques. This includes: determining AGV nodes based on global coordinates and load rate; determining process equipment nodes based on equipment coordinates and operating status; and determining task nodes based on task priority and time tolerance. The AGV nodes, process equipment nodes, and task nodes are combined to obtain heterogeneous nodes. A nonlinear hash function is used to perform feature-level concatenation of the heterogeneous nodes with timestamps and node physical medium access control addresses to generate tamper-resistant spatiotemporal physical fingerprints. Finally, a linear transformation matrix is ​​used to project the heterogeneous nodes... The image is projected onto the same-dimensional feature space to obtain an isomorphic feature tensor. Based on the isomorphic feature tensor, a forward transformation matrix is ​​used to generate a reconstructed original feature set. The L2-norm projection reconstruction residual between the heterogeneous nodes and the reconstructed original feature set is calculated to verify the node retention during the mapping process. A dynamic projection fault tolerance benchmark threshold is constructed based on the condition number of the forward transformation matrix and the electromagnetic environment noise intensity. Logical judgment is performed based on the dynamic projection fault tolerance benchmark threshold, node retention, and spatiotemporal physical fingerprint. Contaminated nodes in the heterogeneous nodes are removed based on the logical judgment results. The automated guided vehicle is then used as the center for topology connection processing based on the remaining heterogeneous nodes. A local heterogeneous scheduling graph is generated based on the topology connection results.

[0080] In one embodiment, determining the dynamic gravity coefficient and exponential repulsion force based on the automated guided vehicle (AGV) node and the task node, and generating a deadlock penalty factor by constructing a label multiset through the joint feature sequence of the local topology in the local heterogeneous scheduling graph includes: extracting the semantic attributes of all heterogeneous nodes in the local heterogeneous scheduling graph, assigning initial labels to each heterogeneous node, performing aggregation iteration processing based on the initial labels to obtain the set of first-order neighbor nodes for each heterogeneous node; extracting the label set of the first-order neighbor node set in the previous aggregation iteration, sorting the label set in lexicographical order to obtain the neighbor label set, and concatenating the current label of each heterogeneous node with the neighbor label set to generate a joint feature sequence representing the local topology of each heterogeneous node; mapping the joint feature sequence to fixed-length discrete labels using a one-way hash function, collecting the fixed-length discrete labels of all heterogeneous nodes in the local heterogeneous scheduling graph to obtain a label multiset; counting the occurrence frequency of each type of label in the label multiset, mapping the occurrence frequency to a fixed-length global topological feature vector as the topological fingerprint of the local heterogeneous scheduling graph, and calculating the cosine similarity between the topological fingerprint and the fingerprint of a pre-set high-risk motif to generate a deadlock penalty factor.

[0081] Step S3: After the physical drive speed command is issued and executed, perform edge event matrix activation matching processing, and after edge event matrix activation matching, capture the spatiotemporal context slice window to generate a spatiotemporal tracing sequence.

[0082] In one embodiment, after the physical drive speed command is issued and executed, edge event matrix activation matching processing is performed, and after edge event matrix activation matching, spatiotemporal context slice windows are extracted to generate spatiotemporal tracing sequences. This includes: after the physical drive speed command is issued and executed, extracting the global modal conflict measurement factor, and triggering a perception confidence anomaly event when the global modal conflict measurement factor exceeds the safety tolerance threshold or the dynamic confidence gating coefficient of any feature tensor decays, indicating a sudden change in the physical state of the environment; and extracting the cosine similarity between the topological fingerprint and the pre-set high-risk phantom fingerprint in real time, and when the cosine similarity exceeds the similarity threshold and an elastic repulsion field is triggered, indicating that the local spatial topology has entered a congested state. Real-time monitoring of the environmental adaptive speed adjustment coefficient. When the environmental adaptive speed adjustment coefficient triggers the underlying master controller to execute deceleration or emergency braking commands, it indicates that an extreme intervention event of the physical chassis has occurred. When any of the following events occurs: a sudden change in the physical state of the environment, a local spatial topology entering a congested state, or an extreme intervention event of the physical chassis, the system uses a circular buffer to backtrack to the historical state and extend to the future state, capturing a spatiotemporal context slice window containing the preceding causes and subsequent results. The system then structures and packages the physical trajectory coordinates of the automated guided vehicle and the environmental context representation vector in the spatiotemporal context slice window to generate a spatiotemporal tracing sequence, and adds trigger event type label encoding to the spatiotemporal tracing sequence and stores it in the cloud spatiotemporal graph database.

[0083] Step S4: Construct an energy manifold model to define the total scalar energy, and when receiving a product quality defect alarm signal, calculate the real-time energy scalar corresponding to the timestamp of the spatiotemporal tracing sequence, compare it with the total scalar energy, and output the product defect tracing result.

[0084] In one embodiment, an energy manifold model is constructed to define the total scalar energy. Upon receiving a product quality defect alarm signal, the real-time energy scalar corresponding to the timestamp of the spatiotemporal tracing sequence is calculated and compared with the total scalar energy to output the product defect tracing result. This includes: constructing an energy manifold model containing an encoder network, a decoder network, and a latent space energy assessment network; projecting the normal feature tensor generated without defects onto a low-dimensional Riemannian manifold through the encoder network; reconstructing the low-dimensional Riemannian manifold into the original high-dimensional space based on the decoder network to generate reconstructed features; calculating the Euclidean distance between the normal feature tensor and the reconstructed features as the orthogonal projection error of the normal feature tensor on the surface of the normal Riemannian manifold; defining the total scalar energy based on the weighted result of the orthogonal projection error and the latent energy of the manifold surface; receiving a product quality defect alarm signal; retrieving the physical trajectory of the product during its flow in the workshop from the spatiotemporal tracing sequence; calculating the real-time energy scalar; comparing it with the total scalar energy; determining the disaster contribution; and outputting the product defect tracing result.

[0085] In one embodiment, receiving a product quality defect alarm signal, retrieving the physical trajectory of the product during its flow in the workshop from the spatiotemporal tracing sequence, calculating the real-time energy scalar, and comparing it with the total scalar energy to determine the disaster contribution and output the product defect tracing result includes: based on the alarm batch identifier, retrieving the physical trajectory experienced by the corresponding batch during its flow in the workshop from the spatiotemporal tracing sequence in the cloud spatiotemporal graph database, and calculating the real-time energy scalar corresponding to each timestamp in the spatiotemporal tracing sequence; when the real-time energy scalar is greater than the total scalar energy, locking the location of the anomaly, and determining the partial derivative of the anomaly location with respect to the feature tensor, performing Hadamard product processing on the partial derivative and sensitivity weight to obtain the process-aware disaster contribution; using the feature tensor corresponding to the maximum process-aware disaster contribution as the disaster factor, generating a defect report containing accurate coordinates, time period, process status, and weighted causes, and outputting the defect tracing result.

[0086] According to another embodiment of the present invention, an edge-cloud collaborative scheduling system based on multimodal and deadlock-prevention heterogeneous graphs is also provided, the system comprising:

[0087] The modal adaptive fusion module is used to generate feature tensors based on the processing results of multimodal raw data, and to calculate the environmental context representation vector and environmental adaptive speed adjustment coefficient corresponding to the feature tensors using orthogonal projection and second-order distance calculation techniques based on transmission cost.

[0088] The anti-deadlock heterogeneous modeling module is used to construct a heterogeneous scheduling graph, calculate the expected value of the candidate movement based on the heterogeneous scheduling graph, and combine it with the environmental context representation vector and the environmental adaptive speed adjustment coefficient to determine the physical drive speed command of the automated guided vehicle.

[0089] The event-driven interception module is used to perform edge event matrix activation matching processing after the physical drive speed command is issued and executed, and to capture the spatiotemporal context slice window to generate a spatiotemporal tracing sequence after the edge event matrix activation matching.

[0090] The defect tracing module is used to construct an energy manifold model to define the total scalar energy, and when a product quality defect alarm signal is received, it calculates the real-time energy scalar corresponding to the timestamp of the spatiotemporal tracing sequence, compares it with the total scalar energy, and outputs the product defect tracing result.

[0091] To facilitate understanding of the above technical solutions of the present invention, the working principle or operation method of the present invention in actual process will be described in detail below.

[0092] This embodiment utilizes a three-layer asynchronous architecture to achieve dynamic task allocation, path optimization, and absolute physical safety safeguards under complex process constraints. Key aspects include disturbance-resistant multimodal fusion, spatial modeling based on a deadlock-resistant heterogeneous graph network (Hetero-DQN), temporal decision-making with debouncing mechanisms, and precise defect tracing based on energy manifold mapping. Specifically, this embodiment first achieves disturbance-resistant production environment perception through a multimodal fusion mechanism based on dynamic confidence and conflict resolution; secondly, it extracts spatial topological constraints based on a heterogeneous graph neural network with deadlock-resistant motif recognition and elastic constraint fields; subsequently, it captures temporal evolution patterns and issues physical commands by combining gated recurrent units (GRUs) and a debouncing event triggering mechanism; finally, it achieves macro-level strategy control and precise defect tracing across the entire chain through cloud-based asynchronous large-scale model pre-playing and spatiotemporal knowledge graphs. Figure 1 The diagram shown is the overall framework diagram provided in this embodiment. The specific working principle is as follows:

[0093] Step 1: The perception layer uses multimodal adaptive fusion based on dynamic confidence and conflict resolution;

[0094] In complex and dynamically disruptive industrial environments, single-dimensional sensors (such as visual or physical sensors) are prone to localized failures due to factors like occlusion and noise, leading to scheduling accidents. This embodiment designs an anti-interference multimodal adaptive fusion mechanism. The core idea of ​​this mechanism is to achieve data alignment by overcoming the spatiotemporal sampling barriers between different sensors. Then, during feature fusion, the information entropy of each modality is quantified to dynamically remove contaminated features. Finally, when multiple sensing data points severely conflict, a low-level security defense mechanism is forcibly triggered, such as… Figure 2 The diagram shown is a flowchart of the multimodal adaptive fusion mechanism in this embodiment.

[0095] Edge computing nodes acquire real-time multimodal raw data, including vibration, particulate matter, and visual images, from mobile AGV (Automated Guided Vehicle) robots and fixed sensing devices. High-precision timestamp alignment is performed using a circular buffer, and a unified-dimensional feature tensor is generated through latent space mapping. This latent space mapping employs a multi-branch asymmetric autoencoder architecture. First, high-dimensional feature vectors are extracted from different modal data through specific pre-encoding layers (visual data is input into a CNN branch, and one-dimensional vibration and particulate matter sequences are input into a 1D-ResNet branch). Second, the high-dimensional feature vectors are input into an orthogonal projection layer to remove redundant shared information between modalities. Finally, the data is mapped to a fixed-dimensional shared latent space (e.g., 256 dimensions) to generate a unified-dimensional feature tensor: visual features. Physical perception characteristics and exception type encoding .

[0096] Specifically, to address the timing misalignment issue caused by the varying sampling frequencies of different sensors in the workshop (e.g., 30Hz for vision, 1000Hz for vibration), a ring-shaped buffer with hardware timestamp compensation was constructed at the bottom layer. This buffer extracts the raw data from various sensors and generates a feature tensor with uniform dimensions through time-sliding window interpolation and latent space mapping: visual environment features. Physical sensing characteristics (vibration / particulate matter, etc.) and exception type encoding .

[0097] To accurately quantify the degree of sensor damage in complex, high-precision manufacturing environments (such as machining or wafer fabrication workshops containing a large number of regular geometric structures), this embodiment breaks through the limitation of traditional global information entropy ignoring spatial topology and innovatively introduces spatial local structure entropy based on information geometry. When a mode is physically obscured (e.g., structural blurring caused by cutting fluid / water vapor) or subjected to strong electromagnetic interference (e.g., a surge in noise), the high-frequency gradient field distribution of its feature map will deviate significantly from the statistical prior distribution of the normal physical structure.

[0098] To prevent damaged modes from contaminating global features, the spatial information entropy of each modal feature is calculated in real time. The calculation process is as follows:

[0099] (1) Local structural gradient mapping extraction: The Laplacian operator is used to extract the multi-channel feature tensor. We perform a second-order differential operation on the spatial dimension to obtain a gradient feature map representing the degree of local high-frequency abrupt changes. The mean of the absolute values ​​along the channel dimension is calculated to generate a structural response matrix representing spatial topological changes. Its spatial coordinates The value at that location is:

[0100] ;

[0101] (2) Structural distribution normalization: The local structural response is normalized to a structural pseudo-probability matrix that satisfies the probability distribution characteristics, and the coordinates are obtained. Weights of relative structural information contained at each location :

[0102] ;

[0103] in, To prevent extremely small smoothing constants with denominators of zero (e.g., values ​​such as...) ).

[0104] (3) Calculation of spatial local structural entropy: Based on the above pseudo-probability distribution of structure, calculate the spatial local structural entropy of this modal feature. :

[0105] ;

[0106] In obtaining the degree of damage to the modal structure This was then introduced as an exponential decay term to generate dynamic confidence gating coefficients:

[0107] ;

[0108] in, The discrete Laplacian operator is used to extract high-frequency information about edges and textures in the feature space. These are the learnable weight matrix and bias vector in the gated network, respectively. The sigmoid activation function is used to map the linear output to... interval, This is the Hadamard product (element-by-element multiplication). is a hyperparameter, the structural entropy decay coefficient, used by the controller to penalize the confidence level of damaged modes.

[0109] Then, a weighted fusion is performed to generate an environmental context representation vector. :

[0110] ;

[0111] At the same time, a modal conflict measurement factor was introduced. When there are significant discrepancies in the information provided by different modalities (e.g., extremely low confidence in visual perception versus extremely high confidence in physical perception), the divergence (such as variance or KL divergence) of the latent space distribution of each modality's features is calculated as a conflict factor. .

[0112] Physical security-driven speed hard constraint delivery: from the fusion context Decoding the basic regional risk scalar To ensure the absolute physical safety of the AGV, this embodiment directly incorporates the modal conflict factor into the underlying servo control logic to calculate the environmental adaptive speed adjustment coefficient. .

[0113] Specific implementation of modal conflict resolution and underlying velocity constraints: To accurately quantify conflicts, this embodiment uses the second-order Wasserstein distance based on optimal transmission theory to calculate the distribution divergence of each modal feature in the latent space, as a modal conflict metric. The specific implementation steps and calculation formulas are as follows:

[0114] Distributed parameterized output: The pre-amplifier multi-branch encoder no longer outputs a single exact vector, but instead outputs the outputs for each mode. The parameters of the multivariate Gaussian distribution in the latent space, i.e., the mean vector. With the diagonal covariance matrix At this time, the first The feature distribution of each mode is represented as follows: .

[0115] Second-order Wasserstein distance calculation: Extracting any two different modalities (e.g., visual modality distribution) With physical perception modal distribution Based on optimal transmission theory, the minimum transmission cost required to transform one distribution pattern into another is calculated, i.e., the second-order Wasserstein distance. :

[0116] ;

[0117] in Let be the squared value of the second-order Wasserstein distance between the two probability distributions. Let be the mean vector of the distributions of the visual modality and the physical perception modality in the latent space. Here is the corresponding covariance matrix. The trace of the matrix (i.e., the sum of the elements on the main diagonal).

[0118] Since the covariance matrix in this embodiment is designed as a diagonal matrix, the above formula can be greatly simplified, thereby reducing the computing power consumption of edge computing:

[0119] ;

[0120] in For visual modal distribution in the latent space Mean and standard deviation in each dimension The feature dimension of the shared latent space (e.g., 256 dimensions).

[0121] Divergence aggregation: Traverse all available mode combinations and take the largest pairwise Wasserstein distance as the global mode conflict metric at the current time step.

[0122] ;

[0123] Compared to conventional variance, this distance is more sensitive to capturing the geometric shift costs between heterogeneous manifolds. When there are significant discrepancies between different modal intelligences, The factor is significantly increased, and it is directly incorporated into the underlying speed regulation calculation formula to calculate the environmental adaptive speed regulation coefficient. :

[0124] ;

[0125] in, For risk sensitivity coefficient, This is the conflict penalty coefficient. Based on the bias.

[0126] The physical significance and beneficial effects of this mechanism are that it not only filters noise at the network level, but also provides protection at the physical execution level, when serious conflicts or interference between sensors are detected. When the speed increases, even if the scheduling network considers the path ahead safe, the underlying adjustment coefficient will still be affected. It will also be forced to decay exponentially by mathematical mechanisms, forcing the AGV into a deceleration cruise or emergency braking state. This cross-modal fusion mechanism has industrial-grade fault tolerance and extremely high redundancy.

[0127] Step 2: Spatial constraint modeling of the edge layer based on the anti-deadlock heterogeneous graph network;

[0128] Traditional graph aggregation methods tend to overlook the risk of deadlock in AGV groups and handle hard constraints too rigidly. This embodiment constructs a local heterogeneous graph at the edge nodes, encompassing AGVs, process equipment, and tasks. ,like Figure 3 The diagram shown is a flowchart of the robot scheduling decision-making process based on heterogeneous graph networks and gated recurrent units (GRUs) in this embodiment. After extracting local spatial constraints through the graph network, the constraints are input into the GRU network for temporal evolution and decision output.

[0129] Edge computing nodes use the environmental context representation vector output in step one. Based on the current workshop status, a local heterogeneous scheduling graph covering AGVs, process equipment, and task nodes is constructed in real time. .

[0130] (1) Node definition and graph construction rules: AGV nodes include a unique ID and global coordinates. Current velocity vector and load rate data; process equipment nodes include equipment physical bounding box coordinates and current working status (idle / processing / fault); task nodes include task priority, target coordinates and time tolerance.

[0131] By using a node-specific linear transformation matrix, the aforementioned heterogeneous data is uniformly projected into a feature space of the same dimension. To ensure the accuracy of the projected heterogeneous data and to prevent data tampering and loss during transmission or mapping from the underlying layer, a closed-loop consistency verification mechanism based on pseudo-inverse reconstruction and spatiotemporal hashing is introduced in the node feature homogenization stage. The following verification steps are specifically executed:

[0132] Spatiotemporal physical fingerprint anchoring (anti-tamper verification beforehand): on heterogeneous nodes The original feature vector Before the linear transformation layer (for node types), edge nodes are compared with the current high-precision timestamp using a non-linear hash function. and node physical media access control (MAC) address Feature-level spatiotemporal physical fingerprints are generated that are resistant to replay and tampering. :

[0133] ;

[0134] Feature forward projection: by node type Dedicated linear transformation matrix The original heterogeneous data is projected onto a feature space of uniform dimension to generate isomorphic feature tensors. :

[0135] ;

[0136] Bidirectional mapping reconstruction and residual quantization (loss prevention and distortion verification): feature extraction To verify the information retention rate of the mapping process, the forward transformation matrix was then used. Moore-Penrose generalized pseudoinverse matrix In the reverse reconstruction of the bypass execution features, the original reconstructed feature set is generated. :

[0137] ;

[0138] Subsequently, the L2-norm projection reconstruction residual between the original input features and the reconstructed features is calculated. :

[0139] ;

[0140] Dynamic isolation and blocking decision-making: through real-time monitoring of the transformation matrix condition number With the current electromagnetic environment noise intensity Construct a dynamic projection fault tolerance benchmark threshold :

[0141] ;

[0142] in The preset sensitivity adjustment coefficient is used, and the following judgment logic is executed:

[0143] If detected If the node has suffered unacceptable information loss or computational overflow during isomorphic projection, then the feature tensor carries a physical fingerprint. If no match is found during the secondary verification at the edge, it is determined that illegal data tampering has occurred.

[0144] Any node that fails the above verification is marked as a polluted node, and its entry into the heterogeneous graph is forcibly terminated. The topological connections in the model trigger resampling of the perception layer, thereby achieving absolute physical security at the lowest level of spatial modeling.

[0145] After completing the closed-loop verification and removing contaminated nodes, the local area is defined using a dynamic topology radius. That is, with the current AGV as the center, all highly reliable related nodes whose physical distance is within a preset radius (e.g., 15 meters) and whose topological connection is within 2 hops are extracted.

[0146] (2) Construction of dynamic task gravitational field: in local heterogeneous scheduling graph After construction, a dynamic task gravity field mechanism based on spatiotemporal properties is introduced to accurately quantify the driving preferences of task nodes on AGV nodes. This mechanism is applied to AGV nodes. With task nodes Extract the time tolerance (i.e., time tolerance threshold) of the task. With waiting time Normalization is performed:

[0147] ;

[0148] And combine task priority (value) AGV current load rate and the actual physical distance between the two Calculate the dynamic gravitational coefficient :

[0149] ;

[0150] in, Scaling weights for task value; A minimal smoothing constant to prevent division-to-zero crashes; This is a time urgency sensitivity coefficient.

[0151] (3) Calculation of exponential repulsive force: During the graph convolution propagation process, the traditional binary mask limitation is broken through, and a continuous elastic constraint field based on spatial distance is constructed to generate exponential repulsive force. The constraint field adopts a local polar coordinate system with the geometric center of AGV as the pole. The baseline modeling position is the physical outer contour edge of the AGV. The basic rule is: the repulsive force is zero outside the safe distance, and it approaches the dynamic braking process red line distance. It showed a dramatic increase at times. (Regarding nodes) with neighboring nodes Generates exponential repulsive force :

[0152] ;

[0153] in, The elastic repulsion coefficient, The actual physical distance between nodes is used. Simultaneously, a pre-defined deadlock phantom identifier is used to detect high-risk deadlock topologies and generate a deadlock penalty factor. .

[0154] In the process of preventing deadlock motif penalties and elastic repulsion aggregation, this embodiment breaks through the traditional binary mask (compliance is 1, violation is 0) and constructs a continuous elastic constraint field. When the graph network detects that the physical distance between nodes is approaching the process boundary distance... At this time, the repulsive force increases exponentially. Simultaneously, a deadlock phantom identifier is pre-installed in the graph topology. When a high-risk deadlock topology is detected in a local subgraph (e.g., multiple AGVs entering the same unidirectional connected component to form a loop waiting, or bidirectional entry into a narrow passage), a topological deadlock penalty factor is extracted. .

[0155] (4) Deadlock Module Configuration and Penalty Factor Generation: Deadlock module refers to a predefined subgraph topology that is prone to paralysis, mainly including directed ring waiting modules (such as multiple vehicles connected end to end in a one-way channel) and narrow channel bidirectional conflict modules.

[0156] This embodiment employs a local graph hashing technique based on the Weisfeiler-Lehman (WL) algorithm to achieve fast isomorphic matching between real-time local subgraphs and a pre-set deadlock phantom library. Since industrial site scheduling has extremely high real-time requirements, traditional exact graph isomorphism algorithms have NP-hard time complexity. The WL graph hashing algorithm, however, can generate a unique topological fingerprint (HashSignature) for any topology in polynomial time through iterative aggregation of neighbor node features, thereby achieving efficient structural similarity calculation. The specific implementation steps are as follows:

[0157] Initial semantic label assignment for nodes (iteration 0): Extracting real-time local subgraphs The semantic attributes of all nodes (including AGV nodes and process equipment nodes) are determined based on discrete features such as node type, direction of movement, and path occupancy status for each node. Assign initial labels For example, a fully loaded AGV traveling in the forward direction can be coded as tag A1 using a preset dictionary, and the processing equipment in operation can be coded as tag M2.

[0158] Neighborhood topological feature sorting and aggregation (Part 1) Sub-iteration): In each aggregation iteration ( ,in In this embodiment, the maximum number of receptive field hops is set to... Within the spatial constraints covering a 2-hop range, obtain the nodes. set of all first-order neighbor nodes Extract the labels of all neighboring nodes from the previous iteration. The set of labels is sorted strictly in lexicographical order (Sort operation). The purpose of sorting is to ensure that the spatial arrangement order of neighboring nodes is permutation invariant during the topological aggregation process.

[0159] Local structure hash mapping and label update: Node Its own current tag Concatenate strings with the sorted neighbor tag set (using symbols) (Representation), generating a joint feature sequence representing its local topology, and using a lightweight one-way hash function to map this joint sequence into new fixed-length discrete labels. :

[0160] ;

[0161] After this process, the node The new label encapsulates and integrates its Local spatial topology information within the jump range.

[0162] Global graph topological fingerprint vector generation: after After the next iteration, collect the subgraph. Collect the labels of all nodes in all iteration steps, construct a multiset of labels for the entire subgraph, count the frequency of each type of label in the multiset, and map it to a fixed-length global topological feature vector. This feature vector is the topological fingerprint of the current local spatiotemporal subgraph.

[0163] Motif isomorphism similarity calculation and penalty factor triggering: The same steps described above are used beforehand to calculate offline the dangerous subgraphs (such as directed circular waiting motifs) in the deadlock motif library. Standard topological fingerprint During real-time scheduling at the edge, the fingerprint of the current subgraph is calculated. The cosine similarity between the fingerprints and the fingerprints of each pre-set high-risk phantom is used as a structural similarity. :

[0164] ;

[0165] When detected When the preset isomorphism determination safety threshold (e.g., 0.85) is reached, the current spatial topology is determined to be highly fitted to a high-risk deadlock phantom. At this point, low-level defense intervention is triggered, and the following formula is applied: An exponential deadlock penalty factor is generated to forcibly cut off the Q-value of actions in the graph network that lead to the high-risk topology, thereby achieving deadlock avoidance at its source.

[0166] (5) Feature aggregation and secure output: The positive gravity coefficient repulsive force and By forcibly introducing an attention aggregation mechanism, information transmission along high-risk paths is blocked, resulting in the final representation of the spatial structure after deep aggregation. ,node In the Hidden state of a layer Aggregate updates are performed by combining gravity, repulsion, and deadlock penalties:

[0167] ;

[0168] in, Based on attention weights, This is a preset, minimal anti-crash smoothing constant. The physical meaning of this aggregation formula is: when a deadlock topology exists ( (approaching 0) or extremely close to dynamic obstacles ( When the number of cases increases dramatically, the transmission of feature information along that path is forcibly interrupted by mathematical mechanisms, forcing the neural network to represent the data in space more efficiently. The search for other safe topology solutions eliminates the fundamental defects of AGV cluster deadlock and mathematical calculation division-by-zero crash from the root.

[0169] Step 3: Timing evolution analysis and instruction issuance at the edge layer based on GRU and hysteresis control;

[0170] Edge computing nodes extract historical graph representation sequences within a preset time sliding window and input them into a gated recurrent unit (GRU). Through an update gate mechanism, historical information and current local observations are adaptively fused to generate a spatiotemporal hidden state representation.

[0171] Sliding window and GRU implementation details: The duration of the preset time sliding window is set to... seconds, sliding step size set to The historical graph representation sequence is a set of spatial structure representations generated at each discrete time step within this window. The update gate of the GRU determines how much of the hidden state from the previous time step is retained. And use the reset gate to extract the current spatial representation. The evolutionary characteristics in the data are used to generate a representation of the current spatiotemporal hidden state. :

[0172] ;

[0173] Combine it with the candidate action embedding vector After concatenation, the data is input into the value network, and the Q-value of each candidate action is calculated using decoding.

[0174] ;

[0175] in, and These are the weight matrix and bias terms of the value assessment network, respectively. This indicates a feature concatenation operation. The formula integrates spatiotemporal evolution patterns, environmental context states, and specific action features to output a quantitative assessment of future returns.

[0176] To eliminate control jitter, a preset anti-jitter hysteresis threshold is introduced for sliding window analysis. The implementation and threshold range of anti-jitter hysteresis control: The core logic of hysteresis control is similar to a Schmitt trigger. Let the currently executing action be... Only when the optimal Q-value of the new candidate action exceeds the Q-value of the current action, and the difference is greater than the preset stabilization hysteresis threshold. The instruction switch is only triggered at that time.

[0177] ;

[0178] Here The specific range value is set to the normalized Q range. Within this range, the mechanism effectively filters out invalid action switching caused by minor environmental disturbances. After selecting the optimal action, it combines the safe speed adjustment coefficient generated in step 1. Generate actual physical drive speed commands. Send to the vehicle's main controller, The standard physical kinematic parameters are defined for the AGV chassis to perform specific actions under ideal working conditions without any external environmental risks or multimodal conflicts.

[0179] Multi-objective piecewise reward function: In the above temporal evolution analysis and reinforcement learning decision-making process, model training is controlled by the piecewise reward / penalty function. , To hide the state in the current spacetime Below, the AGV executes actions. The total reward value (scalar) is then fed back. This is an absolute physical boundary penalty constant (extremely negative, e.g., set to -1000), triggered only when the underlying safety logic detects a collision between the AGV's physical outline and a device node, or when it enters a static restricted area. This variable acts as a veto mechanism, forcing rapid convergence to avoid disastrous actions. For dynamic multi-objective Pareto optimization, the weighting coefficients must satisfy... The system dynamically adjusts its power based on the priority of the current task and the remaining battery power of the AGV. For example, it increases the power output in low-battery mode. Increased in emergency missions , Performance bonus value, calculated based on actions. The product of the projected velocity on the target heading and the reciprocal of the remaining global path distance is used to motivate the AGV to reach the target in the optimal time. The energy-consumption penalty value (usually negative) is calculated based on the AGV's performance during actions. The sum of the squares of the angular acceleration and linear acceleration at time (i.e. This is used to suppress ineffective sudden stops and starts and high-frequency jitter, and to smooth the energy flow output. For the safety tolerance reward, a continuous elastic constraint field is used, taking the logarithmic function of the shortest physical distance between the current AGV's geometric center and its neighboring nodes. The further away from the red line, the greater the reward; approaching the dynamic braking red line... At that time, the reward dropped sharply, guiding the AGV to maintain the optimal process safety distance.

[0180] Step 4: High-value spatiotemporal context asynchronous slicing and cloud migration mechanism based on edge event-driven approach;

[0181] To overcome the bandwidth overflow and cloud computing power bottlenecks caused by uploading all industrial IoT data to the cloud in traditional methods, this embodiment deploys high-value spatiotemporal data slicers on edge computing nodes. It reuses the anomaly measurement factors generated in the underlying physical defense logic as triggering conditions to perform event-driven sparse sampling and asynchronous cloud uploading. The specific implementation process is as follows:

[0182] (1) Construction of core trigger event matrix: Edge nodes monitor the underlying operating status in real time, and activate the data slicing and cloud migration mechanism when any of the following high-risk physical or topological conditions are met:

[0183] Perceive confidence anomalies: Real-time extraction of modal conflict metric factors from step one. When this factor exceeds the preset safety tolerance threshold (i.e. ), or the dynamic confidence gating coefficient of a certain independent mode. Triggered when a step decay occurs, this event indicates a drastic change in the physical state of the environment (such as severe vibration, dust concentration exceeding the limit, or sudden obstruction of visual sensors).

[0184] Local topological deadlock defense event: Real-time extraction of structural similarity between local subgraphs and the phantom library in step two. and deadlock penalty factor When detected When the safety threshold is exceeded and a continuous elastic repulsive force field is triggered, this event indicates that the local spatial topology has entered a high-risk congestion state.

[0185] Physical chassis extreme intervention events: Real-time monitoring of the environmental adaptive speed adjustment coefficient output in step three. ,when Triggered when the underlying master controller executes a command for significant deceleration or emergency braking.

[0186] (2) Dynamic time window slicing and context packaging: Once the above event matrix is ​​activated, the edge nodes stop the regular low-frequency heartbeat reporting and instead report based on the trigger time. Centered on a circular buffer, the system backtracks to historical states and extends to future states, capturing a spatiotemporal context slice window that includes both preceding causes and subsequent results. .

[0187] (Previous History Retrospective Window Tolerance) indicates the time point since the triggering event. Starting from this point, the time period for tracing back against the historical timeline (e.g., set to 5 seconds) is used to record data within this timeframe, which is called the pre-cause zone. Because anomalies in industrial scenarios (such as congestion lockouts or abnormal chassis vibrations) often do not occur instantaneously, but rather evolve from a series of small, cumulative disturbances... The core objective is to completely capture the causes that led to this high-risk event (such as the sensor degradation process and the process of local topology gradually becoming denser).

[0188] (Later Future Extended Window Latitude) indicates the time point from the triggering event. Starting from this point, the observation duration is extended forward along the future timeline (e.g., set to 3 seconds). The data recorded within this time period is called the post-results area. Its core purpose is to record the dynamic response process and final stable state of the AGV physical chassis and the surrounding environment after triggering underlying defensive interventions (such as emergency braking, deceleration, and obstacle avoidance), in order to evaluate the execution decay and convergence effect of the defensive strategy.

[0189] This spatiotemporal context slice window is not a simple data record, but a core data filter in the edge-cloud asynchronous collaborative architecture of this embodiment. Its specific functions are reflected in the following three dimensions:

[0190] The first function is to construct a complete cause-effect physical evidence chain to support accurate defect tracing: traditional fault alarms often only record the moment the fault occurs (i.e., a single point). This results in a lack of context in the cloud when performing post-event quality defect tracing (such as energy manifold mapping as described in step five), and the slice window is packaged... This continuous time slice encapsulates the complete evolutionary process of steady state, deterioration, outbreak, intervention, and recovery into an independent high-dimensional feature sequence. This provides solid time-series data support for identifying the causes of disasters in the cloud (such as whether the product shifts due to vibration or whether a visual misjudgment causes an emergency stop).

[0191] The second function addresses the bandwidth overflow and storage bottlenecks caused by uploading all data to the cloud: Industrial field multimodal data (visual, high-frequency vibration, etc.) is extremely massive. Using traditional full-data real-time cloud upload solutions would directly paralyze the edge network bandwidth. This window mechanism utilizes event-driven logic at the edge to perform sparse sampling of non-steady-state, high-value data, maintaining only a low-frequency heartbeat during normal operation. Only when a high-risk event occurs are these few seconds of high-frequency data slices in the circular buffer packaged and asynchronously uploaded to the cloud. This approach reduces the overall network uplink bandwidth utilization by over 90% while preserving 100% fault context.

[0192] The third function is to eliminate timestamp misalignment in multi-sensor data at abrupt changes (alignment buffer): Under extreme conditions (such as AGV collisions or sudden stops), the data bus transmission delay of various heterogeneous sensors (vision cameras, one-dimensional vibration sensors) will fluctuate nonlinearly. If only single-point data is extracted, modal fusion can easily fail due to minor timestamp misalignment. This slicing window opens a buffer margin at the edge, allowing for realignment before packaging. Within the specified range, interpolation or smoothing algorithms are used to perform secondary high-precision alignment of the physical timestamps of each modality data to ensure that the high-dimensional tensors uploaded to the cloud are absolutely synchronized in time and space.

[0193] The AGV physical trajectory coordinates, high-precision timestamps, and fusion environment context representation vector generated in step one within the sliding window are used. Structured packaging is performed to generate high-value spatiotemporal tracing sequences. The sequence is tagged with the corresponding trigger event type and asynchronously pushed into the cloud spatiotemporal graph database during the idle period of computing power at the edge, thus completely decoupling the bandwidth conflict between high-frequency real-time scheduling and low-frequency heavy source tracing.

[0194] Step 5: Precise defect tracing based on process tolerance and energy manifold mapping;

[0195] Traditional industrial traceability systems often remain at the level of software logs or macroscopic trajectories, making it difficult to establish a precise causal link between underlying microscopic physical disturbances (such as high-frequency vibrations caused by AGVs braking suddenly at specific coordinates) and final product quality defects (such as dispensing misalignment and wafer scratches). The cloud-based defect attribution mechanism in this embodiment breaks down data silos. Its core idea is to treat the multimodal environmental characteristics under normal production conditions as a low-energy stable manifold. By calculating the energy partial derivatives of abnormal slices and combining them with the physical tolerance of the current specific process, the core physical causes leading to defects are decoupled in reverse. The specific working principle includes the following three core dimensions:

[0196] Asynchronous high-value spatiotemporal slicing and energy manifold modeling: To avoid bandwidth collapse caused by uploading all raw data to the cloud, the edge only extracts high-value spatiotemporal tracing sequences containing cause and effect when high-risk events are triggered (such as a surge in modal conflicts, getting trapped in a deadlock-prevention repulsion field, or emergency braking at the underlying level). It is asynchronously deployed to the cloud, where an energy distribution-based model is pre-trained based on defect-free normal historical feature tensors. This model maps high-dimensional environmental context feature tensors to Riemannian manifolds. Above, define the scalar energy function. .

[0197] When a quality defect alarm signal for a specific batch of products is received from an external source (such as a defective product notification issued by the MES system), the cloud server triggers an offline, in-depth defect root cause tracing mechanism. The specific implementation process is as follows:

[0198] (1) Batch sequence retrieval and energy manifold modeling:

[0199] Using the alarm batch ID, the cloud retrieves the complete physical trajectory of the batch during its flow in the workshop from the spatiotemporal graph database, and extracts the high-value spatiotemporal sequence uploaded asynchronously in step four. .

[0200] For historical characterization data under normal, defect-free production conditions, a pre-trained energy distribution model (i.e., energy manifold model) is used to map high-dimensional context feature tensors to Riemannian manifolds. Above, define the scalar energy function. And extract the energy distribution of normal samples. The quantile is set as the dynamic baseline environmental energy threshold. .

[0201] Model architecture and parameters: Encoder network It contains 3 fully connected layers (dimensions are as follows) The Mish activation function is used to transform the high-dimensional environmental context feature tensor. Compress and map to a low-dimensional local coordinate space (i.e., Riemannian manifold) The intrinsic dimension, in this embodiment, is set as follows: ).

[0202] Decoder Network The structure is symmetrical to the encoder (dimensions are as follows): ), responsible for representing low-dimensional values ​​on manifolds Reconstruct back to the original high-dimensional space and generate reconstructed features. .

[0203] Latent Space Energy Assessment Network A lightweight multilayer perceptron (MLP) with a two-layer structure is used to evaluate low-dimensional representations. A priori energy in latent space.

[0204] The specific steps of mapping and energy calculation are as follows: Manifold projection is input to the normal feature tensor generated from defect-free historical data. via encoder Projected onto a low-dimensional Riemannian manifold Above, we obtain the manifold coordinates. Manifold distance quantization calculates the Euclidean distance between the original and reconstructed features, which serves as the orthogonal projection error (i.e., reconstruction error) of the feature free on the surface of a normal Riemannian manifold. Scalar energy function construction defines the total scalar energy function. It is composed of the manifold orthogonal projection error and the manifold surface latent energy weighted average, and the specific calculation formula is as follows:

[0205] ;

[0206] in, Represents all learnable parameters of the model ; The square of the L2 norm characterizes the degree to which physical characteristics deviate from the normal process manifold; This is the balance coefficient (0.8 in this example). During model training, the normal samples are minimized. This forces the normal state to adhere tightly to the manifold surface.

[0207] (2) Precise location of spatiotemporal anomalies: Calculate the real-time energy scalar score of each feature vector in the high-value traceability sequence. When the energy score of a certain spatiotemporal node in the sequence satisfies At that time, accurately locate the coordinate point. and time points For sudden environmental anomalies, specifically when an alarm for a particular batch of defective products is received from the MES system, the cloud retrieves the data that the batch has experienced. Sequence, calculate each high-precision timestamp in the sequence. Real-time energy scalar corresponding to the feature Once the energy score of a node is found to meet the requirements... This allows for the precise pinpointing of the spatiotemporal coordinates of an anomaly within a lengthy production cycle. .

[0208] (3) Weighted decoupling of disaster-causing factors based on dynamic process sensitivity matrix: In order to clarify the specific physical causes of defects and eliminate environmental disturbances within the process tolerance, a dynamic process sensitivity matrix is ​​introduced to retrieve the time point of the abnormal slice in real time. Corresponding production process status (e.g., dispensing, curing, routine handling, etc.), and extract the product's condition for each independent environmental mode from the pre-set process library. Process sensitivity weights (visual, vibration, particulate matter, etc.) This weight is inversely proportional to the process tolerance of a specific process. Specifically, it is based on the causal decoupling of the background energy gradient and process sensitivity: after locating the anomaly, a dynamic process sensitivity matrix is ​​introduced. Since different production processes have drastically different tolerances to environmental disturbances (for example, the high-precision dispensing process is extremely sensitive to vibration and moderately sensitive to visual illumination; while the appearance inspection process is the opposite), the current process state is extracted. Below, the product supports various independent environmental modes. (Visual) Physical vibration Process sensitivity weights (etc.) .

[0209] Subsequently, the anomaly energy score is calculated relative to the input features of each independent modality at the bottom layer. The partial derivatives (i.e., the background energy gradient) are used, and combined with the process sensitivity weights, a Hadamard product is applied to calculate the final process-aware disaster contribution. :

[0210] ;

[0211] After extracting weighted average The mode corresponding to the maximum value is used as the core disaster-causing factor. The system automatically generates a defect root cause analysis report containing the exact coordinates, time period, process status and weighted causes. For example, at coordinates (X, Y), at time T, emergency braking was triggered by local congestion, which caused the chassis to vibrate at high frequency, which broke through the process tolerance limit of the current high-precision dispensing process, resulting in product offset defects.

[0212] This formula integrates objective physical perturbations (gradients) and subjective process tolerance (weights), and extracts the weighted values... The maximum value can automatically remove invalid environmental noise interference, directly lock in the core disaster-causing factors, and ultimately not only tell the manager that there is a congestion in a certain place in the workshop, but also provide in-depth attribution conclusions. For example, the high-frequency vibration caused by the AGV in coordinates (X, Y) to avoid the collision broke through the process tolerance bottom line of the current dispensing process, causing the product to deviate, thus providing closed-loop data support for subsequent process improvement and flexible scheduling.

[0213] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multimodal and deadlock-preventing heterogeneous graph-based edge-cloud collaborative scheduling method, characterized in that, The method includes: Based on the multimodal raw data processing results, a feature tensor is generated, and the environmental context representation vector and environmental adaptive speed adjustment coefficient corresponding to the feature tensor are calculated using orthogonal projection and second-order distance calculation techniques for transmission cost. Construct a heterogeneous scheduling graph, calculate the expected value of candidate movement actions based on the heterogeneous scheduling graph, and combine it with the environmental context representation vector and the environmental adaptive speed adjustment coefficient to determine the physical drive speed command of the automated guided vehicle. After the physical drive speed command is issued and executed, edge event matrix activation matching is performed, and after edge event matrix activation matching, spatiotemporal context slice window is captured to generate spatiotemporal tracing sequence; An energy manifold model is constructed to define the total scalar energy. When a product quality defect alarm signal is received, the real-time energy scalar corresponding to the timestamp of the spatiotemporal tracing sequence is calculated, compared with the total scalar energy, and the product defect tracing result is output.

2. The edge-cloud collaborative scheduling method based on multimodal and deadlock-preventing heterogeneous graphs according to claim 1, characterized in that, The process of generating a feature tensor based on the multimodal raw data processing results, and calculating the corresponding environmental context representation vector and environmental adaptive speed adjustment coefficient using orthogonal projection and second-order distance calculation techniques based on transmission cost, includes: Vibration, particulate matter, and visual image data collected by the automated guided vehicle and fixed sensing devices are acquired to obtain multimodal raw data. The multimodal raw data is then time-stamp aligned, and multimodal data is obtained based on the processing results. After orthogonally projecting the multimodal data, a feature tensor is generated. The feature tensor is then differentiated and combined with the channel dimension to construct a structural response matrix to determine the environmental context representation vector. Based on the multivariate Gaussian distribution parameters of the feature tensor output by the pre-encoding layer in the latent space, the mean vector and diagonal covariance matrix are determined, and the feature distribution of the feature tensor is output according to the mean vector and diagonal covariance matrix. The optimal transmission theory is used to calculate the transmission cost required to transform the feature distribution of any feature tensor into another distribution form, the second-order distance is determined, and the feature tensor with the largest second-order distance is selected as the global modal conflict metric factor. The global modal conflict metric factor is introduced into the servo control logic to calculate the environmental adaptive speed adjustment coefficient.

3. The edge-cloud collaborative scheduling method based on multimodal and deadlock-preventing heterogeneous graphs according to claim 2, characterized in that, The process of generating a feature tensor after orthogonal projection of multimodal data, performing differentiation on the feature tensor, and combining it with the channel dimension to construct a structural response matrix to determine the environmental context representation vector includes: Based on the pre-coding layer, high-dimensional feature vectors contained in multimodal data are extracted, and the high-dimensional feature vectors are input into the orthogonal projection layer. After stripping the redundant shared information between each modality, they are mapped to a shared latent space with a fixed dimension to generate a feature tensor of a unified dimension. The feature tensor includes visual features, physical perception features and anomaly type encoding. The second-order differential operation of the feature tensor is performed using the Laplacian operator to obtain the gradient feature map that characterizes the degree of local high-frequency abrupt changes, and the structural response matrix representing spatial topological changes is generated by combining the channel dimension. Based on the normalization of the structural response matrix into a structural pseudo-probability matrix that satisfies the probability distribution characteristics, the relative information weights contained at spatial coordinates are obtained, and the spatial local structural entropy of the feature tensor is calculated through the structural pseudo-probability matrix. The spatial local structural entropy is used as an exponential decay term to generate dynamic confidence gating coefficients. The environmental context representation vector is then determined by weighted fusion of the feature tensor and the dynamic confidence gating coefficients.

4. The edge-cloud collaborative scheduling method based on multimodal and deadlock-preventing heterogeneous graphs according to claim 1, characterized in that, The construction of the heterogeneous scheduling graph, the calculation of the expected value of candidate movement actions based on the heterogeneous scheduling graph, and the determination of the physical drive speed command of the automated guided vehicle by combining the environmental context representation vector and the environmental adaptive speed adjustment coefficient include: Based on the workshop status and the coordinates of the automated guided vehicles (AGVs), a heterogeneous scheduling diagram covering AGVs, process equipment, and task nodes is constructed using physical fingerprint anchoring and forward projection technology. The dynamic gravity coefficient and exponential repulsion force are determined based on the autonomous vehicle node and the task node, and the deadlock penalty factor is generated by constructing a label multiset through the joint feature sequence of the local topology in the local heterogeneous scheduling graph. The dynamic gravity coefficient, exponential repulsion force, and deadlock penalty factor are introduced into the attention aggregation mechanism to output the final representation of the spatial structure after deep aggregation, and determine the spatial topological constraint results to eliminate the deadlock situation of the automated guided vehicle cluster. The evolutionary features of the final representation of the spatial structure are extracted by using the reset gate, and the spatiotemporal hidden state representation is generated. The spatiotemporal hidden state representation, environmental context representation vector and action embedding vector are concatenated and input into the value network to decode and calculate the expected value of each movement candidate action of the automated guided vehicle. The optimal action is selected from among all candidate actions, the one whose expected value is greater than the expected value of the current action of the automated guided vehicle. The optimal action is then generated by combining the environmental adaptive speed adjustment coefficient and the physical drive speed command is sent to the automated guided vehicle master controller.

5. The edge-cloud collaborative scheduling method based on multimodal and deadlock-preventing heterogeneous graphs according to claim 4, characterized in that, The heterogeneous scheduling diagram, constructed based on workshop status and automated guided vehicle (AGV) coordinates, and utilizing physical fingerprint anchoring and forward projection technology to encompass AGVs, process equipment, and task nodes, includes: The automated guided vehicle nodes are determined based on global coordinates and load rate, the process equipment nodes are determined based on equipment coordinates and working status, and the task nodes are determined based on task priority and time tolerance. The heterogeneous nodes are obtained by combining the automated guided vehicle nodes, process equipment nodes and task nodes. Then, a nonlinear hash function is used to concatenate the heterogeneous nodes with timestamps and node physical medium access control addresses at the feature level to generate tamper-resistant spatiotemporal physical fingerprints. Heterogeneous nodes are projected onto a feature space of the same dimension using a linear transformation matrix to obtain an isomorphic feature tensor. Based on the isomorphic feature tensor, a forward transformation matrix is ​​used to generate a reconstructed original feature set. The L2-norm projection reconstruction residual between the heterogeneous nodes and the reconstructed original feature set is calculated to verify the node retention during the mapping process. A dynamic projection fault tolerance benchmark threshold is constructed based on the condition number of the forward transformation matrix and the electromagnetic environment noise intensity, and the logic judgment is performed based on the dynamic projection fault tolerance benchmark threshold, node retention degree and spatiotemporal physical fingerprint. Based on the logical judgment results, polluted nodes in the heterogeneous nodes are removed, and the Automated Guided Vehicle (AGV) is used as the center to perform topology connection processing based on the remaining heterogeneous nodes. A local heterogeneous scheduling graph is generated based on the topology connection results.

6. The edge-cloud collaborative scheduling method based on multimodal and deadlock-preventing heterogeneous graphs according to claim 5, characterized in that, The process of determining the dynamic gravity coefficient and exponential repulsion force based on the automated guided vehicle nodes and task nodes, and generating a deadlock penalty factor by constructing a label multiset through the joint feature sequence of the local topology in the local heterogeneous scheduling graph includes: Extract the semantic attributes of all heterogeneous nodes in the local heterogeneous scheduling graph, assign initial labels to each heterogeneous node, and perform aggregation iteration processing based on the initial labels to obtain the first-order neighbor node set of each heterogeneous node; Extract the label set of the first-order neighbor node set from the previous aggregation iteration, sort the label set in lexicographical order to obtain the neighbor label set, and concatenate the current label of each heterogeneous node with the neighbor label set to generate a joint feature sequence representing the local topology of each heterogeneous node. By using a one-way hash function to map the joint feature sequence to fixed-length discrete labels, the fixed-length discrete labels of all heterogeneous nodes in the local heterogeneous scheduling graph are collected to obtain a label multiset; The frequency of occurrence of various labels in the label multiset is counted, and the occurrence frequency is mapped to a fixed-length global topological feature vector as the topological fingerprint of the local heterogeneous scheduling graph. The cosine similarity between the topological fingerprint and the pre-set high-risk motif fingerprint is calculated to generate a deadlock penalty factor.

7. The edge-cloud collaborative scheduling method based on multimodal and deadlock-preventing heterogeneous graphs according to claim 6, characterized in that, The step of performing edge event matrix activation matching processing after the physical drive speed command is issued and executed, and generating a spatiotemporal tracing sequence by capturing a spatiotemporal context slice window after the edge event matrix activation matching includes: After the physical drive speed command is issued and executed, the global modal conflict measurement factor is extracted. When the global modal conflict measurement factor exceeds the safety tolerance threshold, or the dynamic confidence gating coefficient of any feature tensor decays, a perception confidence change event is triggered, indicating that the physical state of the environment has changed abruptly. The cosine similarity between the topological fingerprint and the pre-set high-risk phantom fingerprint is extracted in real time. When the cosine similarity exceeds the similarity threshold and triggers an elastic repulsion field, it indicates that the local spatial topology has entered a congested state. Real-time monitoring of the environmental adaptive speed adjustment coefficient; when the environmental adaptive speed adjustment coefficient triggers the underlying master controller to execute deceleration or emergency braking commands, it indicates that an extreme physical chassis intervention event has occurred. When any of the following events occurs: a sudden change in the physical state of the environment, a congestion of the local spatial topology, or an extreme intervention event in the physical chassis, the circular buffer is used to backtrack to the historical state and extend to the future state, capturing a spatiotemporal context slice window that includes the preceding causes and the subsequent results. The physical trajectory coordinates of the automated guided vehicle and the environmental context representation vector in the spatiotemporal context slice window are structured and packaged to generate a spatiotemporal tracing sequence. Trigger event type labels are added to the spatiotemporal tracing sequence and stored in the cloud spatiotemporal graph database.

8. The edge-cloud collaborative scheduling method based on multimodal and deadlock-preventing heterogeneous graphs according to claim 7, characterized in that, The constructed energy manifold model defines the total scalar energy, and upon receiving a product quality defect alarm signal, calculates the real-time energy scalar corresponding to the timestamp of the spatiotemporal tracing sequence, compares it with the total scalar energy, and outputs the product defect tracing results, including: An energy manifold model is constructed that includes an encoder network, a decoder network, and a latent space energy assessment network. The defect-free generated normal feature tensor is projected onto a low-dimensional Riemannian manifold through the encoder network. Based on the decoder network, the low-dimensional Riemannian manifold is reconstructed into the original high-dimensional space to generate reconstructed features. The Euclidean distance between the normal feature tensor and the reconstructed features is calculated as the orthogonal projection error of the normal feature tensor on the surface of the normal Riemannian manifold. The total scalar energy is defined based on the weighted result of orthogonal projection error and manifold surface latent energy, wherein the formula for calculating the total scalar energy is as follows: ； In the formula, Represents total scalar energy. Represents the balance coefficient. Represents the environmental context representation vector. Indicates the decoder network, Indicates the encoder network. Represents a latent space energy assessment network; The system receives product quality defect alarm signals, retrieves the physical trajectory of the product during its flow in the workshop from the spatiotemporal tracing sequence, calculates the real-time energy scalar, compares it with the total scalar energy, determines the disaster contribution, and outputs the product defect tracing results.

9. The edge-cloud collaborative scheduling method based on multimodal and deadlock-preventing heterogeneous graphs according to claim 8, characterized in that, The process of receiving a product quality defect alarm signal, retrieving the physical trajectory of the product during its flow in the workshop from the spatiotemporal tracing sequence, calculating the real-time energy scalar, comparing it with the total scalar energy, determining the disaster contribution, and outputting the product defect tracing result includes: Based on the alarm batch identifier, the physical trajectory of the corresponding batch during the workshop flow is retrieved in reverse from the spatiotemporal tracing sequence of the cloud spatiotemporal graph database, and the real-time energy scalar corresponding to each timestamp in the spatiotemporal tracing sequence is calculated. When the real-time energy scalar is greater than the total scalar energy, the location of the anomaly is locked, and the partial derivative of the anomaly location with respect to the feature tensor is determined. The partial derivative is then multiplied by the sensitivity weight to obtain the process-aware disaster contribution. The feature tensor corresponding to the maximum contribution of process perception to disaster is used as the disaster-causing factor. A defect report containing accurate coordinates, time period, process status and weighted causes is generated, and the defect source tracing results are output.

10. An edge-cloud collaborative scheduling system based on multimodal and deadlock-preventing heterogeneous graphs, used to implement the edge-cloud collaborative scheduling method based on multimodal and deadlock-preventing heterogeneous graphs as described in any one of claims 1-9, characterized in that, The system includes: The modal adaptive fusion module is used to generate feature tensors based on the processing results of multimodal raw data, and to calculate the environmental context representation vector and environmental adaptive speed adjustment coefficient corresponding to the feature tensors using orthogonal projection and second-order distance calculation techniques based on transmission cost. The anti-deadlock heterogeneous modeling module is used to construct a heterogeneous scheduling graph, calculate the expected value of the candidate movement based on the heterogeneous scheduling graph, and combine it with the environmental context representation vector and the environmental adaptive speed adjustment coefficient to determine the physical drive speed command of the automated guided vehicle. The event-driven interception module is used to perform edge event matrix activation matching processing after the physical drive speed command is issued and executed, and to capture the spatiotemporal context slice window to generate a spatiotemporal tracing sequence after the edge event matrix activation matching. The defect tracing module is used to construct an energy manifold model to define the total scalar energy, and when a product quality defect alarm signal is received, it calculates the real-time energy scalar corresponding to the timestamp of the spatiotemporal tracing sequence, compares it with the total scalar energy, and outputs the product defect tracing result.

Images