Man-machine cooperation intelligent control system of injection molding machine

Abstract

The invention provides an injection molding machine man-machine cooperation intelligent control system comprising an acquisition device used for acquiring state data of an operator; the signal preprocessing module is used for eliminating motion artifacts in the state data by adopting an improved multivariate empirical mode decomposition algorithm; the cognitive state decoding module is used for inputting a concentration time sequence of state data and outputting cognitive state parameters of an operator based on a space-time diagram convolutional network model; the digital twin synchronization engine is used for realizing millisecond-level alignment of injection molding machine data and state data through an OPC UA protocol; and the collaborative optimization engine is used for dynamically adjusting the injection molding process of the injection molding machine and the complexity of a human-computer interface based on a depth deterministic strategy gradient algorithm. By integrating a functional operator state monitoring technology, a digital twinborn model and a dynamic optimization algorithm, real-time perception of an operator cognitive state and intelligent regulation and control of injection molding process parameters are realized.

Description

Technical Field

[0001] The present application relates to the field of intelligent manufacturing technology, and in particular to a human-machine collaborative intelligent control system for an injection molding machine. Background Art

[0002] Traditional digital twin prototypes of injection molding machines have significant limitations. They rely solely on physical sensors such as temperature, pressure, and displacement to build digital models of the equipment, lacking the ability to monitor the operator's cognitive status. Moreover, in terms of process optimization, they are based solely on equipment data and do not take human factors into consideration. This leads to a high rate of human error. According to statistics, about 23% of scrap is caused by operational errors.

[0003] In recent years, with the development of near-infrared technology and the upgrading of data acquisition equipment, functional near-infrared spectroscopy (fNIRS) has emerged as an emerging non-invasive method for monitoring brain function, characterized by high compliance, strong anti-interference capabilities, portability, ease of implementation, and low cost. Although fNIRS technology has been used in the medical field for brain function research, there is a gap in its industrial application. fNIRS-based emotion analysis can monitor operator status in real time, reducing error rates; optimize human factors engineering and improve production efficiency; non-invasive measurement is easily accepted and has low interference; the equipment is portable and flexible, suitable for a variety of scenarios; it provides data support for decision-making, helps build a bidirectional human-computer coupling closed loop, and promotes efficient human-computer collaboration. Therefore, emotion recognition systems based on functional near-infrared spectroscopy technology are widely needed and have broad prospects. Summary of the Invention

[0004] In view of this, this application proposes an injection molding machine human-machine collaborative intelligent control system, which realizes real-time perception of the operator's cognitive state and intelligent regulation of injection molding process parameters by integrating functional operator status monitoring technology, digital twin model and dynamic optimization algorithm.

[0005] To achieve the above objectives, the present application provides an injection molding machine human-machine collaborative intelligent control system, comprising:

[0006] A collection device for collecting operator status data;

[0007] The signal preprocessing module uses an improved multivariate empirical mode decomposition algorithm, combined with wavelet threshold denoising and adaptive filtering to eliminate motion artifacts in the state data;

[0008] The cognitive state decoding module is based on a spatiotemporal graph convolutional network model. Its input is the concentration time series of the cerebral blood oxygen signal and the brain functional connectivity calculated based on the phase-locked value. The module outputs the operator's cognitive state parameters, which include attention level, fatigue level, and decision confidence.

[0009] A digital twin synchronization engine, which includes a high-fidelity injection molding machine model and an operator avatar, collects the injection molding machine's operating parameters in real time via the OPC UA protocol and synchronizes them to the high-fidelity injection molding machine model, achieving millisecond-level alignment of injection molding machine data and status data.

[0010] The collaborative optimization engine, based on a deep deterministic policy gradient algorithm, dynamically adjusts the injection molding process and human-machine interface complexity of the injection molding machine according to the operating parameters of the injection molding machine and the cognitive state parameters of the operator.

[0011] From the above, in the human-machine collaborative intelligent control system of the injection molding machine provided by this application, the operator's status data is effectively collected through the acquisition device. The signal preprocessing module adopts the improved multivariate empirical mode decomposition algorithm (MEMD) combined with wavelet threshold denoising and adaptive filtering technology to significantly eliminate motion artifacts and other noise interference in the status data and improve the signal-to-noise ratio. The cognitive state decoding module is based on the spatiotemporal graph convolutional network (ST-GCN) model. By analyzing the concentration time series of the status data, it accurately decodes the operator's attention level, fatigue level, decision confidence and other key cognitive state parameters, providing a basis for personalized human-machine interaction adjustment. The digital twin synchronization engine is based on OPC The UA protocol achieves millisecond-level precise alignment of the injection molding machine's operating parameters and the operator's status data, constructs a high-fidelity injection molding machine model and an operator's virtual avatar, and not only maps various indicators of the injection molding process in real time, but also supports multiple interaction methods such as AR / VR, enhancing the system's visualization and interactivity. The collaborative optimization engine applies the deep deterministic policy gradient (DDPG) algorithm to dynamically adjust the injection molding process parameters (such as injection speed, holding time, etc.) and the complexity of the human-machine interface according to the actual operation of the injection molding machine and the operator's cognitive state parameters, aiming to reduce human errors, improve production efficiency, and protect the operator's physical and mental health. This application realizes closed-loop collaborative control of the operator's cognitive state and the injection molding machine's operating parameters by integrating multimodal perception, real-time signal processing, cognitive state decoding, digital twin synchronization, and dynamic optimization, providing a feasible technical path for human-machine collaboration in intelligent manufacturing.

[0012] Optionally, the improved multivariate empirical mode decomposition algorithm includes:

[0013]

[0014] Among them, x(t) is the multivariate state data, IMF k (t) is the kth intrinsic mode function, and r(t) is the residual term.

[0015] Based on the above, by adopting the improved multivariate empirical mode decomposition (MEMD) algorithm, adaptive time-frequency analysis is performed on the multivariate state data, and the original state data is decomposed into multiple intrinsic mode functions (IMFs) and a residual term. Each IMF component corresponds to a different time scale oscillation mode, and the residual term represents the overall trend of the signal. Through this MEMD algorithm, the modal aliasing problem is avoided, and the IMF component of the motion artifact is accurately separated from the physiological signal. At the same time, the residual term is used to capture the low-frequency vibration interference unique to the injection molding machine environment, so that this component can be eliminated through an adaptive threshold, reducing the detection error of the reconstructed signal.

[0016] Optionally, the threshold selection formula for the wavelet threshold denoising includes:

[0017]

[0018] Where λ is the wavelet threshold, σ is the noise standard deviation, and N is the signal length;

[0019] According to the wavelet threshold, each intrinsic mode function is subjected to multi-scale wavelet decomposition and denoising processing.

[0020] From the above, by introducing the wavelet threshold formula based on statistical theory, adaptive suppression of residual noise in state data is achieved. This formula can automatically adjust the threshold size according to the signal length and noise level to avoid over-smoothing or insufficient denoising. By performing wavelet denoising on each intrinsic mode function separately, the robustness of the MEMD algorithm in non-stationary, multi-channel state data processing can be significantly improved.

[0021] Optionally, the adaptive filtering adopts a dynamic overlapping block strategy to divide the input state data into multiple sample blocks according to the signal complexity, with an overlap rate of 50% for adjacent sample blocks, and implements seamless filtering through a double buffering mechanism. The filtering output formula is:

[0022]

[0023] Among them, y(n) is the filtered state data at the nth moment, w k is the filter weight, x(nk) is the state data at the nkth moment, M is the filter order, and IMU noise (n) is the IMU noise reference, and λ(n) is the IMU noise coupling coefficient.

[0024] From the above, the adaptive filtering scheme of the present application combines dynamic overlapping blocking and double buffering mechanism, which significantly improves the ability to suppress residual noise and motion artifacts in state data while ensuring real-time performance. Among them, the dynamic overlapping blocking strategy flexibly divides the data block size according to the signal complexity (typical range 128 to 512 samples), and the 50% overlap rate effectively reduces the boundary effect. The double buffering mechanism ensures seamless connection of the filtering process. It also constructs an adaptive filtering structure with auxiliary noise reference by fusing the head motion information collected by the IMU sensor, which greatly enhances the targeted denoising.

[0025] Optionally, the high-fidelity injection molding machine model is built based on ANSYS Twin Builder, mapping multiple operating parameters of the injection molding machine in real time and supporting thermal-fluid-solid coupling simulation;

[0026] Cognitive state parameters are embedded in the operator's virtual avatar, and a visual brain load heat map is generated through Unity 3D, supporting AR / VR multimodal interaction.

[0027] As mentioned above, by integrating advanced modeling and simulation tools (such as ANSYS Twin Builder) and virtual reality platforms (such as Unity 3D), efficient management and visualization of the injection molding machine's operating status and the operator's cognitive status are achieved.

[0028] Optionally, the structure of the spatiotemporal graph convolutional network model includes:

[0029] Construction of the spatiotemporal graph: The nodes in the network are set based on the brain region locations defined by the international 10-5 system, and the edge weights between nodes are calculated using the phase lock value. The calculation formula is:

[0030]

[0031] Where N represents the number of samples, Δφ(t) represents the phase difference between the signals of two brain regions at time point t;

[0032] Spatiotemporal convolution layer: Fusion of graph convolution and temporal convolution, the feature update formula is as follows:

[0033]

[0034] in, is the adjacency matrix, is the degree matrix, H (l) Represents the feature matrix of the lth layer, W G and W T are the weight parameters of graph convolution and temporal convolution, σ represents the activation function, and Conv1D represents a stack of convolution operations, which are used to extract feature changes in the time dimension.

[0035] From the above, it can be seen that the spatiotemporal graph convolutional network (ST-GCN) model significantly improves the cognitive state decoding accuracy and industrial applicability of state data through PLV-guided dynamic graph construction and spatiotemporal convolution fusion. It is the core algorithm module for realizing "brain-computer" intelligent interaction in the human-machine collaborative control system of injection molding machines. Its millisecond-level inference delay and high robustness can directly support real-time decision-making and process optimization in the production environment.

[0036] Optionally, the implementation of the collaborative optimization engine includes:

[0037] Based on the deep deterministic policy gradient algorithm, a state-action-reward closed-loop optimization framework is constructed. The state space integrates the operating parameters of the injection molding machine and the cognitive state parameters of the operator. The action space includes injection molding process adjustment and human-machine interface optimization. The reward function is:

[0038] R = α·Production efficiency + β·Operator comfort - γ·Error rate;

[0039] Among them, production efficiency represents the number of qualified products per unit time, operator comfort is generated based on fatigue level, error rate is the ratio of the number of operating errors to the total number of operations, and α, β, and γ represent weight coefficients respectively.

[0040] Based on the above, a closed-loop system of state perception, dynamic decision-making and reward feedback is established through the deep deterministic policy gradient (DDPG) algorithm, which enables the control strategy to be adaptively adjusted according to real-time working conditions and operator status. It also supports the optimization of α, β, and γ by combining offline training with online fine-tuning. Through reinforcement learning, the policy network is continuously optimized, so that the overall performance of the system can be gradually improved in long-term operation, the human error rate can be reduced, and production stability can be improved.

[0041] Optionally, the dynamic adjustment strategy of the collaborative optimization engine includes at least one of the following:

[0042] When the operator's attention level remains below the set threshold for a period of time, the injection molding machine deceleration mode is triggered, the injection speed is reduced by 5%-15%, and the holding time is extended by 0.3 seconds;

[0043] When the operator's decision confidence falls below the set threshold, AR-assisted decision-making is triggered, with a 3D animation of expert operation guidance superimposed;

[0044] Dynamically adjust the number of parameters displayed and alarm thresholds on the human-machine interface based on the operator's fatigue level.

[0045] As mentioned above, by monitoring the operator's attention level in real time and automatically adjusting the injection molding process parameters when a decrease in attention is detected, the risk of production errors caused by distraction is effectively reduced. The feedback mechanism based on the operator's decision confidence can provide intuitive and easy-to-understand professional guidance to the operator at critical moments, enhancing operational accuracy, which is particularly important in complex or high-risk operating scenarios. Dynamic adjustment of the human-machine interface complexity and alarm sensitivity, personalized according to the operator's actual fatigue state, ensures high work efficiency and low error rate even after long working hours. Through this dynamic adjustment strategy, true human-machine collaboration is achieved, improving the automation and intelligence level of injection molding production.

[0046] Optionally, the acquisition device includes a wearable fNIRS device, which uses a 16-channel flexible probe array to cover the operator's prefrontal cortex to acquire the operator's brain blood oxygen signal;

[0047] The wearable fNIRS device also integrates a nine-axis inertial sensor to track the operator's head movement in real time and compensate for motion artifacts.

[0048] From the above, the acquisition device can specifically be a wearable fNIRS device. The 16-channel flexible probe array of the wearable fNIRS device covers the operator's prefrontal cortex, effectively acquiring brain blood oxygen signals reflecting brain activity. The brain blood oxygen signals include oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) signals. In addition, the integrated nine-axis inertial sensor can track the operator's head movement in real time, perform precise motion artifact compensation, and ensure data accuracy.

[0049] Optionally, it also includes multimodal sensors and federated learning modules;

[0050] The multimodal sensor includes at least one of an eye tracker, a heart rate belt, and a skin conductance sensor for comprehensively monitoring the operator's physiological state;

[0051] The federated learning module is used to aggregate data from multiple devices and update the global optimization strategy.

[0052] As mentioned above, the integration of multimodal sensors and federated learning modules has significantly improved the intelligence level and practicality of the injection molding machine's human-machine collaborative intelligent control system. It not only enables the system to more accurately capture the operator's actual working status, but also realizes cross-device knowledge sharing and strategy optimization through federated learning, thereby greatly improving production efficiency, reducing the human error rate, and improving the operator's work experience.

[0053] These and other aspects of the present application will become more apparent from the following description of the embodiment(s). BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is an architectural diagram of an injection molding machine human-machine collaborative intelligent control system provided in an embodiment of the present application;

[0055] Figure 2 The integrated layout and anti-interference design diagram of the fNIRS probe in the industrial helmet provided in the embodiment of the present application;

[0056] Figure 3 A schematic diagram of the processing process of a cognitive state decoding module provided in an embodiment of the present application;

[0057] Figure 4 This is a flowchart of a deep deterministic policy gradient (DDPG) algorithm provided in an embodiment of the present application.

[0058] It should be understood that the sizes and shapes of the blocks in the above structural diagrams are for reference only and should not constitute an exclusive interpretation of the embodiments of this application. The relative positions and inclusion relationships between the blocks presented in the structural diagrams are only schematic representations of the structural relationships between the blocks, and do not limit the physical connection methods of the embodiments of this application. DETAILED DESCRIPTION

[0059] The technical solution provided by this application is further described below with reference to the accompanying drawings and examples. It should be understood that the system structure and business scenarios provided in the examples of this application are mainly for illustrating possible implementation methods of the technical solution of this application and should not be interpreted as the sole limitation of the technical solution of this application. It is known to those skilled in the art that with the evolution of the system structure and the emergence of new business scenarios, the technical solution provided by this application is also applicable to similar technical problems.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art in the art of this application. In the event of any inconsistency, the meaning described in this specification or the meaning derived from the contents recorded in this specification shall prevail. In addition, the terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit this application.

[0061] The embodiment of the present application proposes an injection molding machine human-machine collaborative intelligent control system, which realizes real-time perception of the operator's cognitive state and intelligent regulation of injection molding process parameters by integrating functional near-operator state monitoring technology, digital twin model and dynamic optimization algorithm. The system includes an acquisition device, a signal preprocessing module, a cognitive state decoding module, a digital twin synchronization engine and a collaborative optimization engine, wherein the acquisition device is used to collect the operator's state data; the signal preprocessing module adopts an improved multivariate empirical mode decomposition algorithm, combined with wavelet threshold denoising and adaptive filtering to eliminate motion artifacts in the state data; the cognitive state decoding module is based on a spatiotemporal graph convolutional network model, and the input is the concentration time series of the brain blood oxygen signal and the brain functional connectivity calculated based on the phase lock value, and the output is the operator's cognitive state parameters, which include attention level, fatigue level and decision confidence; the digital twin synchronization engine includes a high-fidelity injection molding machine model and an operator virtual avatar, which is transmitted through OPC The UA protocol collects the operating parameters of the injection molding machine in real time and synchronizes them to the high-fidelity injection molding machine model, achieving millisecond-level alignment of the injection molding machine data and status data; the collaborative optimization engine is based on a deep deterministic policy gradient algorithm, which dynamically adjusts the injection molding process and human-machine interface complexity of the injection molding machine according to the operating parameters of the injection molding machine and the cognitive state parameters of the operator.

[0062] In some embodiments, the acquisition device can be a wearable fNIRS device for collecting the operator's brain oxygenation signal, or a vital sign sensor for collecting vital sign data such as heart rate and skin conductivity. Alternatively, it can be a facial data acquisition device for collecting the operator's facial expressions. This acquisition device collects the operator's state data for input into the backend for preprocessing, cognitive state decoding, and human-computer collaborative optimization.

[0063] Refer to the following Figures 1-4 An exemplary description is given of an injection molding machine human-machine collaborative intelligent control system provided in this application.

[0064] Figure 1 The figure shows an injection molding machine human-machine collaborative intelligent control system provided by the embodiment of the present application, referring to Figure 1 The system includes acquisition devices and multimodal sensors arranged in the perception layer, industrial-grade Wi-Fi6 and OPC UA communication protocols arranged in the transmission layer, signal preprocessing modules and cognitive state decoding modules arranged in the data processing layer, and also includes a digital twin synchronization engine and a collaborative optimization engine.

[0065] The acquisition device may be a wearable fNIRS device that uses a multi-channel flexible probe array to cover the operator's prefrontal cortex to acquire the operator's brain blood oxygen signal. Figure 2As shown in the figure, the wearable fNIRS device embeds a 16-channel flexible probe into an industrial safety helmet. The flexible probe adopts a layered structure and a magnetic fixed module design. Its layered structure includes a flexible substrate made of PDMS material, a 735nm / 850nm dual-wavelength light source emitter, a photodetector and an electromagnetic shielding layer. It is driven by an adaptive light source to achieve dynamic light intensity adjustment (5-50mW / cm 2 The 16 flexible probes are spaced 20mm apart to cover key points in the operator's prefrontal cortex, enabling the collection of brain oxygenation signals, including oxyhemoglobin (HbO) and deoxyhemoglobin (HbR), in industrial scenarios. Furthermore, the wearable fNIRS device integrates a nine-axis inertial measurement unit (IMU) with a 100Hz sampling frequency. Data is transmitted via Bluetooth 5.0, enabling real-time tracking of the operator's head movement and precise motion artifact compensation to ensure data accuracy.

[0066] The multimodal sensors include but are not limited to an eye tracker, a heart rate belt, and a skin conductance sensor, and are used to comprehensively monitor the physiological state of the operator, and to supplement and improve the operator's cognitive state information provided by the fNIRS device.

[0067] The signal preprocessing module adopts an improved multivariate empirical mode decomposition (MEMD) algorithm, combined with wavelet threshold denoising and adaptive filtering technology, to significantly eliminate motion artifacts and other noise interference in fNIRS data and improve the signal-to-noise ratio.

[0068] The cognitive state decoding module is based on a spatiotemporal graph convolutional network (ST-GCN) model. The input is the concentration time series of cerebral blood oxygen signals (HbO and HbR) and brain functional connectivity calculated based on phase-locked values ​​(PLVs). The module outputs the operator's cognitive state parameters, which include attention level, fatigue level, and decision confidence.

[0069] The digital twin synchronization engine includes a high-fidelity injection molding machine model and an operator avatar. The high-fidelity injection molding machine model, built using ANSYS Twin Builder, maps multiple machine operating parameters (e.g., over 200 parameters such as screw position, melt temperature, and in-mold pressure) in real time, supporting thermal-fluid-solid coupled simulation with an accuracy error of <0.5%. The operator avatar is embedded with cognitive state parameters, and a visual brain load heat map is generated using Unity 3D. This supports AR / VR multimodal interaction, providing an immersive experience for the operator, allowing them to monitor their cognitive state in real time and make corresponding adjustments. The digital twin synchronization engine collects the injection molding machine's operating parameters in real time via the OPC UA protocol and synchronizes them to the high-fidelity injection molding machine model, achieving millisecond-level alignment of the injection molding machine data and fNIRS data.

[0070] The collaborative optimization engine is based on the deep deterministic policy gradient (DDPG) algorithm. It dynamically adjusts the injection molding process (such as injection speed, holding time, etc.) and the complexity of the human-machine interface according to the operating parameters of the injection molding machine and the operator's cognitive state parameters, aiming to reduce human errors, improve production efficiency and protect the physical and mental health of the operator.

[0071] In some embodiments, the signal preprocessing module performs adaptive time-frequency analysis on a multivariate signal (e.g., a 16-channel fNIRS signal) by using an improved multivariate empirical mode decomposition (MEMD) algorithm, decomposing the original signal (16-channel fNIRS signal and IMU data) into multiple intrinsic mode functions (IMFs) and a residual term. Specifically, the MEMD algorithm includes:

[0072]

[0073] Where x(t) is the multivariate fNIRS data, IMF k (t) is the kth intrinsic mode function, and r(t) is the residual term.

[0074] Based on the multiple intrinsic mode functions (IMFs) decomposed above and a residual term, a wavelet threshold formula based on statistical theory is introduced to perform multi-scale wavelet decomposition and denoising on each intrinsic mode function. This formula can automatically adjust the threshold value according to the signal length and noise level to avoid over-smoothing or insufficient denoising. By performing wavelet denoising on each intrinsic mode function separately, the robustness of the MEMD algorithm in processing non-stationary, multi-channel fNIRS data can be significantly improved. The threshold selection formula for this wavelet threshold denoising includes:

[0075]

[0076] Among them, λ is the wavelet threshold, σ is the noise standard deviation, and N is the signal length.

[0077] The adaptive filtering in this embodiment uses a dynamic overlapping block strategy, dividing the input fNIRS data into blocks of 128-512 samples based on signal complexity. This signal complexity can be determined in real time based on the wavelet entropy value. For example, when the entropy value exceeds the threshold, the signal is considered high-complexity and 512-sample blocks are used to improve frequency resolution; otherwise, 128-sample blocks are used to reduce latency. The overlap rate of adjacent sample blocks is 50% to reduce boundary effects and ensure seamless data flow. Seamless data filtering is also achieved through the use of a dual-buffer mechanism (Buffer A / B). For example, while Buffer A processes the current block, Buffer B simultaneously loads the next block of data. A dual-port RAM is used to implement a ping-pong operation, and switching latency can be less than 1μs.

[0078] The adaptive filtering is achieved by combining variable step size factor with multimodal noise suppression. The step size update formula is:

[0079]

[0080] Where, α = 0.1, β = 0.01, and the input signal energy ||x(n)|| 2 Calculated using 16-bit fixed-point arithmetic to avoid floating-point overhead, the denominator 1+β·||x(n)|| 2 It is implemented using a LUT (lookup table) to pre-store 256 normalized values ​​to reduce real-time division operations.

[0081] This step-size update formula can achieve a balance between convergence speed and steady-state error. When the signal energy is high (such as when the operator's head moves violently), μ(n) automatically decreases to suppress overshoot. When the energy is low (such as when the operator is at rest), μ(n) increases to accelerate convergence.

[0082] The filter output formula uses dual-channel input, including the main signal filtering part and the motion artifact compensation part. The filter output formula is:

[0083]

[0084] Where y(n) is the filtered fNIRS data (HbO / HbR concentration change) at time n, w k is the filter weight, x(nk) is the fNIRS data (including noise) at the nkth moment, M is the filter order, and IMU noise (n) is the IMU noise reference, and λ(n) is the IMU noise coupling coefficient.

[0085] According to the above step size update formula and filter output formula, the interaction relationship between the parameters is:

[0086] When the operator's head moves violently: ||x(n)|| 2 Increase → μ(n) decrease → weight w k The update amplitude is reduced to avoid filter instability due to motion artifacts, and λ(n) is increased to 0.8 to enhance IMU noise cancellation;

[0087] At rest: ||x(n)|| 2 Decrease → μ(n) increases to 0.1 → weight w k It converges quickly and reduces λ(n) to 0.2, reducing the interference of the IMU channel on physiological signals.

[0088] like Figure 3As shown, the cognitive state decoding module of the embodiment of the present application is based on the spatiotemporal graph convolutional network (ST-GCN) model. Through PLV-guided dynamic graph construction and spatiotemporal convolution fusion, the cognitive state decoding accuracy and industrial applicability of fNIRS signals are significantly improved. The input layer of the cognitive state decoding module is the concentration time series of cerebral blood oxygen signals (HbO and HbR) and brain functional connectivity calculated based on phase-locked value (PLV). The spatiotemporal graph convolution layer is used for spatial convolution and temporal convolution, and the attention mechanism module is used to calculate the feature importance weight. Then, the operator's cognitive state parameters (attention level, fatigue level and decision confidence) are output in parallel, and each task shares the underlying feature extraction module. The construction process of the ST-GCN model includes:

[0089] Construction of the spatiotemporal graph: The nodes in the network were set based on the brain region locations defined by the international 10-5 system, and the edge weights between nodes were calculated using the phase lock value (PLV). The edge weight calculation formula is:

[0090]

[0091] Where N represents the number of samples, Δφ(t) represents the phase difference between the signals of two brain regions at time point t, and the PLV value range is [0,1]. The larger the value, the stronger the phase synchronization.

[0092] Spatiotemporal convolution layer: Fusion of graph convolution and temporal convolution, the feature update formula is as follows:

[0093]

[0094] in, is the adjacency matrix, which represents the connection relationship between nodes. is the degree matrix, used for normalization processing, H (l) Represents the feature matrix of the lth layer, W G and W T They are the weight parameters of graph convolution and temporal convolution, σ represents the activation function, which is used to introduce nonlinear factors, and Conv1D represents a stack of convolution operations, which is used to extract feature changes in the time dimension.

[0095] In some embodiments, the ST-GCN model can be optimized using a cross-entropy loss function to improve classification accuracy, achieving a high accuracy of 92.3% in the fNIRS signal classification task.

[0096] In some embodiments, the collaborative optimization engine of the present application is based on the deep deterministic policy gradient (DDPG) algorithm to build a state-action-reward closed-loop optimization framework. By sensing the operator's cognitive state and equipment operating parameters in real time, the injection molding process and the human-machine interaction interface are dynamically adjusted to achieve human-machine collaborative optimization. In terms of state perception, the physical parameters of the injection molding machine, such as melt temperature, in-mold pressure, screw position, etc., are integrated with the operator's cognitive state, namely attention level (0-100%), fatigue level (1-5 levels), and decision confidence (0-1). Subsequently, the DDPG algorithm makes dynamic decisions based on the current state and generates optimization actions, covering process parameter adjustments, such as injection speed changes within the range of ±5%-15%, holding time changes by ±0.3 seconds, melt temperature set value changes by ±2°C, and human-machine interface adjustments, including reducing the number of displayed parameters from 20 to 5, relaxing the alarm threshold by 20%, and triggering AR-assisted decision-making. Finally, the reward function quantifies production efficiency, operator comfort and error rate to form a feedback loop, continuously optimize the strategy, and continuously improve the overall system performance.

[0097] In terms of state space implementation, equipment data uses the OPC UA protocol to collect the operating parameters of the injection molding machine in real time, such as melt temperature and in-mold pressure, and can be synchronized to the digital twin synchronization engine at a speed of milliseconds; while operator data is extracted through fNIRS and multimodal sensors (eye tracker, heart rate monitor) to extract brain blood oxygen signals (HbO / HbR) and physiological indicators, and then decoded by the ST-GCN model to obtain cognitive state parameters such as attention, fatigue level and decision confidence.

[0098] In terms of action space implementation, process adjustments output continuous action values ​​through the Actor network of the DDPG algorithm, such as injection speed adjustment. These values ​​can be directly mapped to equipment control instructions, such as servo motor speed adjustment; interface adjustments are based on fatigue levels, dynamically simplifying the interface, such as hiding non-critical parameters, or triggering AR guidance, such as superimposing expert operation animations.

[0099] The reward function is:

[0100] R = α·Production efficiency + β·Operator comfort - γ·Error rate;

[0101] Production efficiency represents the number of qualified products per unit time, as measured by the digital twin. Operator comfort is generated based on fatigue level (reverse mapping from 1 to 5, e.g., fatigue level 5 corresponds to comfort level 0.2). The error rate is the ratio of the number of operational errors to the total number of operations, as measured by system logs. α, β, and γ are weight coefficients. Dynamic strategy optimization is achieved through offline training and online fine-tuning. During the offline training phase, α, β, and γ are optimized using historical data, e.g., α = 0.6, β = 0.3, and γ = 0.1. During the online fine-tuning phase, the weights are dynamically adjusted based on the real-time production environment.

[0102] According to the above-mentioned DDPG algorithm, the dynamic adjustment strategy of the collaborative optimization engine of this embodiment may include:

[0103] When the operator's attention level remains below the set threshold for a period of time, the injection molding machine deceleration mode is triggered, the injection speed is reduced by 5%-15%, and the holding time is extended by 0.3 seconds;

[0104] When the operator's decision confidence falls below the set threshold, AR-assisted decision-making is triggered, with a 3D animation of expert operation guidance superimposed;

[0105] Dynamically adjust the number of parameters displayed and alarm thresholds on the human-machine interface based on the operator's fatigue level.

[0106] For example, after working continuously for 2 hours, the operator's fatigue level rises to level 4, and the attention level also drops to 60%. At this time, the DDPG algorithm generates corresponding optimization actions based on the current state (fatigue level 4). In terms of process parameters, the injection speed is reduced by 10% and the holding time is extended by 0.3 seconds to reduce the operator's operating pressure; in terms of the human-machine interface, the interface display parameters are reduced from 20 to 5, and the alarm threshold is relaxed by 20%, thereby reducing the operator's cognitive load.

[0107] After these optimization actions were completed, a reward function was used for evaluation. While production efficiency decreased by 5% in the short term, operator comfort improved significantly by 30%, and the error rate decreased by 8%. Based on this, the system automatically fine-tuned the weights in the reward function, for example increasing the β value representing operator comfort from 0.3 to 0.4, thereby balancing production efficiency and operational safety. Ultimately, the optimization measures achieved positive results, significantly reducing the operator error rate from 15% to 7%, and reducing fatigue scores by 25%.

[0108] Reference Figure 4 As shown, the human-machine collaborative intelligent control system of the embodiment of the present application adopts a five-layer architecture reconstruction, realizes multimodal data fusion and real-time strategy iteration through layered decoupling, and the entire closed-loop system forms a two-way optimization of cognitive-physical space through real-time feedback of production efficiency, comfort, and error rate. The specific architecture is as follows:

[0109] The underlying multimodal data input layer integrates multimodal sensors and uses Bluetooth 5.0 and OPC UA protocols to achieve millisecond-level time alignment and synchronization of multi-source sensor data, including fNIRS dual-wavelength light intensity data (735nm / 850nm), IMU nine-axis motion data, and eye tracker (120Hz) / heart rate belt / skin conductance data.

[0110] The preprocessing layer first eliminates motion artifacts through MEMD decomposition combined with IMU dynamic compensation. Then, dynamic block (256 samples / 50% overlap) variable step-size adaptive filtering is performed. Finally, the ST-GCN model fuses the HbO / HbR concentration series with PLV brain functional connectivity features to output cognitive parameters such as the operator's attention (0-100%), fatigue level (1-5), and decision confidence (0-1) in JSON format.

[0111] The digital twin layer builds a dual-mapping system, which synchronizes 200+ physical parameters of the injection molding machine (melt temperature, in-mold pressure, etc.) in real time based on the ANSYS Twin Builder thermal-fluid-solid coupling model (error <0.5%). The Unity 3D virtual avatar simultaneously renders the operator's brain load heat map and supports AR / VR interaction.

[0112] The collaborative optimization layer constructs a 20-dimensional state space with the injection molding machine operating parameters and cognitive state parameters. The action space includes process adjustment (injection speed ±5%-15%, holding time ±0.3s, etc.) and interface optimization (display parameters are reduced to 5 items, AR guidance triggering, etc.). The reward function adopts a dynamic weight R = 0.6E + 0.3C - 0.1E (β = 0.4 when fatigued at night), and the Q value maximization strategy update is achieved through the Actor-Critic network (τ = 0.01).

[0113] The output layer sends process adjustment instructions to the servo system through the OPC UA protocol and implements interface adaptation based on fatigue levels: Hololens 2 projects AR operation instructions, dynamically hides non-critical parameters, relaxes the alarm threshold by 20% when fatigue ≥ level 3, and supports gesture / voice calling of detailed panels.

[0114] In summary, in the human-machine collaborative intelligent control system for the injection molding machine provided in the embodiment of the present application, the multi-channel flexible probe array of the wearable fNIRS device covers the operator's prefrontal cortex, effectively collecting brain blood oxygen signals reflecting brain activity. The brain blood oxygen signals include oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) signals. In addition, the integrated nine-axis inertial sensor can track the operator's head movement in real time, perform precise motion artifact compensation, and ensure data accuracy. The signal preprocessing module adopts an improved multivariate empirical mode decomposition algorithm (MEMD) combined with wavelet threshold denoising and adaptive filtering technology to significantly eliminate motion artifacts and other noise interference in fNIRS data, thereby improving the signal-to-noise ratio. The cognitive state decoding module is based on the spatiotemporal graph convolutional network (ST-GCN) model. By analyzing the time series of brain blood oxygen signal concentration and brain functional connectivity, it accurately decodes the operator's attention level, fatigue level, decision confidence and other key cognitive state parameters, providing a basis for personalized human-machine interaction adjustment. The digital twin synchronization engine uses OPC The UA protocol achieves millisecond-level precise alignment of the injection molding machine's operating parameters with fNIRS data, constructs a high-fidelity injection molding machine model and an operator's virtual avatar, and not only maps various indicators of the injection molding process in real time, but also supports multiple interaction methods such as AR / VR, enhancing the visualization and interactivity of the system. The collaborative optimization engine applies the deep deterministic policy gradient (DDPG) algorithm to dynamically adjust the injection molding process parameters (such as injection speed, holding time, etc.) and the complexity of the human-machine interface according to the actual operation of the injection molding machine and the operator's cognitive state parameters, aiming to reduce human errors, improve production efficiency, and protect the operator's physical and mental health. This application realizes closed-loop collaborative control of the operator's cognitive state and the injection molding machine's operating parameters by integrating multimodal perception, real-time signal processing, cognitive state decoding, digital twin synchronization, and dynamic optimization, providing a feasible technical path for human-machine collaboration in intelligent manufacturing.

[0115] It should be noted that the embodiments described in this application are only a part of the embodiments of this application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the above detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the application for protection, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0116] The words "first, second, third, etc." or module A, module B, module C and other similar terms in the specification and claims are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that the specific order or sequence can be interchanged where permitted so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0117] In the above description, the numbers representing the steps involved do not necessarily mean that the steps must be executed. Intermediate steps may also be included or replaced by other steps. If permitted, the order of the previous and next steps may be interchanged or executed simultaneously.

[0118] The term "comprising" as used in the specification and claims should not be construed as limiting to what is listed thereafter; it does not exclude other elements or steps. Thus, it should be interpreted as specifying the presence of the features, integers, steps, or components mentioned, but not excluding the presence or addition of one or more other features, integers, steps, or components, or groups thereof. Thus, the expression "a device comprising means A and B" should not be limited to a device consisting solely of components A and B.

[0119] The term "one embodiment" or "an embodiment" mentioned in this specification means that the specific features, structures, or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present application. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing in various places in this specification do not necessarily refer to the same embodiment, but may refer to the same embodiment. In addition, in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other. The technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0120] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present application has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, all of which fall within the scope of protection of the present invention.

Claims

1. An injection molding machine human-machine collaborative intelligent control system, characterized in that: include: A collection device for collecting operator status data; The signal preprocessing module uses an improved multivariate empirical mode decomposition algorithm, combined with wavelet threshold denoising and adaptive filtering to eliminate motion artifacts in the state data; The cognitive state decoding module is based on a spatiotemporal graph convolutional network model. It takes as input the concentration time series of state data and outputs the operator's cognitive state parameters, which include attention level, fatigue level, and decision confidence. A digital twin synchronization engine, which includes a high-fidelity injection molding machine model and an operator avatar, collects the injection molding machine's operating parameters in real time via the OPC UA protocol and synchronizes them to the high-fidelity injection molding machine model, achieving millisecond-level alignment of injection molding machine data and status data. The collaborative optimization engine, based on a deep deterministic policy gradient algorithm, dynamically adjusts the injection molding process and human-machine interface complexity of the injection molding machine according to the operating parameters of the injection molding machine and the cognitive state parameters of the operator.

2. The system according to claim 1, wherein: The improved multivariate empirical mode decomposition algorithm includes: Among them, x(t) is the multivariate state data, IMF k (t) is the kth intrinsic mode function, and r(t) is the residual term.

3. The system according to claim 2, characterized in that The threshold selection formula for the wavelet threshold denoising includes: Where λ is the wavelet threshold, σ is the noise standard deviation, and N is the signal length; According to the wavelet threshold, each intrinsic mode function is subjected to multi-scale wavelet decomposition and denoising processing.

4. The system according to claim 3, characterized in that The adaptive filtering adopts a dynamic overlapping block strategy to divide the input state data into multiple sample blocks according to the signal complexity. The overlap rate of adjacent sample blocks is 50%, and seamless filtering is achieved through a double buffering mechanism. The filter output formula is: Among them, y(n) is the filtered state data at the nth moment, w k is the filter weight, x(nk) is the state data at the nkth moment, M is the filter order, and IMU noise (n) is the IMU noise reference, and λ(n) is the IMU noise coupling coefficient.

5. The system according to claim 1, wherein: The high-fidelity injection molding machine model is built based on ANSYS Twin Builder, which maps multiple operating parameters of the injection molding machine in real time and supports thermal-fluid-solid coupling simulation; Cognitive state parameters are embedded in the operator's virtual avatar, and a visual brain load heat map is generated through Unity 3D, supporting AR / VR multimodal interaction.

6. The system according to claim 1, wherein: The structure of the spatiotemporal graph convolutional network model includes: Construction of the spatiotemporal graph: The nodes in the network are set based on the brain region locations defined by the international 10-5 system, and the edge weights between nodes are calculated using the phase lock value. The calculation formula is: Where N represents the number of samples, Δφ(t) represents the phase difference between the signals of two brain regions at time point t; Spatiotemporal convolution layer: Fusion of graph convolution and temporal convolution, the feature update formula is as follows: in, is the adjacency matrix, is the degree matrix, H (l) Represents the feature matrix of the lth layer, W G and W T are the weight parameters of graph convolution and temporal convolution, σ represents the activation function, and Conv1D represents a stack of convolution operations, which are used to extract feature changes in the time dimension.

7. The system according to claim 1, wherein: The implementation of the collaborative optimization engine includes: Based on the deep deterministic policy gradient algorithm, a state-action-reward closed-loop optimization framework is constructed. The state space integrates the operating parameters of the injection molding machine and the cognitive state parameters of the operator. The action space includes injection molding process adjustment and human-machine interface optimization. The reward function is: R = α·Production efficiency + β·Operator comfort - γ·Error rate; Among them, production efficiency represents the number of qualified products per unit time, operator comfort is generated based on fatigue level, error rate is the ratio of the number of operating errors to the total number of operations, and α, β, and γ represent weight coefficients respectively.

8. The system according to claim 1, wherein: The dynamic adjustment strategy of the collaborative optimization engine includes at least one of the following: When the operator's attention level remains below the set threshold for a period of time, the injection molding machine deceleration mode is triggered, the injection speed is reduced by 5%-15%, and the holding time is extended by 0.3 seconds; When the operator's decision confidence falls below the set threshold, AR-assisted decision-making is triggered, with a 3D animation of expert operation guidance superimposed; Dynamically adjust the number of parameters displayed and alarm thresholds on the human-machine interface based on the operator's fatigue level.

9. The system according to claim 1, wherein: The acquisition device includes a wearable fNIRS device, which uses a 16-channel flexible probe array to cover the operator's prefrontal cortex to acquire the operator's brain blood oxygen signal; The wearable fNIRS device also integrates a nine-axis inertial sensor to track the operator's head movement in real time and compensate for motion artifacts.

10. The system according to claim 1, wherein: It also includes multimodal sensors and federated learning modules; The multimodal sensor includes at least one of an eye tracker, a heart rate belt, and a skin conductance sensor for comprehensively monitoring the operator's physiological state; The federated learning module is used to aggregate data from multiple devices and update the global optimization strategy.

Images