Industrial data acquisition, analysis and processing method and system based on deep reinforcement learning

Abstract

The application relates to the technical field of industrial intelligent collection, and discloses an industrial data collection, analysis and processing method and system based on deep reinforcement learning. The method comprises the following steps: based on the historical upstream collection sequence and the downstream quality detection sequence of a multi-process cascade production line, calculating the physical time delay between processes through a cross-correlation function, training a downstream quality predictor by aligning the data set in time sequence to obtain a virtual reward, calculating a confidence decay coefficient in a sliding window according to a prediction residual and weighting and fusing the virtual reward to drive collection frequency decision updating, calculating a cross-process coupling strength score based on a historical sequence of the confidence decay coefficient and performing collection frequency redistribution. The application solves the problems that in the prior art, there is a lack of a cross-process collection channel information contribution quantification mechanism, and collection resource structural redistribution cannot be performed according to actual channel contribution.

Description

Technical Field

[0001] This application relates to the field of industrial intelligent data acquisition technology, and in particular to an industrial data acquisition, analysis and processing method and system based on deep reinforcement learning. Background Technology

[0002] In the industrial manufacturing sector, intelligent data acquisition in multi-stage cascaded production lines is a core element for achieving process quality control. Existing solutions, such as deep reinforcement learning-based industrial data acquisition methods, typically model the dynamic adjustment of sensor acquisition frequency as a Markov decision process. This process uses observable process parameters for the current stage as the state space, acquisition frequency level switching as the action space, and readily available quality inspection feedback as the reward signal, driving a deep Q-network to optimize the acquisition strategy online. These methods already possess a certain degree of adaptability in single-stage scenarios, dynamically adjusting sensor acquisition density based on real-time changes in process parameters, thus achieving a balance between data acquisition accuracy and computational resource consumption to some extent.

[0003] However, existing deep reinforcement learning-based industrial data acquisition methods have fundamental flaws in multi-process cascade scenarios. Existing methods implicitly assume that the reward signal upon which the acquisition decision depends occurs within the same time step as the acquisition action, meaning there is no temporal misalignment between quality inspection feedback and upstream acquisition behavior. However, in multi-process cascaded production lines such as welding, painting, and final assembly, the product quality results corresponding to the sensor acquisition decisions of upstream processes must undergo processing in downstream processes for several hours or even tens of hours before being output by quality inspection equipment. This objectively existing physical time delay between processes means that the reward signal obtained by the deep reinforcement learning agent when executing acquisition decisions is actually a delayed feedback from several batches ago. Therefore, the agent continuously iterates on an incorrect state-reward correspondence, and the optimization direction of the acquisition strategy systematically deviates from the true process quality target. Summary of the Invention

[0004] This application provides an industrial data acquisition, analysis and processing method and system based on deep reinforcement learning, which solves the problems of physical time delay between deep reinforcement learning acquisition decisions and downstream quality feedback in multi-process cascaded production lines, resulting in timing misalignment of reward signals and inability to adaptively constrain prediction errors. It also solves the problems of existing solutions lacking a mechanism for quantifying the contribution of cross-process acquisition channel information and being unable to perform structural reallocation of acquisition resources based on the actual contribution of the channel.

[0005] Firstly, this application provides an industrial data acquisition, analysis, and processing method based on deep reinforcement learning, the method comprising:

[0006] Step S1: Based on the historical upstream acquisition sequence and downstream quality inspection sequence of the multi-process cascaded production line, calculate the time offset between the two through the cross-correlation function to obtain the physical time delay between processes;

[0007] Step S2: Remap the timestamps of the upstream acquisition sequence and the downstream quality detection sequence by the physical delay between processes, train the downstream quality predictor with the remapped time-aligned dataset, and use the downstream quality predictor to predict the real-time upstream acquisition feature vector to obtain the virtual reward.

[0008] Step S3: When the downstream real quality detection result arrives, calculate the root mean square prediction residual between the predicted value corresponding to the virtual reward and the downstream real quality detection result within the sliding window. Convert the root mean square prediction residual into a confidence decay coefficient according to a piecewise linear mapping rule. When the root mean square prediction residual does not exceed the first threshold, the confidence decay coefficient takes the maximum value. When the root mean square prediction residual exceeds the second threshold, the confidence decay coefficient takes the zero value. When the root mean square prediction residual is between the first threshold and the second threshold, the confidence decay coefficient is calculated by linear interpolation. The confidence decay coefficient and the virtual reward are weighted and fused together to serve as the reward signal for the deep reinforcement learning agent, thereby driving the decision update of the acquisition frequency of each sensor channel.

[0009] Step S4: Calculate the cross-process coupling strength score based on the historical sequence of the confidence decay coefficient, redistribute the acquisition frequency of each sensor channel according to the cross-process coupling strength score, and synchronize the updated acquisition frequency vector to the state space of the deep reinforcement learning agent.

[0010] Secondly, this application provides an industrial data acquisition, analysis, and processing system based on deep reinforcement learning, the industrial data acquisition, analysis, and processing system based on deep reinforcement learning comprising:

[0011] The data acquisition module is used to calculate the time offset between the historical upstream acquisition sequence and downstream quality inspection sequence of a multi-process cascaded production line by using a cross-correlation function, thereby obtaining the physical time delay between processes.

[0012] The mapping module is used to remap the physical delay between processes to the upstream acquisition sequence and the downstream quality detection sequence using timestamps. The downstream quality predictor is trained using the remapped time-aligned dataset. The downstream quality predictor predicts the real-time upstream acquisition feature vector to obtain a virtual reward.

[0013] The driving module is used to calculate the root mean square prediction residual between the predicted value corresponding to the virtual reward and the downstream real quality detection result within a sliding window when the downstream real quality detection result arrives. The root mean square prediction residual is converted into a confidence decay coefficient according to a piecewise linear mapping rule. When the root mean square prediction residual does not exceed a first threshold, the confidence decay coefficient takes the maximum value. When the root mean square prediction residual exceeds a second threshold, the confidence decay coefficient takes the zero value. When the root mean square prediction residual is between the first threshold and the second threshold, the confidence decay coefficient is calculated by linear interpolation. The confidence decay coefficient and the virtual reward are weighted and fused together to serve as the reward signal for the deep reinforcement learning agent, so as to drive the decision update of the acquisition frequency of each sensor channel.

[0014] The allocation module is used to calculate the cross-process coupling strength score based on the historical sequence of the confidence decay coefficient, perform acquisition frequency reallocation on each sensor channel according to the cross-process coupling strength score, and synchronize the updated acquisition frequency vector to the state space of the deep reinforcement learning agent.

[0015] Thirdly, a deep reinforcement learning-based industrial data acquisition, analysis, and processing device is provided, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the deep reinforcement learning-based industrial data acquisition, analysis, and processing device to execute the aforementioned deep reinforcement learning-based industrial data acquisition, analysis, and processing method.

[0016] Fourthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the aforementioned deep reinforcement learning-based industrial data acquisition, analysis and processing method.

[0017] The technical solution provided in this application calculates the temporal offset between the historical upstream acquisition sequence and the downstream quality detection sequence using a cross-correlation function. This explicitly introduces the objectively existing inter-process physical delay in multi-process cascade scenarios as a quantifiable system structure parameter into the deep reinforcement learning framework, fundamentally eliminating the state-reward misalignment problem caused by ignoring delay in existing technologies. Based on this, the construction of a temporally aligned dataset enables the training data of the downstream quality predictor to achieve a true physical correspondence between upstream and downstream data. The resulting virtual reward can then replace the not-yet-arrived real downstream quality detection result within the inter-process physical delay, allowing the deep reinforcement learning agent to obtain an instantaneous reward signal at each decision step and drive the online update of the acquisition frequency. This overcomes the inherent limitation of existing technologies where severe lag in reward signals in multi-process scenarios leads to slow policy convergence.

[0018] Furthermore, the continuous calculation of the root mean square prediction residual within the sliding window provides a quantitative evaluation basis for the real-time prediction quality of the downstream quality predictor. The piecewise linear mapping rule continuously transforms the root mean square prediction residual into a confidence decay coefficient, which smoothly transitions between the first and second thresholds using linear interpolation. This avoids the drastic impact of sudden changes in the confidence decay coefficient on the deep reinforcement learning agent's strategy, and automatically reduces the weight of virtual rewards and increases the weight of real feedback when the predictor's performance declines, thus achieving adaptive dynamic switching between the two types of reward sources. The historical sequence of confidence decay coefficient is transformed into a cross-process coupling strength score through exponential decay weighted operation. This score elevates the problem of acquisition frequency adjustment within a single process to the level of cross-process resource structural configuration. Based on the quantitative ranking of the Pearson correlation coefficient between each sensor channel and the downstream quality detection sequence, inefficient acquisition channels are down-processed, and process boundary node channels are up-processed. The updated acquisition frequency vector is synchronized to the state space of the deep reinforcement learning agent, so that the agent's subsequent strategy optimization is based on the acquisition configuration that matches the contribution of real process information. From the mechanism design level, the global rationality of data acquisition resource allocation in multi-process cascade scenarios is guaranteed. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of an embodiment of the industrial data acquisition, analysis and processing method based on deep reinforcement learning in this application.

[0021] Figure 2 This is a schematic diagram of the Pearson correlation coefficient distribution of the sensor channels in an embodiment of this application. Detailed Implementation

[0022] This application provides a method and system for industrial data acquisition, analysis, and processing based on deep reinforcement learning. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0023] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the industrial data acquisition, analysis, and processing method based on deep reinforcement learning in this application includes:

[0024] Step S1: Based on the historical upstream acquisition sequence and downstream quality inspection sequence of the multi-process cascaded production line, calculate the time offset between the two through the cross-correlation function to obtain the physical time delay between processes;

[0025] Specifically, a multi-process cascaded production line refers to an industrial production line consisting of multiple sequentially connected processes such as welding, painting, and final assembly. Material transfer relationships exist between these processes, with the output of the upstream process directly entering the downstream process for processing. The historical upstream acquisition sequence refers to the time series of process parameters recorded by sensors in the upstream process at a fixed acquisition cycle during a historical production period. The downstream quality inspection sequence refers to the binary result time series of whether a batch of products is qualified or not, recorded by the downstream process after performing quality inspection on each batch within the corresponding historical period. The cross-correlation function is a statistical method for measuring the similarity of two time series at different time offsets. By calculating the average product of the two sequences at different offsets within a preset time delay search range, the offset corresponding to the largest average product is taken as the time series offset between the two sequences. This offset is the physical time delay between processes, and its value reflects the objective time interval between the output of the upstream process and the quality inspection result of the downstream process.

[0026] Step S2: Remap the timestamps of the upstream acquisition sequence and the downstream quality inspection sequence by the physical delay between processes. Train the downstream quality predictor with the remapped time-aligned dataset. The downstream quality predictor predicts the real-time upstream acquisition feature vector to obtain the virtual reward.

[0027] Specifically, timestamp remapping refers to pairing the timestamp of each record in the upstream acquisition sequence with its corresponding downstream quality inspection result after the physical delay between processes, thereby eliminating the misalignment of upstream and downstream data caused by the delay. After remapping, each upstream acquisition record and its delayed downstream quality inspection result form a one-to-one correspondence, and this pairing set constitutes a time-aligned dataset. The downstream quality predictor uses the time-aligned dataset as training data to learn the mapping relationship between the upstream acquisition feature vector and the downstream quality pass probability. During the online operation of the production line, the real-time upstream acquisition feature vector is input into the trained downstream quality predictor, which outputs the downstream quality pass probability at the current moment. This probability is linearly transformed and mapped to the negative-to-positive interval to obtain a virtual reward. The virtual reward replaces the real downstream quality inspection result that has not yet arrived within the physical delay between processes, serving as the immediate reward signal for the deep reinforcement learning agent at the current moment.

[0028] Step S3: When the downstream real quality detection result arrives, calculate the root mean square prediction residual between the virtual reward prediction value and the downstream real quality detection result within the sliding window. Convert the root mean square prediction residual into a confidence decay coefficient according to the piecewise linear mapping rule. When the root mean square prediction residual does not exceed the first threshold, the confidence decay coefficient takes the maximum value. When the root mean square prediction residual exceeds the second threshold, the confidence decay coefficient takes the zero value. When the root mean square prediction residual is between the first threshold and the second threshold, the confidence decay coefficient is calculated by linear interpolation. The confidence decay coefficient and the virtual reward are weighted and fused together as the reward signal of the deep reinforcement learning agent to drive the decision update of the acquisition frequency of each sensor channel.

[0029] Specifically, the root mean square (RMS) prediction residual measures the deviation between the downstream quality predictor's prediction and the actual downstream quality detection results. It is calculated by taking the root mean square of the absolute differences of the most recent W predictions within a sliding window of length W. The value of W determines the time span of the residual evaluation; if W is too small, it becomes overly sensitive to short-term fluctuations, while if W is too large, it results in a delayed response to predictor performance degradation. The recommended range for W is 30 to 100 batches. The confidence decay coefficient is determined based on the RMS prediction residual according to a piecewise linear mapping rule. The first threshold is set as the acceptable upper bound of the prediction residual, and the second threshold is set as the failure lower bound of the prediction residual. The linear interpolation between the two thresholds ensures that the confidence decay coefficient changes continuously with the prediction residual rather than abruptly, avoiding drastic perturbations to the deep reinforcement learning agent's strategy caused by abrupt changes in the reward signal.

[0030] Step S4: Calculate the cross-process coupling strength score based on the historical sequence of confidence decay coefficient, redistribute the acquisition frequency of each sensor channel according to the cross-process coupling strength score, and synchronize the updated acquisition frequency vector to the state space of the deep reinforcement learning agent.

[0031] Specifically, the cross-process coupling strength score is a weighted average calculated by assigning exponential decay weights to the historical sequence of confidence decay coefficients. It reflects the overall level of the current process's ability to contribute information about downstream quality over a recent period, with the confidence decay coefficient of more recent batches having a greater impact on the score. When the cross-process coupling strength score is consistently below the preset coupling strength threshold, it indicates a structural deficiency in the information coupling channel between the current process and the downstream process. Channels are sorted based on the Pearson correlation coefficient between each sensor channel and the downstream quality detection sequence. Channels with low correlation coefficients are identified as inefficient acquisition channels, and their acquisition frequency levels are lowered to release acquisition resources. Channels at process boundaries are upgraded due to their high correlation with downstream quality. The updated acquisition frequency vector replaces the frequency coding component in the deep reinforcement learning agent's state vector with one-hot encoding. At the same time, the historical sample weights corresponding to the old acquisition frequency configuration in the experience replay buffer are reset to zero, ensuring that subsequent policy updates by the agent rely only on valid samples generated under the new acquisition configuration.

[0032] In one specific embodiment, step S1 includes:

[0033] Extract data records of N consecutive production batches from the historical production database, and align the upstream collection sequence and downstream quality inspection sequence based on the completion timestamp of each batch to obtain a batch-aligned record set;

[0034] Input the historical upstream acquisition sequence and downstream quality inspection sequence in the batch alignment record set into the cross-correlation function, calculate the cross-correlation number step by step within the time delay search range, take the time offset corresponding to the maximum cross-correlation number as the single batch time delay candidate value, and take the median of the single batch time delay candidate values ​​of all N batches to obtain the inter-process physical time delay.

[0035] Calculate the confidence interval of N single-batch delay candidate values ​​based on the inter-process physical delay. When the ratio of the confidence interval width to the inter-process physical delay exceeds the preset ratio threshold, expand N to 2N and re-execute the above median calculation until the ratio of the confidence interval width to the inter-process physical delay does not exceed the preset ratio threshold.

[0036] The physical delay between processes is used to remap the timestamps of the upstream acquisition sequence and the downstream quality inspection sequence. The remapped upstream acquisition sequence is then paired with the corresponding downstream quality inspection sequence according to the timestamp to obtain a time-aligned dataset.

[0037] Specifically, the process of forming the batch alignment record set involves aligning the upstream acquisition sequences and downstream quality inspection sequences of N consecutive batches from the historical production database, using the completion timestamp of each batch as the alignment benchmark. Upstream and downstream data for the same batch are grouped into the same record entry. An initial value for N is recommended to be no less than two thousand batches. This value is based on the fact that the statistical stability of the time series offset calculated by the cross-correlation function increases with the sample size. When the number of batches is insufficient, the median result has a lower tolerance for a small number of abnormal batches, leading to a greater deviation in the estimation of inter-process physical delays. The candidate value for single-batch delay is obtained by calculating the average cross-correlation coefficient sequence formed by the products of the upstream acquisition sequence and the downstream quality inspection sequence at each time offset within the delay search range. The value corresponding to the offset with the largest average product is taken from the cross-correlation coefficient sequence. The median, rather than the mean, is taken for all N batches of candidate single-batch delays because the median has a natural resistance to extreme outliers. Abnormal delays in a single production batch caused by equipment failure or human intervention will not deviate from the final inter-process physical delay.

[0038] The width of the confidence interval reflects the dispersion of N single-batch delay candidate values. When the ratio of the width to the inter-process physical delay exceeds a preset threshold, it indicates insufficient statistical stability of the estimation results under the current sample size. It is recommended that the preset threshold be one-tenth, meaning the confidence interval width is allowed to not exceed one-tenth of the inter-process physical delay. In this case, N is doubled, and the median calculation is re-executed until the ratio meets the condition. This iterative expansion mechanism ensures the statistical reliability of the inter-process physical delay. Timestamp remapping involves adding the inter-process physical delay to the timestamp of each record in the upstream acquisition sequence, then retrieving the matching timestamp from the downstream quality inspection sequence. These two are paired to form an aligned record. The set of all paired records is the time-series aligned dataset. In this dataset, the upstream acquisition feature vector of each record and the downstream quality inspection result physically correspond to the same batch of products, eliminating the data misalignment problem caused by inter-process delays.

[0039] In one specific embodiment, step S2 includes:

[0040] The time-aligned dataset is divided into training, validation, and test sets according to a preset ratio. The downstream quality predictor is iteratively trained based on the training and validation sets, with the binary cross-entropy loss function as the training objective. Training is terminated when the validation set loss does not decrease within a preset number of consecutive rounds. The F1 score of the trained downstream quality predictor is calculated on the test set. When the F1 score is lower than a preset passing threshold, the time-aligned dataset is expanded and retrained until the F1 score is not lower than the preset passing threshold, thus obtaining the downstream quality predictor.

[0041] The real-time upstream feature vector is input into the downstream quality predictor for forward inference to obtain the downstream quality pass probability at the current moment.

[0042] Based on the downstream quality pass probability, an interval transformation is performed according to the linear mapping rule to map the downstream quality pass probability from the zero-one interval to the negative-one-positive-one interval, and a virtual reward is obtained.

[0043] The virtual rewards are bound one by one to the timestamps of their corresponding real-time upstream collected feature vectors, and written into the virtual reward sequence in the order of the timestamps to obtain the virtual reward sequence.

[0044] Specifically, the time-aligned dataset is divided into training, validation, and test sets in an 8:1:1 ratio. This ratio ensures sufficient training data while reserving enough samples for both the validation and test sets to evaluate generalization ability. The downstream quality predictor takes upstream feature vectors as input and outputs binary labels indicating whether the downstream quality is good or bad. The binary cross-entropy loss function measures the difference between the predicted probability of good or bad quality and the true binary label; a smaller loss value indicates a more accurate learning of the mapping relationship between upstream features and downstream quality. Early stopping is triggered when the validation loss does not decrease within ten consecutive training epochs. The value of this epoch strikes a balance between preventing overfitting and allowing sufficient training time. The F1 score comprehensively considers the predictor's accuracy and recall in identifying good and bad products. A preset good or bad quality threshold of 0.85 is recommended; a threshold below this indicates insufficient ability of the predictor to discriminate downstream quality. In this case, the time-aligned dataset is expanded, and the predictor is retrained until the threshold is met.

[0045] The downstream quality pass probability is a scalar value within the zero-to-one interval output by the downstream quality predictor after performing forward inference on the current real-time upstream collected feature vector. This value reflects the predictor's confidence in the final output of a qualified product from the current upstream process state. The linear mapping rule for the downstream quality pass probability from the zero-to-one interval to the negative-to-positive-to-one interval is to multiply the pass probability by two and then subtract one. When the pass probability is one, the virtual reward is positive, indicating that the deep reinforcement learning agent has received the maximum positive incentive; when the pass probability is zero, the virtual reward is negative, indicating that the agent has received the maximum negative incentive; when the pass probability is 0.5, the virtual reward is exactly zero, indicating no clear incentive direction. This mapping transforms the prediction result into a standardized reward signal format within the deep reinforcement learning framework. The virtual reward is bound to the timestamp of the corresponding real-time upstream collected feature vector and written into the virtual reward sequence. The purpose of timestamp binding is to accurately retrieve the virtual reward prediction value corresponding to the actual downstream quality detection result when the physical delay between processes arrives, in order to calculate the prediction residual between the two.

[0046] In one specific embodiment, in step S3, when the downstream actual quality detection result arrives, the root mean square prediction residual between the predicted value corresponding to the virtual reward and the downstream actual quality detection result is calculated within a sliding window, including:

[0047] Maintain a sliding window residual queue of length W. Whenever a downstream real quality inspection result arrives, retrieve the predicted value of the virtual reward corresponding to the physical delay between the timestamp of the corresponding downstream real quality inspection result and the process from the virtual reward sequence. Calculate the absolute difference between the predicted value and the downstream real quality inspection result, write the absolute difference into the sliding window residual queue, and pop the earliest record to obtain the updated sliding window residual queue.

[0048] Based on all W absolute differences in the updated sliding window residual queue, the root mean square prediction residual is obtained by squaring each absolute difference, taking the mean, and then taking the square root.

[0049] Specifically, the sliding window residual queue is a fixed-length, first-in-first-out queue of length W. Each record in the queue stores an absolute difference, and the recommended value for W is between thirty and one hundred batches. When W is too small, the queue only reflects the prediction error of a small number of recent batches, becoming overly sensitive to short-term fluctuations, leading to frequent and drastic changes in the root mean square prediction residual. When W is too large, the queue reacts slowly to recent declines in the performance of downstream quality predictors, causing the adjustment of the confidence decay coefficient to lag behind changes in actual predicted quality. A recommended value of fifty batches strikes a balance between response speed and smoothing stability. The predicted value corresponding to the virtual reward is the downstream quality pass probability stored in the record whose timestamp differs from the timestamp of the currently arriving downstream actual quality inspection result by the inter-process physical delay. The timestamp retrieval matching rule is to search for records in the virtual reward sequence whose timestamp difference equals the inter-process physical delay. This matching rule ensures that the predicted values ​​participating in the residual calculation and the actual inspection results physically correspond to the same batch of products, rather than being mismatched between different batches.

[0050] The absolute difference is calculated as the absolute value of the difference between the predicted value corresponding to the virtual reward and the binary label of the downstream real quality detection result. The downstream real quality detection result is calculated in binary form with 1 for qualified and 0 for unqualified. The predicted value corresponding to the virtual reward is a continuous probability value within the interval of 0 and 1. The absolute value of the difference between the two reflects the magnitude of the deviation in a single prediction. When the absolute difference is written to the sliding window residual queue, the earliest record is popped simultaneously to ensure that the queue length is always maintained at W. The queue content is updated as new data arrives in each batch. The root mean square prediction residual is a scalar obtained by squaring all W absolute differences in the queue, calculating the arithmetic mean of the W squared values, and then taking the square root of the mean. This calculation method assigns higher weight to larger deviations and is more sensitive to the identification of occasional serious misjudgments by the downstream quality predictor compared to the simple mean. The value of the root mean square prediction residual ranges from zero to one; the larger the value, the worse the recent prediction quality of the predictor.

[0051] In one specific embodiment, in step S3, the root mean square prediction residual is converted into a confidence decay coefficient according to a piecewise linear mapping rule. The confidence decay coefficient is at its maximum value when the root mean square prediction residual does not exceed a first threshold, and at zero when the root mean square prediction residual exceeds a second threshold. When the root mean square prediction residual is between the first and second thresholds, the confidence decay coefficient is calculated using linear interpolation, including:

[0052] The root mean square prediction residual is compared with the first threshold. When the root mean square prediction residual does not exceed the first threshold, the confidence decay coefficient is set to the maximum value of one.

[0053] The root mean square prediction residual is compared with a second threshold. When the root mean square prediction residual exceeds the second threshold, the confidence decay coefficient is set to zero.

[0054] When the root mean square prediction residual is between the first threshold and the second threshold, the linear interpolation ratio is calculated using the difference between the root mean square prediction residual and the first threshold as the numerator and the difference between the second threshold and the first threshold as the denominator. The value after subtracting the linear interpolation ratio is used as the confidence decay coefficient to obtain the confidence decay coefficient.

[0055] Specifically, the confidence decay coefficient is a scalar ranging from 0 to 1. Its value reflects the weight of the virtual reward in the reward signal of the deep reinforcement learning agent at the current moment. When the confidence decay coefficient is 1, the reward signal is entirely composed of virtual rewards; when the confidence decay coefficient is 0, the reward signal is entirely composed of the value obtained by linearly mapping the downstream real quality detection results. The recommended value for the first threshold is 0.1. This value corresponds to the upper bound of the recent prediction deviation of the downstream quality predictor being within an acceptable level. A root mean square prediction residual of no more than 0.1 indicates that the predictor's judgment of the downstream quality is highly consistent with the real detection results. At this point, completely replacing real feedback with virtual rewards to drive agent learning will not introduce significant errors. The recommended value for the second threshold is 0.3. This value corresponds to the lower bound of the predictor's predicted quality decreasing to an unreliable level. A root mean square prediction residual exceeding 0.3 indicates a systematic deviation between the predictor's output and the real detection results. At this point, the virtual reward can no longer accurately reflect the real quality of the process state, and the confidence decay coefficient is set to 0 to shield the interference of virtual rewards.

[0056] The linear interpolation calculation between the two thresholds involves subtracting the first threshold of 0.1 from the root mean square prediction residual, dividing the difference by the difference between the second threshold of 0.3 and the first threshold of 0.1 (0.2), to obtain the linear interpolation ratio. This ratio reflects the relative position of the root mean square prediction residual within the range of 0.1 to 0.3; a larger ratio indicates that the prediction residual is closer to the unreliable boundary. The value obtained by subtracting this linear interpolation ratio from 1 is used as the confidence decay coefficient, which decreases linearly with the increase of the root mean square prediction residual. It is set to 1 at the first threshold of 0.1 and 0 at the second threshold of 0.3. This piecewise linear mapping rule ensures a continuous and smooth transition of the confidence decay coefficient as the predictor performance changes, avoiding abrupt changes in the confidence decay coefficient that could drastically impact the reward signal on the deep reinforcement learning agent's policy, thus ensuring the stability of the agent's decision-making at the acquisition frequency.

[0057] In one specific embodiment, step S3 involves fusing the confidence decay coefficient with the virtual reward as a weighted fusion to serve as the reward signal for the deep reinforcement learning agent, thereby driving the decision update of the acquisition frequency of each sensor channel. This includes:

[0058] Multiply the confidence decay coefficient by the virtual reward, multiply the value after subtracting the confidence decay coefficient by the value after linearly mapping the downstream real quality detection result to the negative-to-positive interval, and sum the products of the two to obtain the reward signal;

[0059] The reward signal, the current state vector, and the current acquisition frequency action are written into the experience playback buffer. Based on the batch samples sampled in the experience playback buffer, the Bellman target update is performed on the Q network of the deep reinforcement learning agent. The acquisition frequency level selection results of each sensor channel output by the updated Q network are written into the acquisition frequency vector to obtain the updated acquisition frequency vector.

[0060] Specifically, the reward signal is calculated by multiplying the confidence decay coefficient by the virtual reward, and then adding the product of the value obtained by subtracting the confidence decay coefficient from 1 and the linear mapping value of the downstream real quality inspection result. The sum of these two products is the reward signal for that time. The linear mapping rule for the downstream real quality inspection result is the same as the mapping rule for the virtual reward: multiply the binary label in the 0 to 1 interval by 2 and then subtract 1. The mapping value for qualified products is 1, and the mapping value for unqualified products is -1. When the confidence decay coefficient is 1, the reward signal equals the virtual reward, and the weight of the real inspection result item is 0; when the confidence decay coefficient is 0, the reward signal equals the linear mapping value of the real inspection result, and the weight of the virtual reward item is 0; when the confidence decay coefficient is between 0 and 1, the two are summed according to their respective weights, and the reward signal transitions continuously between the virtual reward and the real mapping value. The current state vector is formed by concatenating the normalized vector of the real-time acquired values ​​of each sensor channel with the one-hot encoded vector of the current acquisition frequency level of each channel. The acquisition frequency action is the frequency level switching operation performed on the specified sensor channel in the current decision. The three together constitute an experience sample written into the experience playback buffer.

[0061] The experience replay buffer adopts a first-in, first-out (FIFO) structure, with a recommended capacity of 50,000 records. Each time, 128 samples are randomly sampled from the buffer to form a batch sample for Q-network parameter updates. The Bellman target update execution logic is to use the reward signal of each record in the batch sample, plus a discount factor, multiplied by the maximum Q-value of the target Q-network for the next state, as the target Q-value for the current state action pair. A discount factor of 0.95 is recommended, balancing long-term cumulative rewards and recent immediate rewards. The Q-network outputs the corresponding Q-value for each selectable acquisition frequency action in the current state vector. The action with the largest Q-value is taken as the acquisition frequency level selection result for that time. The frequency level selection results for each sensor channel are arranged in channel number order to form an acquisition frequency vector. Each element in the acquisition frequency vector corresponds to the currently selected acquisition frequency level for a sensor channel. This vector serves as the update source for the frequency-coded components in the state vector at the next time step.

[0062] In one specific embodiment, step S4 includes:

[0063] The confidence decay coefficient historical sequence of production batches with a maintenance length of D is used. Each batch in the confidence decay coefficient historical sequence is assigned an exponential decay weight. The weighted sum of the product of the confidence decay coefficient of each batch and the corresponding weight is divided by the sum of all weights to obtain the cross-process coupling strength score.

[0064] When the cross-process coupling strength score is lower than the preset coupling strength threshold, calculate the Pearson correlation coefficient between the real-time upstream acquisition feature vector and the downstream quality detection sequence of each sensor channel, and filter the inefficient acquisition channel set and the process boundary node channel set according to the ascending order of the Pearson correlation coefficient.

[0065] The acquisition frequency level of each channel in the inefficient acquisition channel set is lowered by one level, and the acquisition frequency level of each channel in the process boundary node channel set is raised by one level to obtain the updated acquisition frequency vector.

[0066] The updated acquisition frequency vector replaces the frequency coding component in the state vector of the deep reinforcement learning agent in the form of one-hot coding, and the historical sample weights corresponding to the old configuration in the experience replay buffer are reset to zero.

[0067] Specifically, the cross-process coupling strength score is a scalar calculated by exponentially decaying weighted average of the historical confidence decay coefficients of D production batches. The recommended value for D is the number of production batches corresponding to 7 days. This value covers the fluctuations of a complete production cycle, reflecting the recent contribution trend of the data collection configuration to downstream quality prediction without being affected by single-day anomalies due to a short historical data span. The exponential decay weight is calculated by assigning a weight exp(0.1×(iD)) to the i-th batch, where i=D corresponds to the latest batch with the largest weight, and i=1 corresponds to the earliest batch with the smallest weight. The decay coefficient of 0.1 ensures that the weight of the latest batch is approximately 2.7 times that of the earliest batch, making the score more sensitive to recent changes in the confidence decay coefficient. The recommended preset coupling strength threshold is 0.4. A value below 0.4 indicates that the weighted average of the recent confidence decay coefficient is consistently low, and the current process data collection channel configuration does not contribute enough to downstream quality prediction, triggering a data collection frequency reallocation process.

[0068] The Pearson correlation coefficient measures the linear correlation between the real-time upstream acquisition feature vectors of each sensor channel and the downstream quality detection sequence. Its value ranges from -1 to 1; a larger absolute value indicates a stronger correlation between the channel's acquisition data and downstream quality. After sorting all sensor channels by Pearson correlation coefficients in ascending order, the top 20% of channels are selected as the inefficient acquisition channel set. This proportion strikes a balance between releasing sufficient acquisition resources and avoiding excessive reduction of effective channels. The bottom 10% of channels are selected as the process boundary node channel set. Channels in this set have the highest correlation with downstream quality; increasing their acquisition frequency helps improve the input feature quality of the downstream quality predictor. In the inefficient acquisition channel set, the frequency level of each channel is lowered by one level in the preset frequency level sequence, and in the process boundary node channel set, the frequency level of each channel is raised by one level. After the updated acquisition frequency vector is expanded in the form of one-hot encoding, the frequency encoding component in the state vector of the deep reinforcement learning agent is replaced. The historical sample weights corresponding to the old acquisition frequency configuration in the experience replay buffer are reset to 0, so that the agent's subsequent Q-network updates only rely on the effective experience samples generated under the new acquisition configuration, avoiding the interference of old configuration experience on the new strategy.

[0069] Figure 2 This is a schematic diagram of the Pearson correlation coefficient distribution of the sensor channels in an embodiment of this application. Figure 2 The diagram illustrates the Pearson correlation coefficient calculation results between the real-time upstream acquisition feature vectors and the downstream quality inspection sequence of all 16 sensor channels on the production line in a trigger event where the cross-process coupling strength score falls below the preset coupling strength threshold of 0.40. The correlation coefficients of each channel are arranged in ascending order from bottom to top. Among them, the Pearson correlation coefficients of channels CH01, CH02, and CH03 are all negative, located in the lowest 20% percentile of the ascending order, and are identified as an inefficient acquisition channel set, marked with a diagonal-filled pattern. Their acquisition frequency levels are each lowered by one level in the preset frequency level sequence. The Pearson correlation coefficients of channels CH15 and CH16 are 0.75 and 0.81, respectively, located in the highest 10% percentile of the ascending order, and are identified as a process boundary node channel set, marked with a cross-filled pattern. Their acquisition frequency levels are each raised by one level. The remaining channels are ordinary channels, and their acquisition frequency levels remain unchanged. This completes a dynamic reallocation of cross-process acquisition resources.

[0070] The above describes the industrial data acquisition, analysis, and processing method based on deep reinforcement learning in the embodiments of this application. The following describes the industrial data acquisition, analysis, and processing system based on deep reinforcement learning in the embodiments of this application. One embodiment of the industrial data acquisition, analysis, and processing system based on deep reinforcement learning in the embodiments of this application includes:

[0071] The data acquisition module is used to calculate the time offset between the historical upstream acquisition sequence and downstream quality inspection sequence of a multi-process cascaded production line by using a cross-correlation function, thereby obtaining the physical time delay between processes.

[0072] The mapping module is used to remap the physical delay between processes to the upstream acquisition sequence and the downstream quality detection sequence using timestamps. The downstream quality predictor is trained using the remapped time-aligned dataset. The downstream quality predictor predicts the real-time upstream acquisition feature vector to obtain a virtual reward.

[0073] The driving module is used to calculate the root mean square prediction residual between the predicted value corresponding to the virtual reward and the downstream real quality detection result within a sliding window when the downstream real quality detection result arrives. The root mean square prediction residual is converted into a confidence decay coefficient according to a piecewise linear mapping rule. When the root mean square prediction residual does not exceed a first threshold, the confidence decay coefficient takes the maximum value. When the root mean square prediction residual exceeds a second threshold, the confidence decay coefficient takes the zero value. When the root mean square prediction residual is between the first threshold and the second threshold, the confidence decay coefficient is calculated by linear interpolation. The confidence decay coefficient and the virtual reward are weighted and fused together to serve as the reward signal for the deep reinforcement learning agent, so as to drive the decision update of the acquisition frequency of each sensor channel.

[0074] The allocation module is used to calculate the cross-process coupling strength score based on the historical sequence of the confidence decay coefficient, perform acquisition frequency reallocation on each sensor channel according to the cross-process coupling strength score, and synchronize the updated acquisition frequency vector to the state space of the deep reinforcement learning agent.

[0075] This invention also provides an industrial data acquisition, analysis, and processing device based on deep reinforcement learning, which can be a server. The device includes a processor, memory, display screen, input device, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores the data corresponding to this embodiment. The network interface communicates with external terminals via a network connection. The computer program is executed by the processor to implement the above-described method.

[0076] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the industrial data acquisition, analysis and processing method based on deep reinforcement learning.

[0077] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0078] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the existing solution, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a deep reinforcement learning-based industrial data acquisition, analysis, and processing device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0079] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for industrial data acquisition, analysis, and processing based on deep reinforcement learning, characterized in that, The method includes: Step S1: Based on the historical upstream acquisition sequence and downstream quality inspection sequence of the multi-process cascaded production line, calculate the time offset between the two through the cross-correlation function to obtain the physical time delay between processes; Step S2: Remap the timestamps of the upstream acquisition sequence and the downstream quality detection sequence by the physical delay between processes, train the downstream quality predictor with the remapped time-aligned dataset, and use the downstream quality predictor to predict the real-time upstream acquisition feature vector to obtain the virtual reward. Step S3: When the downstream real quality detection result arrives, calculate the root mean square prediction residual between the predicted value corresponding to the virtual reward and the downstream real quality detection result within the sliding window. Convert the root mean square prediction residual into a confidence decay coefficient according to a piecewise linear mapping rule. When the root mean square prediction residual does not exceed the first threshold, the confidence decay coefficient takes the maximum value. When the root mean square prediction residual exceeds the second threshold, the confidence decay coefficient takes the zero value. When the root mean square prediction residual is between the first threshold and the second threshold, the confidence decay coefficient is calculated by linear interpolation. The confidence decay coefficient and the virtual reward are weighted and fused together to serve as the reward signal for the deep reinforcement learning agent, thereby driving the decision update of the acquisition frequency of each sensor channel. Step S4: Calculate the cross-process coupling strength score based on the historical sequence of the confidence decay coefficient, redistribute the acquisition frequency of each sensor channel according to the cross-process coupling strength score, and synchronize the updated acquisition frequency vector to the state space of the deep reinforcement learning agent.

2. The industrial data acquisition, analysis, and processing method based on deep reinforcement learning according to claim 1, characterized in that, Step S1 includes: Extract data records of N consecutive production batches from the historical production database, and perform batch alignment between the upstream collection sequence and the downstream quality inspection sequence based on the completion timestamp of each batch to obtain a batch-aligned record set; The historical upstream acquisition sequence and downstream quality inspection sequence in the batch alignment record set are input into the cross-correlation function. The cross-correlation number is calculated step by step within the time delay search range. The time offset corresponding to the maximum value of the cross-correlation number is taken as the candidate value of single batch time delay. The median of the candidate values ​​of single batch time delay for all N batches is taken to obtain the physical time delay between processes. Based on the inter-process physical delay, calculate the confidence interval of N single-batch delay candidate values. When the ratio of the confidence interval width to the inter-process physical delay exceeds a preset ratio threshold, expand N to 2N and re-execute the above median calculation until the ratio of the confidence interval width to the inter-process physical delay does not exceed the preset ratio threshold. The physical delay between processes is used to remap the timestamps of the historical upstream acquisition sequence and the downstream quality detection sequence. The remapped upstream acquisition sequence is then paired with the corresponding downstream quality detection sequence one by one according to the timestamp to obtain a time-aligned dataset.

3. The industrial data acquisition, analysis, and processing method based on deep reinforcement learning according to claim 1, characterized in that, Step S2 includes: The time-aligned dataset is divided into a training set, a validation set, and a test set according to a preset ratio. The downstream quality predictor is iteratively trained based on the training set and the validation set, with the binary cross-entropy loss function as the training objective. Training is terminated when the validation set loss does not decrease within a preset number of consecutive rounds. The F1 score of the trained downstream quality predictor is calculated on the test set. When the F1 score is lower than a preset passing threshold, the time-aligned dataset is expanded and retrained until the F1 score is not lower than the preset passing threshold, thus obtaining the downstream quality predictor. The real-time upstream feature vector is input into the downstream quality predictor for forward inference to obtain the downstream quality pass probability at the current moment. Based on the downstream quality pass probability, an interval transformation is performed according to a linear mapping rule to map the downstream quality pass probability from the zero-one interval to the negative-one-positive-one interval, thereby obtaining a virtual reward. The virtual rewards are bound one by one to the timestamps of their corresponding real-time upstream collected feature vectors, and written into the virtual reward sequence in the order of the timestamps to obtain the virtual reward sequence.

4. The industrial data acquisition, analysis, and processing method based on deep reinforcement learning according to claim 3, characterized in that, In step S3, when the downstream actual quality detection result arrives, the root mean square prediction residual between the predicted value corresponding to the virtual reward and the downstream actual quality detection result is calculated within a sliding window, including: Maintain a sliding window residual queue of length W. Whenever a downstream real quality inspection result arrives, retrieve the predicted value of the virtual reward corresponding to the physical delay between the timestamp of the corresponding downstream real quality inspection result and the process time from the virtual reward sequence. Calculate the absolute difference between the predicted value and the downstream real quality inspection result. Write the absolute difference into the sliding window residual queue and pop the earliest record to obtain the updated sliding window residual queue. Based on all W absolute differences in the updated sliding window residual queue, the root mean square prediction residual is obtained by squaring each absolute difference, taking the mean, and then taking the square root.

5. The industrial data acquisition, analysis, and processing method based on deep reinforcement learning according to claim 4, characterized in that, In step S3, the root mean square prediction residual is converted into a confidence decay coefficient according to a piecewise linear mapping rule. The confidence decay coefficient is at its maximum value when the root mean square prediction residual does not exceed a first threshold; it is at zero when the root mean square prediction residual exceeds a second threshold; and it is calculated by linear interpolation when the root mean square prediction residual is between the first and second thresholds, including: The root mean square prediction residual is compared with a first threshold. When the root mean square prediction residual does not exceed the first threshold, the confidence decay coefficient is set to the maximum value of one. The root mean square prediction residual is compared with a second threshold. When the root mean square prediction residual exceeds the second threshold, the confidence decay coefficient is set to zero. When the root mean square prediction residual is between the first threshold and the second threshold, the linear interpolation ratio is calculated using the difference between the root mean square prediction residual and the first threshold as the numerator and the difference between the second threshold and the first threshold as the denominator. The value after subtracting the linear interpolation ratio is used as the confidence decay coefficient to obtain the confidence decay coefficient.

6. The industrial data acquisition, analysis, and processing method based on deep reinforcement learning according to claim 5, characterized in that, In step S3, the confidence decay coefficient and the virtual reward are weighted and fused together to serve as the reward signal for the deep reinforcement learning agent, thereby driving the decision update of the acquisition frequency of each sensor channel, including: Multiply the confidence decay coefficient by the virtual reward, multiply the value after subtracting the confidence decay coefficient by the value after linearly mapping the downstream real quality detection result to the negative-to-positive interval, and sum the products of the two to obtain the reward signal; The reward signal, the current state vector, and the current acquisition frequency action are written into the experience playback buffer. Based on the batch samples sampled in the experience playback buffer, the Bellman target update is performed on the Q network of the deep reinforcement learning agent. The acquisition frequency level selection results of each sensor channel output by the updated Q network are written into the acquisition frequency vector to obtain the updated acquisition frequency vector.

7. The industrial data acquisition, analysis, and processing method based on deep reinforcement learning according to claim 6, characterized in that, Step S4 includes: The confidence decay coefficient historical sequence of production batches with a maintenance length of D is used. Each batch in the confidence decay coefficient historical sequence is assigned an exponential decay weight. The weighted sum of the product of the confidence decay coefficient of each batch and the corresponding weight is divided by the sum of all weights to obtain the cross-process coupling strength score. When the cross-process coupling strength score is lower than the preset coupling strength threshold, the Pearson correlation coefficient between the real-time upstream acquisition feature vector of each sensor channel and the downstream quality detection sequence is calculated, and the inefficient acquisition channel set and the process boundary node channel set are screened according to the ascending order of the Pearson correlation coefficient. The acquisition frequency level of each channel in the inefficient acquisition channel set is lowered by one level, and the acquisition frequency level of each channel in the process boundary node channel set is raised by one level to obtain the updated acquisition frequency vector. The updated acquisition frequency vector replaces the frequency coding component in the state vector of the deep reinforcement learning agent in a one-hot coding form, and the historical sample weights corresponding to the old configuration in the experience replay buffer are reset to zero.

8. An industrial data acquisition, analysis, and processing system based on deep reinforcement learning, characterized in that, For implementing the deep reinforcement learning-based industrial data acquisition, analysis, and processing method as described in any one of claims 1-7, the deep reinforcement learning-based industrial data acquisition, analysis, and processing system comprises: The data acquisition module is used to calculate the time offset between the historical upstream acquisition sequence and downstream quality inspection sequence of a multi-process cascaded production line by using a cross-correlation function, thereby obtaining the physical time delay between processes. The mapping module is used to remap the physical delay between processes to the upstream acquisition sequence and the downstream quality detection sequence using timestamps. The downstream quality predictor is trained using the remapped time-aligned dataset. The downstream quality predictor predicts the real-time upstream acquisition feature vector to obtain a virtual reward. The driving module is used to calculate the root mean square prediction residual between the predicted value corresponding to the virtual reward and the downstream real quality detection result within a sliding window when the downstream real quality detection result arrives. The root mean square prediction residual is converted into a confidence decay coefficient according to a piecewise linear mapping rule. When the root mean square prediction residual does not exceed a first threshold, the confidence decay coefficient takes the maximum value. When the root mean square prediction residual exceeds a second threshold, the confidence decay coefficient takes the zero value. When the root mean square prediction residual is between the first threshold and the second threshold, the confidence decay coefficient is calculated by linear interpolation. The confidence decay coefficient and the virtual reward are weighted and fused together to serve as the reward signal for the deep reinforcement learning agent, so as to drive the decision update of the acquisition frequency of each sensor channel. The allocation module is used to calculate the cross-process coupling strength score based on the historical sequence of the confidence decay coefficient, perform acquisition frequency reallocation on each sensor channel according to the cross-process coupling strength score, and synchronize the updated acquisition frequency vector to the state space of the deep reinforcement learning agent.

9. An industrial data acquisition, analysis, and processing device based on deep reinforcement learning, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the industrial data acquisition, analysis and processing method based on deep reinforcement learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, it causes the processor to execute the industrial data acquisition, analysis and processing method based on deep reinforcement learning as described in any one of claims 1 to 7.

Images