Machine vision production line efficiency evaluation and optimization management system

Abstract

The invention relates to the technical field of industrial manufacturing digital management, in particular to a machine vision production line performance evaluation and optimization management system, which comprises a data acquisition module for triggering a high-speed industrial camera array, a vibration sensor and an RFID reader through a central synchronous controller to synchronously acquire product images, equipment operation and material circulation data; the data processing and fusion module extracts product quality features based on CNN, and fuses multi-modal data through time sequence alignment normalization and an attention mechanism; the dynamic efficiency evaluation module calculates OEE, FPY and a production line balance rate in real time by means of a deep neural network; the optimization strategy generation module is used for reinforcing the learning agent to output optimization instructions such as equipment parameter adjustment; and the control execution module converts the instruction into an industrial protocol format, issues the instruction to the PLC, and verifies the effect to form a closed loop. According to the method, the data relevance and the evaluation real-time performance are improved, the dynamic state of the adaptive production line is optimized, and the efficiency improvement is facilitated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital management of industrial manufacturing, in particular to a machine vision production line efficiency evaluation and optimization management system. BACKGROUND

[0002] The current production line management scheme involving machine vision has many limitations and cannot meet the efficient evaluation and optimization requirements. The specific problems are as follows: Poor synchronization of machine vision data collection: Although some production lines have introduced high-speed industrial cameras and other machine vision equipment to collect product image data, a unified synchronization mechanism has not been established, resulting in inconsistent timestamps between the product image data obtained by machine vision and the device operation state data collected by vibration sensors and the material flow data collected by RFID readers. The corresponding relationship between "product visual features-device state-material flow" cannot be accurately associated, and it is difficult to locate the root cause of product quality problems (such as surface defects) (such as device vibration abnormalities); Machine vision is disconnected from efficiency evaluation and optimization: In existing systems, machine vision data are mainly used for single product quality inspection, and they are not deeply combined with production line efficiency evaluation (such as first pass yield FPY and overall equipment effectiveness OEE). Efficiency evaluation still relies on manual statistics, which is lagging and subjective. At the same time, optimization strategies are not based on real-time feedback from machine vision quality data, and device parameters and production rhythm cannot be dynamically adjusted according to changes in product visual features (such as rising defect rate), which means that the real-time advantage of machine vision has not been translated into production line optimization power.

[0003] Therefore, a machine vision production line efficiency evaluation and optimization management system is proposed to solve the above problems. SUMMARY

[0004] The present application aims to provide a machine vision production line efficiency evaluation and optimization management system to solve the problems raised in the background.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical solutions: A machine vision production line efficiency evaluation and optimization management system, comprising: A data collection module comprising a high-speed industrial camera array, vibration sensors and RFID readers deployed on the production line, for synchronously collecting image data of products, operation state data of devices and flow data of materials; A data processing and fusion module connected to the data collection module, comprising: An image processing unit based on a deep learning convolutional neural network model for feature extraction of image data to obtain product quality feature vectors; a data standardization unit, configured to perform time alignment and normalization processing on the running state data and the flow data to generate standardized equipment state data; a multi-modal data fusion unit, configured to perform fusion on the product quality feature vector and the standardized equipment state data by using a fusion algorithm based on an attention mechanism to generate a production line fusion feature set; a dynamic performance evaluation module, connected to the data processing and fusion module, configured to input the production line fusion feature set into a trained performance evaluation model to calculate, in real time, key performance indicators including at least an overall equipment effectiveness (OEE), a first-pass yield (FPY) and a production line balancing rate; an optimization strategy generation module, connected to the dynamic performance evaluation module, and internally provided with a reinforcement learning agent, the reinforcement learning agent taking the key performance indicators as state inputs of a reward function, learning an output optimization action strategy by interacting with an environment, the optimization action strategy including a device parameter adjustment instruction, a production rhythm control instruction or a re-planning instruction for a feeding mechanical arm path; a control execution module, connected to the optimization strategy generation module and a programmable logic controller (PLC) on the production line, configured to convert the optimization action strategy into a control instruction conforming to an industrial communication protocol and send the control instruction to a corresponding execution unit to form a closed-loop optimization control system.

[0006] As a preferred solution, in the data acquisition module, the high-speed industrial camera array, the vibration sensor and the RFID reader perform data acquisition through a synchronous triggering mechanism, wherein the synchronous triggering mechanism is managed by a central synchronization controller, the central synchronization controller generates a synchronization pulse signal to simultaneously trigger the high-speed industrial camera array to capture product multi-angle image data, the vibration sensor to collect device vibration frequency and amplitude data, and the RFID reader to read material RFID tag data, so as to ensure that all collected data have consistent time stamps; the data acquisition module further packs the collected image data, running state data and flow data into time sequence data packets and transmits the time sequence data packets to the data processing and fusion module through an industrial Ethernet.

[0007] As a preferred solution, the image processing unit extracts features from the image data based on a deep learning convolutional neural network model to obtain a product quality feature vector, including: preprocessing the collected image data, including image denoising, size adjustment and pixel value normalization, to generate standardized image data; inputting the standardized image data into a pre-trained deep learning convolutional neural network model to perform forward propagation processing through multiple convolutional layers, activation function layers and pooling layers to generate multi-level feature maps; performing feature fusion processing on the multi-level feature maps to extract high-level semantic features through a fully connected layer or a global pooling layer in the convolutional neural network to generate an initial feature vector; The initial feature vector is dimensionally reduced and standardized to obtain a final feature vector representing product quality.

[0008] As a preferred solution, the data standardization unit is configured to perform time series alignment and normalization on the operating state data and the flow data to generate standardized equipment state data, including: The operating state data and the flow data are received from the data acquisition module, wherein the operating state data includes equipment vibration frequency and amplitude data, and the flow data includes material RFID tag data; The operating state data and the flow data are timestamped to extract the original time information of each data point; Based on the synchronization time reference provided by the central synchronization controller, the operating state data and the flow data after timestamping are time series aligned, and the data is resampled to a unified time sequence by a linear interpolation method to ensure that all data points have consistent time intervals; The operating state data and the flow data after time series alignment are normalized, and the minimum-maximum normalization algorithm is used to map the data values to the range of [0, 1] to generate normalized data; The normalized data is combined into standardized equipment state data and output to the multi-modal data fusion unit.

[0009] As a preferred solution, the multi-modal data fusion unit uses an attention mechanism-based fusion algorithm to fuse the product quality feature vector and the standardized equipment state data to generate a production line fusion feature set, including: The product quality feature vector from the image processing unit and the standardized equipment state data from the data standardization unit are received; The product quality feature vector and the standardized equipment state data are respectively input into a linear mapping layer to generate corresponding query vectors, key vectors and value vectors; The dot product similarity between the query vector and the key vector is calculated, and the similarity is normalized by softmax to generate attention weights; The value vector is weighted and summed using the attention weights to generate a weighted context feature vector; The weighted context feature vector and the product quality feature vector are spliced and combined to generate a preliminary fusion feature vector; The preliminary fusion feature vector is processed by a full connection layer and an activation function to generate a final production line fusion feature set.

[0010] As a preferred solution, the dynamic performance evaluation module is configured to input the production line fusion feature set into a trained performance evaluation model to calculate at least key performance indicators including equipment comprehensive efficiency OEE, first pass yield FPY and production line balance rate in real time, including: receiving a line fusion feature set from the multi-modal data fusion unit; performing time window division processing on the line fusion feature set to generate a feature sequence of a fixed time length; inputting the feature sequence into a trained performance evaluation model, the performance evaluation model being based on a deep neural network architecture, receiving the feature sequence through an input layer, performing nonlinear transformation and feature extraction through multiple hidden layers, and outputting a preliminary performance index vector, performing inverse normalization processing on the preliminary performance index vector, mapping the model output value to an actual physical range, and generating a standardized performance index value; based on the standardized performance index value, analyzing the numerical values of the equipment overall efficiency OEE, the first pass yield FPY, and the line balance rate; outputting the analyzed key performance indicators to the optimization strategy generation module.

[0011] As a preferred solution, the optimization strategy generation module is built-in with a reinforcement learning agent, which takes the key performance indicators as the state input of the reward function, learns the output optimization action strategy by interacting with the environment, and the optimization action strategy includes device parameter adjustment instructions, production rhythm control instructions, or feeding mechanical arm path re-planning instructions, including: receiving the key performance indicators from the dynamic performance evaluation module; inputting the key performance indicators as the current environment state into the state space of the reinforcement learning agent; the reinforcement learning agent calculates the action value function based on the current environment state through the policy network to generate a candidate action set; performing exploration and utilization balancing processing on the candidate action set, and selecting the optimal action using an ε-greedy strategy; mapping the optimal action to the original action strategy, including device parameter adjustment amount, production rhythm adjustment value, or mechanical arm path point sequence; performing compliance checking on the original action strategy to ensure that it meets the production line safety constraints and operating range; encoding the compliant original action strategy into specific device parameter adjustment instructions, production rhythm control instructions, or feeding mechanical arm path re-planning instructions; outputting the optimization action strategy to the control execution module.

[0012] As a preferred solution, the control execution module is connected to the optimization strategy generation module and the programmable logic controller PLC on the production line, used to convert the optimization action strategy into control instructions that meet the industrial communication protocol, and issue them to the corresponding execution unit, including: receiving the optimization action strategy from the optimization strategy generation module, the optimization action strategy including device parameter adjustment instructions, production rhythm control instructions, or feeding mechanical arm path re-planning instructions; The received optimization action strategy is parsed and processed to extract the action type and specific parameter value; According to the extracted action type, a corresponding industrial communication protocol template is selected, which includes Modbus, Profinet or EtherNet / IP; The extracted specific parameter value is filled into the data field of the selected industrial communication protocol template to generate an original control instruction frame; The original control instruction frame is subjected to check code calculation and packaging processing to generate a standard control instruction conforming to the industrial communication protocol; The standard control instruction is issued to the programmable logic controller PLC on the production line through the industrial Ethernet; After the programmable logic controller PLC parses and executes the standard control instruction, the control execution module acquires the production line data after execution through the data acquisition module to verify the instruction execution effect and form a closed-loop optimization control system.

[0013] As can be seen from the technical solutions provided by the above-mentioned application, the machine vision production line efficiency evaluation and optimization management system provided by the application has the following beneficial effects: More accurate and comprehensive data acquisition and processing: through the central synchronous controller, synchronous data acquisition of high-speed industrial cameras, vibration sensors and RFID readers is realized to ensure consistent data timestamps and avoid time misalignment; with the help of deep learning CNN, product quality features are extracted, time sequence alignment and normalization processing of equipment / flow data are performed, and multi-modal data are fused through the attention mechanism to break through the limitation of single data and provide high-quality data support for subsequent analysis; Real-time and accurate efficiency evaluation: based on the trained deep neural network model, OEE, FPY and production line balance rate are calculated in real time according to the preset time window, replacing traditional post-event manual statistics, and efficiency fluctuations can be found in time; the evaluation process is fully automated, manual subjective errors are reduced through model calculation and reverse normalization processing, and the accuracy and consistency of the indicators are guaranteed; Intelligent and safe optimization strategy: the built-in reinforcement learning agent takes real-time efficiency indicators as input to dynamically generate optimization actions such as device parameter adjustment and production rhythm control, which can adapt to dynamic changes in the production line and balance strategy exploration and utilization; before generating the strategy, compliance check is performed to ensure compliance with production line safety constraints and operation range, avoiding the risk of illegal adjustment; Form a closed-loop optimization to ensure the effect: the control execution module converts the optimization strategy into instructions conforming to the industrial protocol (Modbus, Profinet, etc.) and issues them to the PLC, and then acquires production line data through the data acquisition module to verify the effect, forming a closed loop of "collection-processing-evaluation-optimization-execution-feedback" to ensure that the optimization action is accurately implemented and effectively improves the production line efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 Figure 1 is a schematic diagram of the overall structure of a machine vision production line performance evaluation and optimization management system according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0016] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in combination with the drawings and specific embodiments of the present application.

[0017] As shown in Figure 1 An embodiment of the present application provides a machine vision production line performance evaluation and optimization management system, which comprises: A data acquisition module comprising a high-speed industrial camera array, a vibration sensor and an RFID reader deployed on the production line, for synchronously acquiring image data of products, running state data of equipment and flow data of materials; A data processing and fusion module connected to the data acquisition module, which comprises: An image processing unit for feature extraction of the image data based on a deep learning convolutional neural network model, to obtain a product quality feature vector; A data standardization unit for time series alignment and normalization processing of the running state data and the flow data, to generate standardized equipment state data; A multi-modal data fusion unit for fusion of the product quality feature vector and the standardized equipment state data using a fusion algorithm based on an attention mechanism, to generate a production line fusion feature set; A dynamic performance evaluation module connected to the data processing and fusion module, for inputting the production line fusion feature set into a trained performance evaluation model, to calculate key performance indicators including at least equipment comprehensive efficiency OEE, first pass yield FPY and production line balancing rate in real time; An optimization strategy generation module connected to the dynamic performance evaluation module, which has a reinforcement learning agent built-in, the reinforcement learning agent taking the key performance indicators as state inputs of a reward function, learning an output optimization action strategy through interaction with the environment, the optimization action strategy including equipment parameter adjustment instructions, production rhythm control instructions or re-planning instructions for the path of a feeding robot arm; A control execution module connected to the optimization strategy generation module and a programmable logic controller PLC on the production line, for converting the optimization action strategy into control instructions conforming to an industrial communication protocol, and issuing them to the corresponding execution unit to constitute a closed-loop optimization control system.

[0018] In this embodiment, in the data acquisition module, the high-speed industrial camera array, the vibration sensor and the RFID reader perform data acquisition through a synchronous triggering mechanism, wherein the synchronous triggering mechanism is managed by a central synchronization controller, the central synchronization controller generates a synchronization pulse signal to simultaneously trigger the high-speed industrial camera array to capture product multi-angle image data, the vibration sensor to collect equipment vibration frequency and amplitude data, and the RFID reader to read material RFID tag data, so as to ensure that all collected data have consistent time stamps; the data acquisition module also packs the collected image data, running state data and flow data into time sequence data packets and transmits them to the data processing and fusion module through an industrial Ethernet; Further, the data acquisition module mainly consists of a high-speed industrial camera array, a vibration sensor and an RFID reader deployed on the production line, and the core function is to synchronously collect three types of key data: first, multi-angle image data of the product (for subsequent extraction of product quality features); second, running state data of the equipment (specifically, equipment vibration frequency and amplitude data, reflecting the equipment running health degree); third, flow data of the material (specifically, material RFID tag data, recording material identity, flow location and other information); the module ensures that the data collected by the three types of equipment have consistent time stamps through the synchronous triggering mechanism managed by the central synchronization controller, avoiding time misalignment; at the same time, the collected image data, running state data and flow data are packed into time sequence data packets and transmitted in real time to the data processing and fusion module through an industrial Ethernet, providing complete and accurate raw data support for subsequent data analysis and efficiency evaluation of the system; The high-speed industrial camera array includes: Core hardware composition: composed of multiple industrial high-speed cameras, deployed at key observation points (such as after product processing, after assembly completion, etc.) of product flow according to product transmission path and detection requirements of the production line, to ensure that images of the product from different perspectives can be captured; Data acquisition function: only start acquisition when receiving the synchronization pulse signal issued by the central synchronization controller, real-time capture multi-angle image data of the product (such as product surface defects, size precision, assembly state, etc. visual information); during the acquisition process, each frame of image is automatically labeled with the time stamp corresponding to the current synchronization pulse, to ensure the time consistency of the image data and other types of data; Data output: real-time transmission of the collected multi-angle image data to the data packing and transmission unit as the raw data source for subsequent image processing unit to extract product quality feature vectors; The vibration sensor includes: Core hardware composition: Industrial-grade high-precision vibration sensors are installed on vibration-sensitive parts (such as the equipment spindle and base) of key equipment on the production line (such as processing machine tools, conveyor belt motors, assembly robotic arms, etc.) to ensure accurate capture of vibration signals during equipment operation; Data acquisition function: In response to the synchronous pulse signal of the central synchronous controller, the vibration frequency and amplitude data of the equipment are collected at the moment of pulse triggering (this data directly reflects the operating status of the equipment, such as whether the equipment has abnormal vibration, component wear and other problems); the collected data is marked with a synchronous timestamp in real time to avoid time deviation from other data; Data output: The collected equipment vibration frequency and amplitude data (i.e., equipment operating status data) are transmitted to the data packaging and transmission unit to provide the original equipment status data for the timing alignment and normalization processing of the data standardization unit; RFID readers include: Core hardware components: Deployed at key points of material flow on the production line (such as material entry, process conversion, finished product exit, etc.), equipped with a high-sensitivity radio frequency identification module that supports reading RFID tag information on the surface of materials or integrated inside them; Data collection function: When triggered by the synchronous pulse of the central synchronous controller, the RFID tag data of the material flowing through its identification area is read to obtain the material's unique identification (such as material number), production batch, flow progress, process to which it belongs, and other flow information (i.e. material flow data). If no material passes through in the current cycle, "no valid flow data" is recorded and marked with the corresponding timestamp to ensure the integrity of the data record; Data output: The collected material RFID tag data (or the "no valid circulation data" mark) is transmitted to the data packaging and transmission unit as the circulation data source processed by the data standardization unit; Central synchronization control unit: Core hardware composition: It consists of an industrial-grade central synchronization controller with a built-in high-precision clock module and pulse generation circuit, and can preset the generation cycle of the synchronization pulse (the cycle can be flexibly set according to the production line beat, such as 100ms / time, 200ms / time, etc.); Synchronous triggering function: Generates a unified synchronous pulse signal according to a preset period. This signal is simultaneously sent to the high-speed industrial camera array, vibration sensor, and RFID reader through a dedicated signal line, ensuring that the three types of collection devices start data collection operations at the same time point, eliminating the time difference between different devices collecting data at the source; Timestamp management: Assign a unique timestamp (accurate to the millisecond level) to each synchronization pulse and send it synchronously to the data packaging and transmission unit to ensure that all collected data can be marked with the corresponding unified timestamp, providing a benchmark for timing alignment of subsequent data processing; Data packaging and transmission unit: Data receiving function: real-time receiving of image data transmitted by high-speed industrial camera array, operation state data transmitted by vibration sensor, flow data transmitted by RFID reader, and corresponding timestamp information issued by central synchronization controller; Data packaging function: according to the format of "uniform timestamp + data type identification + original data", the three types of data (or part of the effective data) under the same timestamp are integrated into a structured time series data packet; for example, the data packet under a certain timestamp contains "timestamp T1 + image data (device A view 1-3) + operation state data (vibration frequency f1, amplitude A1) + flow data (material number M001)", ensuring the time correlation of the data in the data packet; Data transmission function: through the industrial Ethernet link, the packaged time series data packet is transmitted to the data processing and fusion module in real time; during transmission, an industrial-level data verification mechanism (such as CRC cyclic redundancy check) is used to ensure that the data is not lost or tampered with during transmission, and to adapt to the complex electromagnetic environment in the industrial field.

[0019] In this embodiment, the data processing and fusion module is connected to the data acquisition module, and the core is composed of an image processing unit, a data standardization unit and a multi-modal data fusion unit, and the overall function is divided into two stages: Data processing stage: two types of core data transmitted by the data acquisition module are processed respectively—feature extraction is performed on product image data through the image processing unit to generate a feature vector representing product quality; time alignment and normalization are performed on device operation state data (vibration frequency, amplitude) and material flow data (RFID tag data) through the data standardization unit to eliminate data dimension difference and time misalignment, and generate standardized device state data; Multi-modal fusion stage: through the multi-modal data fusion unit, a fusion algorithm based on attention mechanism is used to deeply fuse the product quality feature vector and the standardized device state data to generate a production line fusion feature set containing product, device and material related information, and finally output to the dynamic performance evaluation module to provide high-quality feature input for real-time calculation of key performance indicators (OEE, FPY, production line balance rate); The data processing and fusion module includes: Image processing unit: The core function of the image processing unit is to extract a feature vector representing the quality state from the product image data based on a deep learning convolutional neural network (CNN) model. The image processing unit extracts features from image data based on a deep learning convolutional neural network model to obtain a product quality feature vector, including: Preprocessing the collected image data, including image denoising, size adjustment and pixel value normalization, to generate standardized image data; The standardized image data is input into a pre-trained deep learning convolutional neural network model, and forward propagation processing is performed through multiple convolutional layers, activation function layers, and pooling layers to generate multi-level feature maps; The multi-level feature maps are subjected to feature fusion processing, and high-level semantic features are extracted through the fully connected layer or global pooling layer in the convolutional neural network to generate an initial feature vector; The initial feature vector is subjected to dimension reduction and standardization processing to obtain a final feature vector representing product quality; The specific process and functions are as follows: Image preprocessing: receiving product multi-angle image data transmitted by the data acquisition module and performing three basic processes: Image denoising: using an industrial-level denoising algorithm (such as Gaussian filtering, median filtering) to remove industrial environmental noise (such as workshop light interference, equipment reflection) in the image; Size adjustment: uniformly scaling images of different angles and different resolutions to a preset size (such as 224x224 pixels) to adapt to the input requirements of the pre-trained CNN model; Pixel value normalization: mapping the image pixel value from the [0, 255] range to the [0, 1] or [-1, 1] range (according to the pre-trained model parameters) to eliminate the influence of pixel value magnitude differences on model training and feature extraction, and finally generate standardized image data; CNN model forward propagation: inputting the standardized image data into a pre-trained deep learning convolutional neural network model (such as ResNet, VGG series model) and processing it through the multi-layer structure of the model: Convolutional layer: extracting local features of the image (such as product edges, surface texture, defect contours) by sliding calculation through multiple convolutional kernels; Activation function layer: using ReLU (Rectified Linear Unit) activation function to introduce non-linear relationship and enhance the model's expression ability for complex features; Pooling layer: reducing feature map dimension and computation amount while preserving key feature information through maximum pooling or average pooling operation, and finally generating multi-level feature maps (different levels correspond to different levels of abstraction, shallow layer for edge features, deep layer for semantic features); High-level semantic feature extraction: performing feature fusion processing on the multi-level feature maps, compressing high-dimensional feature maps into low-dimensional vectors through the fully connected layer or global pooling layer (such as global average pooling) of the CNN model, extracting high-level semantic features that can reflect the core information of product quality, and generating an initial feature vector (such as 1024-dimensional, 2048-dimensional vector); Feature vector optimization: dimension reduction (using principal component analysis PCA or linear discriminant analysis LDA) and standardization of initial feature vectors to remove redundant features and reduce vector dimension, finally obtaining product quality feature vectors with unified dimension and clear features, and outputting to the multi-modal data fusion unit; Data standardization unit: The core function of the data standardization unit is to perform time series alignment and normalization on the equipment operation state data and material flow data transmitted by the data acquisition module, and generate standardized equipment state data. The data standardization unit is used to perform time series alignment and normalization on the operation state data and flow data, and generate standardized equipment state data, including: Receiving operation state data and flow data from the data acquisition module, wherein the operation state data includes device vibration frequency and amplitude data, and the flow data includes material RFID tag data; Timestamp analysis of operation state data and flow data to extract original time information of each data point; Based on the synchronization time reference provided by the central synchronization controller, the operation state data and flow data after timestamp analysis are processed for time series alignment. The data is resampled to a unified time sequence through linear interpolation method to ensure that all data points have consistent time intervals; The operation state data and flow data after time series alignment are normalized. The minimum-maximum normalization algorithm is used to map the data values to the range of [0, 1] to generate normalized data; Combining the normalized data into standardized equipment state data and outputting to the multi-modal data fusion unit; The specific process and functions are as follows: Data reception and analysis: receiving two types of data transmitted by the data acquisition module: Device operation state data: including device vibration frequency (Hz), vibration amplitude (mm); Material flow data: including material RFID tag information (such as material number, flow node, timestamp); Simultaneously analyze the original timestamps of the two types of data to extract the collection time information of each data point (based on the central synchronization controller time reference of the data acquisition module); Time series alignment: using the synchronization time reference provided by the central synchronization controller of the data acquisition module as the standard, the operation state data and flow data after analysis are processed for time series alignment: For data points with unmatched timestamps (such as vibration sensor collection frequency of 10Hz and RFID reader collection frequency of 5Hz), linear interpolation method is used to resample the data to a unified time sequence (such as every 200ms a data point); Ensure that all data points have consistent time intervals, eliminate time misalignment caused by differences in collection frequency, and realize the data correlation of "device state-material flow under the same timestamp"; Data normalization processing: using the Min-Max Scaling algorithm, the running state data and flow data after time alignment are normalized, and the calculation formula is: , wherein, is the normalized device running state data (such as vibration frequency, vibration amplitude) or material flow data (RFID tag associated data mapped to numerical form), with a value range of [0, 1]; is the original data value of the device running state data or material flow data (mapped to numerical form) (such as the original value of the vibration frequency is 50Hz, and the original value of the vibration amplitude is 0.1mm); is the minimum value in the same type of data set (such as the minimum frequency value in all vibration frequency data, the minimum amplitude value in all vibration amplitude data); is the maximum value in the same type of data set (such as the maximum frequency value in all vibration frequency data, the maximum amplitude value in all vibration amplitude data); Through the formula, the data value is uniformly mapped to the range of [0, 1], eliminating the dimensional differences of different data types (such as the order of magnitude difference between vibration frequency "Hz" and vibration amplitude "mm"), and avoiding the influence of different data magnitudes on subsequent fusion effect; Standardized data generation: combine the normalized device running state data and material flow data in timestamp order to generate structured standardized device state data (such as "timestamp T1: vibration frequency 0.6 (normalized), vibration amplitude 0.4 (normalized), RFID tag M001 (mapped to numerical normalized result)"), and output to the multi-modal data fusion unit; Multi-modal data fusion unit: The core function of the multi-modal data fusion unit is to use a fusion algorithm based on an attention mechanism to fuse the product quality feature vector and the standardized device state data to generate a production line fusion feature set. The multi-modal data fusion unit uses a fusion algorithm based on an attention mechanism to fuse the product quality feature vector and the standardized device state data to generate a production line fusion feature set, including: Receive the product quality feature vector from the image processing unit and the standardized device state data from the data standardization unit; Input the product quality feature vector and the standardized device state data into the linear mapping layer respectively to generate the corresponding query vector, key vector and value vector; Calculate the dot product similarity between the query vector and the key vector, and perform softmax normalization processing on the similarity to generate the attention weight; The attention weight is used to perform a weighted sum operation on the value vector to generate a weighted context feature vector; The weighted context feature vector is spliced with the product quality feature vector to generate a preliminary fusion feature vector; The preliminary fusion feature vector is processed by a full connection layer and an activation function to generate a final production line fusion feature set; The specific process and functions are as follows: Data reception: synchronously receive product quality feature vectors (such as 256-dimensional vectors) from the image processing unit and standardized equipment state data (time-series multi-dimensional data, such as containing 3 feature dimensions every 200 ms) from the data standardization unit; Linear mapping processing: input the two types of data into independent linear mapping layers respectively, and generate three corresponding vectors through linear transformation: For the product quality feature vector, generate a query vector (Query) and a key vector (Key); For the standardized equipment state data, generate a value vector (Value); The purpose of linear mapping is to convert features of different dimensions and different semantics into the same feature space, laying the foundation for subsequent similarity calculation; Attention weight calculation: calculate the dot product similarity between the query vector and the key vector (measure the degree of association between the two types of data, such as the matching degree of “a product quality feature and a device state at a certain time”), and perform softmax normalization processing on the similarity result, the calculation formula is: wherein, is the attention weight matrix calculated by the query vector and the key vector, each element in the matrix corresponds to an association weight of “product quality feature-equipment state feature”, with a value range of [0, 1]; is the query vector obtained by converting the product quality feature vector through the linear mapping layer, with the same dimension as the key vector, used to represent the “query dimension” of the product quality feature; is the key vector obtained by converting the product quality feature vector through the linear mapping layer, used to represent the “key dimension” of the product quality feature, and used to calculate the association degree together with the query vector; is the transpose matrix of the key vector ; is the dimension of the key vector (such as 64 dimensions, 128 dimensions), this parameter is introduced to scale the dot product result of the large-dimensional vector, to avoid the weight being too concentrated or the gradient disappearing after softmax normalization due to the large numerical value; is the softmax normalization function, The normalized attention weight is generated, and the greater the weight value, the higher the correlation degree of the corresponding data (for example, the correlation weight of abnormal device vibration at a certain time and product surface defect is higher); Context feature generation: using attention weight to perform weighted sum operation on value vector (standardized device state data mapping), the calculation formula is (wherein is the value vector), giving higher weight to high correlation data features and lower weight to low correlation features, generating a weighted context feature vector reflecting the "device state-product quality" correlation relationship; Feature splicing and optimization: splice the weighted context feature vector and the product quality feature vector (such as 256-dimensional product vector + 64-dimensional context vector = 320-dimensional preliminary vector), generate a preliminary fusion feature vector; then input the preliminary vector into the full connection layer, and perform nonlinear transformation through ReLU and other activation functions, further integrate feature information, and finally generate a production line fusion feature set (time-sequenced fusion features, such as 320-dimensional fusion vector every 200ms), output to the dynamic performance evaluation module.

[0020] In this embodiment, the dynamic performance evaluation module is used to input the production line fusion feature set into the trained performance evaluation model to calculate key performance indicators including at least device comprehensive efficiency OEE, first pass yield FPY and production line balance rate, including: Receiving the production line fusion feature set from the multi-modal data fusion unit; Performing time window division processing on the production line fusion feature set to generate a feature sequence of fixed time length; Input the feature sequence into the trained performance evaluation model, the performance evaluation model is based on a deep neural network architecture, receives the feature sequence through the input layer, performs nonlinear transformation and feature extraction through multiple hidden layers, and outputs a preliminary performance indicator vector. The preliminary performance indicator vector is subjected to inverse normalization processing, the model output value is mapped to the actual physical range, and a standardized performance indicator value is generated; Based on the standardized performance indicator value, the numerical values of the device comprehensive efficiency OEE, the first pass yield FPY and the production line balance rate are analyzed; Output the analyzed key performance indicators to the optimization strategy generation module; Further, the dynamic performance evaluation module is a key hub connecting the data processing and fusion module and the optimization strategy generation module, and the core function is to convert the production line feature data after multi-modal fusion into quantifiable and interpretable key performance indicators; it receives the production line fusion feature set output by the data processing and fusion module, calculates the three core indicators of equipment overall efficiency (OEE), first pass yield (FPY), and production line balance rate in real time through the trained deep neural network performance evaluation model, provides accurate performance "diagnosis results" for the optimization strategy generation module, and is the key decision basis for the system to realize the "evaluation-optimization" closed loop; the following is a detailed description of the module from the whole to the details: I. Overall function overview: The dynamic performance evaluation module is connected to the data processing and fusion module, and the overall function is divided into three core links of "feature preprocessing-model calculation-index analysis": Feature preprocessing: time window division is performed on the received production line fusion feature set, and the continuous feature data is cut into feature sequences of fixed time length, ensuring that the data input into the model has time sequence correlation and uniformity; Model calculation: input the feature sequence into the trained performance evaluation model (based on deep neural network architecture), and output the preliminary performance indicator vector after normalization through the nonlinear transformation and feature extraction of the model; Index analysis: perform inverse normalization on the preliminary performance indicator vector to map the dimensionless value output by the model back to the actual physical range, generate standardized performance indicator values, and then analyze the specific values of equipment overall efficiency (OEE), first pass yield (FPY), and production line balance rate, and finally output to the optimization strategy generation module to provide data support for the formulation of subsequent optimization action strategies; II. Submodule composition and function: (1) Feature sequence generation unit: The core function of the feature sequence generation unit is to divide the production line fusion feature set into time windows, generate feature sequences that adapt to the input format of the performance evaluation model, and the specific process and function are as follows: Feature set reception: real-time reception of the production line fusion feature set transmitted by the data processing and fusion module; the feature set is time-series data, containing multi-dimensional fusion features (such as 320-dimensional vectors, covering product quality, equipment status, and material flow correlation information) corresponding to each timestamp (such as 200ms / each); Time window parameter configuration: based on the production line beat and performance evaluation requirements, preset a fixed time window length (such as 10s, 30s, which should be consistent with the window parameter during model training), and the window sliding step and feature set timestamp interval are kept synchronous (such as 200ms / step), ensuring that there is no data omission or repetition; Feature sequence cutting: according to the preset time window, the continuous production line fusion feature set is cut in a sliding manner; for example, when the window length is 10s and the timestamp interval is 200ms, each window contains 50 continuous timestamp fusion feature vectors, which are combined to form a feature sequence with a dimension of "50x320"; Sequence verification and output: the integrity of the cut feature sequence is verified (such as checking whether the number of feature vectors in the window meets the standard and whether there are missing values), and after the verification is passed, the feature sequence is output to the performance evaluation model calculation unit, and if the verification fails, the data processing and fusion module is fed back to retransmit the fusion feature set of the corresponding period; (2) Performance evaluation model calculation unit: The performance evaluation model calculation unit is the core of the module, which converts the feature sequence into a preliminary performance index vector through a trained deep neural network model. The specific process and functions are as follows: Model loading and initialization: load the pre-trained performance evaluation model when starting, and the model is based on a deep neural network architecture (such as multi-layer perceptron MLP, long short-term memory network LSTM, which needs to be consistent with the training stage architecture). The loaded parameters include the input layer dimension (matched with the feature sequence dimension, such as "50x320"), the number of hidden layers / neurons, and the output layer dimension (fixed at 3, corresponding to OEE, FPY, and line balance rate); Feature sequence input: receive the compliant feature sequence output by the feature sequence generation unit, and flatten it into a vector format that the model input layer can accept (such as flattening the "50x320" two-dimensional sequence into a "16000" dimensional one-dimensional input vector); Model forward propagation: the input vector is processed by each layer of the model in turn: Input layer: receive the flattened feature sequence vector and pass it to the first hidden layer; Hidden layer: perform linear transformation wherein, is the intermediate result of the hidden layer linear transformation, is the weight matrix of the hidden layer (used to quantify the influence of each dimension of the input vector on the output of the hidden layer), is the input vector of the hidden layer (i.e. the output result of the previous layer), is the bias term of the hidden layer (used to adjust the linear transformation result to avoid fitting bias caused by relying only on the input vector) and the activation function (such as ReLU, Siqmoid) performs nonlinear transformation, gradually extracting deep features related to performance indicators; Output layer: perform linear transformation to output a preliminary performance index vector with a dimension of 3, and each element in the vector is a normalized value in the range of [0, 1], which corresponds to the preliminary prediction results of OEE, FPY, and line balance rate; Primary indicator output: The preliminary performance indicator vector generated by the output layer is transmitted to the indicator de-normalization and analysis unit, and the model calculation timestamp is recorded to ensure time consistency with the subsequent indicator analysis results; (Three) Indicator de-normalization and analysis unit: The core function of the indicator de-normalization and analysis unit is to convert the normalized indicator values output by the model into actual physical performance indicators and analyze the specific values. The specific process and functions are as follows: Primary indicator reception: Receive the preliminary performance indicator vector output by the performance evaluation model calculation unit (such as [0.82, 0.95, 0.88]), and load the "indicator-maximum value mapping table" saved during model training (record the historical minimum values of OEE, FPY, and line balance rate and historical maximum values ); De-normalization processing: Use the de-normalization formula to map each normalized value in the preliminary performance indicator vector back to the actual physical range, the formula is: , where is the de-normalized normalized performance indicator value (with actual physical meaning, such as OEE=82%, FPY=95%), is an element in the preliminary performance indicator vector (normalized value, range [0,1], such as OEE ), is the historical minimum value of the performance indicator in the model training data set (such as OEE , FPY ), is the historical maximum value of the performance indicator in the model training data set (such as OEE , FPY ); For example, if OEE , , , then ; Indicator analysis: Based on the dimension order of the normalized performance indicator value (consistent with the model output layer definition: the first dimension corresponds to OEE, the second dimension corresponds to FPY, and the third dimension corresponds to line balance rate), extract the specific values of the three indicators, such as the analysis result "OEE=92.8%, FPY=95.2%, line balance rate=88.5%"; Indicator output: The analyzed key performance indicators (including specific values and calculation timestamps) are output to the optimization strategy generation module in real time, and stored in the local indicator log library for subsequent performance trend analysis and historical data tracing; Three, key technical principles: (One) Time window division principle: The core purpose of time window division is to ensure that the feature data input to the model has "time series correlation." The calculation of production line efficiency indicators (such as OEE and production line balance rate) requires data on equipment operation, product quality, and material flow within a certain time period. Feature data with a single timestamp cannot reflect the dynamic changes in efficiency. By dividing the data into fixed time windows (such as 10 seconds), the continuous production line fusion feature set is converted into a feature sequence containing multiple time series nodes. This not only preserves the time dimension information of the data (such as the correlation between equipment vibration trends and product defect rates within a certain 10 seconds), but also unifies the data format of the input model (fixed length and fixed dimension), avoiding model calculation errors caused by inconsistent data length. At the same time, it matches the requirement of "periodic performance evaluation" in actual industrial scenarios (for example, updating performance indicators every 10 seconds). (2) Principles of Deep Neural Network Performance Evaluation: The performance evaluation model uses a deep neural network architecture. The core principle is to achieve a complex mapping of "features-indicators" through multi-layer nonlinear transformations: Input layer adaptation: The flattened feature sequence vector is used as input to ensure that the model can receive multi-dimensional, time-series fusion features (covering product, equipment, and material related information); Hidden layer feature extraction: Through the weight matrix of multiple hidden layers ( ) and activation functions (such as ReLU), gradually extracting abstract features that are strongly correlated with performance indicators from the fusion features; for example, extracting the associated features of "equipment failure affecting production efficiency" from "equipment vibration frequency features + product defect features", and extracting the features of "production line bottleneck process" from "material flow speed features + process time-consuming features"; the linear transformation of the hidden layer follows (in, is the intermediate result of the hidden layer linear transformation, is the weight matrix of the hidden layer, is the input vector of the hidden layer, is the bias term of the hidden layer), the activation function is (ReLU function, where is the output value of the activation function, Introducing nonlinearity into the intermediate result of the hidden layer linear transformation enables the model to fit complex feature-index mapping relationships; Output layer mapping: The output layer outputs three normalized preliminary performance index values ​​through linear transformation, achieving accurate conversion from "multi-dimensional fusion features to single index value"; Model training basis: the model training stage takes the "line fusion feature sequence" as the input and the "actual measured OEE / FPY / line balance rate" as the label. The actual label is first mapped to the [0, 1] range through "min-max normalization", and then the weight matrix ( ) and the bias term ( ) are iteratively optimized through the back propagation algorithm (such as gradient descent) to minimize the error between the model prediction value and the normalized label, ensuring that the trained model has high-precision performance prediction capability; (Three) Denormalization principle: In the model training stage, to avoid the influence of the order of magnitude difference of different indicators on the training effect (such as OEE taking the value range of 60%-100% and line balance rate taking the value range of 50%-90%), the actual indicator value is mapped to the [0, 1] range through "min-max normalization". Therefore, the preliminary performance indicator vector output by the model is a dimensionless value in the [0, 1] interval, which cannot directly reflect the actual performance. Denormalization restores the normalized preliminary indicator value to an indicator value with actual physical meaning (such as restored to ) through inverse calculation (formula: ), ensuring that the output OEE, FPY, and line balance rate meet the conventional quantification standards in the industrial scene, facilitating the understanding and use of the optimization strategy generation module.

[0021] In this embodiment, the optimization strategy generation module is built-in with a reinforcement learning agent. The reinforcement learning agent takes the key performance indicators as the state input of the reward function, learns the output optimization action strategy through interaction with the environment, and the optimization action strategy includes device parameter adjustment instructions, production rhythm control instructions, or re-planning instructions for the unloading mechanical arm path, including: Receiving the key performance indicators from the dynamic performance evaluation module; Inputting the key performance indicators as the current environment state into the state space of the reinforcement learning agent; The reinforcement learning agent calculates the action value function through the policy network based on the current environment state to generate a candidate action set; Balancing exploration and utilization of the candidate action set, and selecting the optimal action using the ε-greedy strategy; Mapping the optimal action to the original action strategy, including device parameter adjustment amount, production rhythm adjustment value, or mechanical arm path point sequence; Checking the compliance of the original action strategy to ensure that it meets the production line safety constraints and operating range; Encoding the compliant original action strategy into specific device parameter adjustment instructions, production rhythm control instructions, or re-planning instructions for the unloading mechanical arm path; Outputting the optimization action strategy to the control execution module; Further, the optimization strategy generation module is connected to the dynamic performance evaluation module, and the core function is to receive three types of key performance indicators, including equipment overall efficiency (OEE), first pass yield (FPY), and line balance rate, and convert them into environment state inputs of the reinforcement learning agent. The agent learns through interaction with the production line environment, calculates and generates a candidate set of optimization actions, and after exploration and utilization balance and compliance check, maps the effective actions to specific device parameter adjustment instructions, production cycle control instructions, or re-planning instructions for the unloading robot path, and finally outputs to the control execution module to promote the line performance to the optimal direction. At the same time, the module will continuously iterate the agent decision logic based on the feedback data of the production line after control execution, to ensure the adaptability and effectiveness of the strategy, and the specific operation steps are as follows: Key performance indicator reception: Real-time reception of key performance indicator data output by the dynamic performance evaluation module, including specific values of equipment overall efficiency OEE (such as 92.8%), specific values of first pass yield FPY (such as 95.2%), and specific values of line balance rate (such as 88.5%), and simultaneously receiving the calculation time stamp corresponding to the indicators, to ensure that the state data matches the actual running period of the production line; Indicator standardization processing: Using the normalization algorithm corresponding to the denormalization process of the dynamic performance evaluation module, the actual performance indicator values received are mapped to the [0, 1] range suitable for the state space of the reinforcement learning agent, and the calculation formula is (Wherein, is the standardized state value of the th key performance indicator, corresponding to OEE, FPY, and line balance rate, respectively; is the actual received value of the th indicator; is the historical minimum effective value of the th indicator, such as OEE ; is the theoretical maximum effective value of the th indicator, such as OEE ), to eliminate the interference of different indicator magnitudes on agent decision-making; State vector construction: Combine the three standardized state values in the fixed order of “OEE FPY line balance rate” to generate a current environment state vector with a dimension of 3, for example , and add a time stamp label to the state vector, and output it to the reinforcement learning agent decision unit; Policy network loading and initialization: when the module starts, load the pre-trained reinforcement learning policy network model, model parameters include input layer dimension (fixed as 3, matching the state vector dimension), the number of hidden layers (such as 2 layers), the number of hidden layer neurons (such as 64 per layer), the output layer dimension (matching the number of candidate action set, such as 6 candidate actions corresponding to the output layer dimension of 6); At the same time, initialize the exploration rate (such as 0.1, used to balance exploration and utilization), discount factor (such as 0.9, used to measure the weight of future rewards) and other decision parameters; Action value function calculation: input the state into the policy network and the state vector output by the analysis unit , through the linear transformation of the hidden layer and the ReLU activation function processing, calculate the value of each candidate action, and output the action value vector , the calculation formula is ReLU ReLU (wherein, is the value of the state executing action ; , are the weight matrix and bias term of the first layer hidden layer respectively; , are the weight matrix and bias term of the second layer hidden layer respectively; ReLU is a rectified linear unit activation function, which is used to introduce nonlinear decision-making ability - Greedy policy selection: based on the action value vector and the exploration rate , execute the optimal action selection: when the randomly generated probability value is greater than (such as ), select the action with the maximum action value (that is, use the known optimal strategy); when , randomly select an action in the candidate action set (that is, explore new potential optimization actions), and the selection probability formula is , wherein, is the selection probability of action ; is the total number of candidate action sets; is the action with the maximum action value, which ensures that the existing experience is utilized while avoiding falling into local optimum; Optimal action mapping: map the optimal action output by the reinforcement learning intelligent agent decision unit to specific original action policy parameters: If the optimal action is "device parameter adjustment", it is mapped to the device ID (e.g., "Lathe #3"), parameter type (e.g., "Spindle Speed"), and adjustment amount (e.g., "+50 r / min"); If the optimal action is "production rhythm control", it is mapped to the rhythm adjustment object (e.g., "Assembly Process") and adjustment value (e.g., "-2 s", i.e., the rhythm is shortened from 20 s to 18 s); If the optimal action is "replanning of unloading robot path", it is mapped to the robot ID (e.g., "Robot #2") and new path point sequence (e.g., "(X1, Y1, Z1)→(X2, Y2, Z2)→(X3, Y3, Z3)", a total of 5 path points); Safety constraint compliance check: Based on the pre-set safety constraint threshold of the production line, the original action strategy is checked for compliance: Device parameter adjustment must satisfy "parameter value ∈ [safe lower limit, safe upper limit]", such as the safe range of lathe spindle speed is 500-3000 r / min, if the adjustment amount is "+50 r / min" and the speed is 3020 r / min, it is determined as non-compliant; Production rhythm control must satisfy "adjusted rhythm ≥ minimum production time", such as the minimum production time of assembly process is 15 s, if the adjustment value is "-6 s" and the rhythm is 14 s, it is determined as non-compliant; Unloading robot path must satisfy "all path points ∈ robot working space", if a path point (Z3) exceeds the maximum lifting height of the robot, it is determined as non-compliant; Strategy modification and confirmation: For non-compliant original action strategies, modify them according to the principle of "close to optimal value and comply with constraints", such as adjusting the spindle speed from "+50 r / min" to "+30 r / min" (making the adjusted speed 3000 r / min, complying with the upper limit constraint); After modification, check again until compliance, generate the confirmed original action strategy; Optimization instruction structured coding: Encode the confirmed original action strategy into a specific instruction format that meets the parsing requirements of the control execution module: Device parameter adjustment instruction: The encoding format is "device ID | parameter type | adjusted value | effective time", such as "Lathe #3 | Spindle Speed | 3000 r / min | Immediately"; Production rhythm control instruction: The encoding format is "process ID | pre-adjustment rhythm | post-adjustment rhythm | switching opportunity", such as "Assembly Process #2 | 20 s | 18 s | Start of next batch of materials"; The blanking mechanical arm path re-planning instruction: the coding format is "mechanical arm ID | path point number | path point 1 coordinate | path point 2 coordinate |... | path point n coordinate", such as "mechanical arm #2 | 5 | (100, 200, 50) | (120, 220, 50) | (140, 240, 50) | (160, 220, 50) | (180, 200, 50)"; Instruction verification and output: the coded optimized action strategy instruction is subjected to integrity verification (such as checking whether the instruction field is missing or the coordinate format is correct), and after the verification is passed, the instruction and the corresponding timestamp are packaged and output in real time to the control execution module through industrial Ethernet; if the verification fails, it is fed back to the action strategy mapping and compliance checking unit for recoding.

[0022] In this embodiment, the control execution module is connected to the optimized strategy generation module and the programmable logic controller (PLC) on the production line, for converting the optimized action strategy into control instructions conforming to the industrial communication protocol, and issuing them to the corresponding execution unit, including: receiving the optimized action strategy from the optimized strategy generation module, the optimized action strategy including device parameter adjustment instructions, production rhythm control instructions or blanking mechanical arm path re-planning instructions; analyzing and processing the received optimized action strategy to extract the action type and specific parameter value; According to the extracted action type, select the corresponding industrial communication protocol template, including Modbus, Profinet or EtherNet / IP; Fill the extracted specific parameter value into the data field of the selected industrial communication protocol template to generate an original control instruction frame; Calculate and package the original control instruction frame to generate a standard control instruction conforming to the industrial communication protocol; issue the standard control instruction to the programmable logic controller (PLC) on the production line through industrial Ethernet; After the programmable logic controller (PLC) analyzes and executes the standard control instruction, the control execution module acquires the executed production line data through the data acquisition module to verify the instruction execution effect and form a closed-loop optimization control system; Further, the control execution module is connected to the optimization strategy generation module at one end and interfaces with the PLC on the production line at the other end, and the core functions cover four links: first, strategy receiving and analysis, receiving the optimization action strategy and extracting the action type (device parameter adjustment, production rhythm control, and re-planning of the unloading mechanical arm path) and specific parameter values; second, industrial protocol adaptation, selecting a matching industrial communication protocol template (Modbus, Profinet, or EtherNet / IP) according to the action type; third, instruction generation and packaging, filling the parameter values into the protocol template, generating control instruction frames conforming to the industrial standard and completing the verification; fourth, instruction issuing and closed-loop verification, issuing the instruction to the corresponding PLC for execution, and simultaneously acquiring the production line data after execution through the data acquisition module to verify the instruction effect, finally forming a closed-loop optimization control system of “strategy-instruction-execution-feedback” to ensure accurate landing and effectiveness of the optimization action; the specific operation steps are as follows: Optimization strategy receiving: real-time receiving of the optimization action strategy from the optimization strategy generation module through industrial Ethernet, the strategy format being a structured instruction (such as “device ID|parameter type|adjusted value|effective time” “process ID|pre-adjustment rhythm|post-adjustment rhythm|switching time” “mechanical arm ID|number of path points|path point coordinate sequence”), and simultaneously receiving the timestamp corresponding to the strategy to ensure matching with the production line operation period; Action type identification: keyword analysis of the received optimization action strategy to identify the action type: If the strategy contains “device ID” and “parameter type” (such as “main shaft speed” and “feed amount”), it is determined as a “device parameter adjustment instruction”; If the strategy contains “process ID” and “post-adjustment rhythm”, it is determined as a “production rhythm control instruction”; If the strategy contains “mechanical arm ID” and “path point coordinates”, it is determined as a “re-planning of the unloading mechanical arm path instruction”; Specific parameter extraction: extracting core parameter values for different action types: Device parameter adjustment instruction: extracting “device ID” (such as “lathe #3”), “post-adjustment parameter value” (such as “3000 r / min”), and “effective time” (such as “immediately” and “at the start of the next batch of materials”); Production rhythm control instruction: extracting “process ID” (such as “assembly process #2”), “post-adjustment rhythm value” (such as “18 s”), and “switching time” (such as “after the current batch is completed”); Unloading robot path re-planning instruction: extract "robot ID" (such as "robot #2"), "number of path points" (such as "5"), "path point coordinate sequence" (such as "(100, 200, 50), (120, 220, 50), (140, 240, 50), (160, 220, 50), (180, 200, 50)"). Parameter format verification: verify whether the extracted parameter value format meets the requirements of industrial equipment, such as equipment parameter value should contain units ("r / min", "mm / s"), path point coordinates should be integers or have 1 decimal place, and beat value should be positive. If the format is incorrect (such as parameter value without units, negative coordinates), feedback to the optimization strategy generation module to resend, ensuring parameter validity. Protocol template library loading: when the module is initialized, load the preset industrial communication protocol template library, including standard templates of three types of protocols: Modbus protocol template: suitable for parameter adjustment of small and medium-sized equipment (such as single machine tool, small conveyor belt), template contains fixed format of address field, function code (such as "06H" represents writing single register), data field, and verification field; Profinet protocol template: suitable for control of high-speed synchronous equipment (such as multi-axis robot, continuous production line), template supports real-time data transmission (RT) and non-real-time data transmission (IRT), contains field definition of device name, data identifier, and data length; EtherNet / IP protocol template: suitable for communication of cross-brand equipment (such as PLC and sensors of different manufacturers), template is based on CIP protocol specification, contains structure framework of message header, data segment, and message tail; Protocol automatic matching: according to the action type identified by the strategy receiving and analyzing unit, automatically select the corresponding industrial communication protocol: Device parameter adjustment instruction (such as lathe spindle speed adjustment): if the device is single machine independent control, match Modbus protocol; if the device is production line synchronous control node, match Profinet protocol; Production beat control instruction (such as whole assembly line beat adjustment): due to the need to synchronize multiple process equipment, match Profinet protocol (high real-time requirement) or EtherNet / IP protocol (high cross-device compatibility requirement); Unloading robot path re-planning instruction (such as multi-axis robot path update): due to the need for high-speed transmission of path point coordinate sequence, match Profinet protocol (supports real-time path data delivery); Protocol parameter configuration: configure communication parameters for matched protocols: Modbus protocol: configure slave address (e.g. "1" for Modbus slave address of lathe #3), baud rate (e.g. "115200bps"), data bits ("8 bits"), stop bits ("1 bit"), parity mode ("even parity"); Profinet protocol: configure PROFINET device name of the device (e.g. "Robot_2" for robot arm #2), real-time channel priority ("high priority" for path data transmission); EtherNet / IP protocol: configure IP address of the device (e.g. "192.168.1.102" for PLC of assembly process #2), CIP object identifier (e.g. "0x6B" for motion control object); Raw instruction frame filling: fill the specific parameter values extracted by the strategy receiving and parsing unit into the protocol template data field matched by the industrial protocol adaptation unit: Modbus protocol (device parameter adjustment): if the spindle speed of lathe #3 is adjusted to 3000 r / min, and the Modbus register address corresponding to the spindle speed is "40001", then fill "40001" (address), "06H" (function code), and "0BB8H" (hexadecimal value of 3000) into the template data field; Profinet protocol (robot arm path re-planning): fill the robot arm ID "Robot_2", the number of path points "5", and the X / Y / Z coordinates of each path point (e.g. "100,200,50" corresponds to hexadecimal "64,C8,32") into the template data field; EtherNet / IP protocol (production beat control): fill the process ID "Assembly_2", the adjusted beat "18s" (corresponding to hexadecimal "12"), and the switching timing "01" (indicating after the current batch) into the template data segment; Check code calculation: calculate the check code according to the protocol requirements to ensure that the instruction transmission process is not tampered with or lost: Modbus protocol: use CRC-16 check algorithm to calculate the binary data check value of the address field, function code, and data field, and fill it into the template check field; Profinet protocol: use CRC-32 check algorithm to calculate the check value of the entire message (device name + data identifier + data field) and append it at the end of the message; EtherNet / IP protocol: based on the TCP / IP protocol stack check mechanism, calculate the IP header checksum and TCP checksum to ensure the integrity of the message; Instruction frame encapsulation: encapsulate the protocol template with filled data field and check field into a standard control instruction frame according to the protocol standard format: Modbus instruction frame format: [address field (1 byte) + function code (1 byte) + data field (N bytes) + CRC check code (2 bytes)]; Profinet instruction frame format: [message header (8 bytes, including version, length) + device name (16 bytes) + data identifier (2 bytes) + data field (M bytes) + CRC-32 check code (4 bytes)]; EtherNet / IP instruction frame format: [TCP header (20 bytes) + IP header (20 bytes) + CIP message header (8 bytes) + data segment (K bytes) + message tail (4 bytes)]; Instruction validity verification: integrity and compliance verification is performed on the encapsulated standard control instruction frame, such as checking whether the frame length meets the protocol requirements (Modbus instruction frame is at least 8 bytes), whether the data field value is within the PLC recognizable range (such as Profinet path coordinate value does not exceed the maximum storage value of PLC register), and if the verification is passed, it enters the instruction issuing link, and if it fails, it returns to the original instruction frame filling link for reprocessing; Instruction directional issuing: through industrial Ethernet, the standard control instruction frame that passes the verification is directed to the corresponding production line PLC: Device parameter adjustment instruction: issued to the local PLC corresponding to the target device (such as lathe #3); Production rhythm control instruction: issued to the production line main PLC (such as assembly line main controller), and synchronized to each process PLC by the main PLC; Blanking mechanical arm path re-planning instruction: issued to the mechanical arm special control PLC (such as the motion controller of mechanical arm #2); Execution state receiving: after the PLC receives the standard control instruction frame, it analyzes and executes the instruction, and at the same time feeds back the execution state ("execution success" "execution failure" "execution in progress") to the control execution module. The feedback information includes PLCID, instruction timestamp, and execution result code (such as "00H" for success and "01H" for parameter exceeding PLC control range); Closed-loop effect verification: if the PLC feedbacks "execution success", the control execution module acquires the production line data after the instruction execution through the data acquisition module (high-speed industrial camera array, vibration sensor, RFID reader): After adjusting the device parameters: acquire the device vibration frequency / amplitude data (verify whether the device is stable after adjusting the main shaft speed) and product image data (verify whether the product quality meets the standard after adjusting the parameters); After production rhythm control: acquire RFID material flow data (verify whether the material flow is smooth after rhythm adjustment) and production line balance rate related data (verify the rhythm optimization effect); After the mechanical arm path is re-planned: obtain the mechanical arm motion trajectory data (verify whether the path is executed according to the instruction), and the material grabbing success rate data (verify the path optimization effect); Archive the feedback results: package the instruction issuing record (issuing time, instruction content, target PLC), execution state feedback, and closed-loop effect verification data, and store them into the local database of the module, and feed back the verification results (such as "OEE is improved by 2% after the device parameter is adjusted", "the grabbing success rate is improved by 5% after the mechanical arm path is optimized") to the optimization strategy generation module, so as to provide a basis for subsequent strategy iteration optimization, and complete the closed-loop control.

[0023] Although the embodiments of the present application have been shown and described, it can be understood by those of ordinary skill in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A machine vision production line performance evaluation and optimization management system, characterized by: include: The data acquisition module includes a high-speed industrial camera array, vibration sensors, and RFID readers deployed on the production line to simultaneously collect product image data, equipment operating status data, and material flow data; The data processing and fusion module is connected to the data acquisition module and includes: The image processing unit extracts features from image data based on a deep learning convolutional neural network model to obtain product quality feature vectors; Data standardization unit, used to perform time sequence alignment and normalization processing on operating status data and flow data to generate standardized equipment status data; The multimodal data fusion unit uses an attention-based fusion algorithm to fuse product quality feature vectors with standardized equipment status data to generate a production line fusion feature set; The dynamic performance evaluation module is connected to the data processing and fusion module and is used to input the production line fusion feature set into the trained performance evaluation model to calculate key performance indicators in real time, including at least overall equipment efficiency (OEE), first-pass yield (FPY), and production line balance rate; The optimization strategy generation module is connected to the dynamic performance evaluation module and has a built-in reinforcement learning agent. The reinforcement learning agent uses key performance indicators as the state input of the reward function and outputs optimization action strategies through interactive learning with the environment. The optimization action strategies include equipment parameter adjustment instructions, production rhythm control instructions, or unloading robot arm path replanning instructions. The control execution module is connected to the optimization strategy generation module and the programmable logic controller (PLC) on the production line. It is used to convert the optimization action strategy into control instructions that comply with the industrial communication protocol and send them to the corresponding execution units to form a closed-loop optimization control system.

2. The machine vision production line efficiency evaluation and optimization management system according to claim 1, characterized in that: In the data acquisition module, a high-speed industrial camera array, a vibration sensor, and an RFID reader collect data through a synchronous triggering mechanism. The synchronous triggering mechanism is managed by a central synchronization controller, which generates a synchronous pulse signal and simultaneously triggers the high-speed industrial camera array to capture multi-angle image data of the product, the vibration sensor to collect equipment vibration frequency and amplitude data, and the RFID reader to read material RFID tag data, ensuring that all collected data has a consistent timestamp. The data acquisition module also packages the collected image data, operating status data, and flow data into time series data packets and transmits them to the data processing and fusion module via industrial Ethernet.

3. The machine vision production line efficiency evaluation and optimization management system according to claim 1, characterized in that: The image processing unit extracts features from the image data based on a deep learning convolutional neural network model to obtain a product quality feature vector, including: Preprocess the collected image data, including image denoising, size adjustment and pixel value normalization, to generate standardized image data; The standardized image data is input into a pre-trained deep learning convolutional neural network model, and forward propagation is performed through multiple convolutional layers, activation function layers, and pooling layers to generate a multi-level feature map; Perform feature fusion processing on the multi-level feature maps, extract high-level semantic features through the fully connected layer or global pooling layer in the convolutional neural network, and generate the initial feature vector; The initial feature vector is subjected to dimensionality reduction and normalization to obtain the final feature vector that characterizes product quality.

4. The machine vision production line efficiency evaluation and optimization management system according to claim 1, characterized in that: The data standardization unit is used to perform time sequence alignment and normalization processing on the operating status data and the flow data to generate standardized device status data, including: Receive operating status data and flow data from the data acquisition module, where the operating status data includes equipment vibration frequency and amplitude data, and the flow data includes material RFID tag data; Performing timestamp analysis on the operating status data and flow data to extract the original time information of each data point; Based on the synchronization time reference provided by the central synchronization controller, the operating status data and flow data after timestamp analysis are aligned, and the data are resampled to a unified time series through linear interpolation to ensure that all data points have consistent time intervals. Normalize the operating status data and flow data after time series alignment. Use the minimum-maximum normalization algorithm to map the data values ​​to the range of [0, 1] to generate normalized data. The normalized data are combined into standardized device status data and output to the multimodal data fusion unit.

5. The machine vision production line efficiency evaluation and optimization management system according to claim 1, characterized in that: The multimodal data fusion unit uses a fusion algorithm based on the attention mechanism to fuse the product quality feature vector with the standardized equipment status data to generate a production line fusion feature set, including: receiving the product quality feature vector from the image processing unit and the standardized equipment status data from the data standardization unit; The product quality feature vector and the standardized equipment status data are input into the linear mapping layer respectively to generate the corresponding query vector, key vector and value vector; Calculate the dot product similarity between the query vector and the key vector, and perform softmax normalization on the similarity to generate attention weights; Use the attention weights to perform a weighted sum operation on the value vector to generate a weighted context feature vector; The weighted context feature vector is concatenated with the product quality feature vector to generate a preliminary fusion feature vector; The preliminary fusion feature vector is processed by the fully connected layer and the activation function is transformed to generate the final production line fusion feature set.

6. The machine vision production line efficiency evaluation and optimization management system according to claim 1, characterized in that: The dynamic performance evaluation module is used to input the production line fusion feature set into the trained performance evaluation model and calculate key performance indicators including at least overall equipment efficiency (OEE), first pass yield (FPY), and production line balance rate in real time, including: receiving a production line fusion feature set from a multimodal data fusion unit; The production line fusion feature set is divided into time windows to generate a feature sequence of fixed time length; Input the feature sequence into a trained performance evaluation model. The performance evaluation model is based on a deep neural network architecture. It receives the feature sequence through an input layer, performs nonlinear transformation and feature extraction through multiple hidden layers, and outputs a preliminary performance index vector. The preliminary performance index vector is denormalized and the model output value is mapped to the actual physical range to generate a standardized performance index value. Based on the standardized performance index values, the values ​​of equipment overall efficiency (OEE), first pass yield (FPY) and production line balance rate are analyzed; Output the parsed key performance indicators to the optimization strategy generation module.

7. The machine vision production line efficiency evaluation and optimization management system according to claim 1, characterized in that: The optimization strategy generation module has a built-in reinforcement learning agent. The reinforcement learning agent uses key performance indicators as the state input of the reward function and outputs an optimization action strategy through interactive learning with the environment. The optimization action strategy includes equipment parameter adjustment instructions, production rhythm control instructions, or unloading robot arm path replanning instructions, including: Receive key performance indicators from the dynamic performance evaluation module; Input the key performance indicators into the state space of the reinforcement learning agent as the current environment state; The reinforcement learning agent calculates the action value function based on the current environment state through the policy network and generates a set of candidate actions; The candidate action set is explored and utilized in a balanced manner, and the optimal action is selected using the ε-greedy strategy; Mapping the optimal action to the original action strategy, including the equipment parameter adjustment amount, the production cycle adjustment value or the robot arm path point sequence; Conduct compliance checks on the original action strategy to ensure it complies with production line safety constraints and operating range; Encode the compliant original motion strategy into specific equipment parameter adjustment instructions, production rhythm control instructions, or unloading robot arm path replanning instructions; Output the optimized action strategy to the control execution module.

8. The machine vision production line efficiency evaluation and optimization management system according to claim 1, characterized in that: The control execution module is connected to the optimization strategy generation module and the programmable logic controller (PLC) on the production line, and is used to convert the optimization action strategy into control instructions that comply with the industrial communication protocol and send them to the corresponding execution unit, including: Receive an optimization action strategy from an optimization strategy generation module, wherein the optimization action strategy includes an equipment parameter adjustment instruction, a production rhythm control instruction, or a blanking robot arm path replanning instruction; Parse the received optimization action strategy and extract the action type and specific parameter values; Selecting a corresponding industrial communication protocol template according to the extracted action type, wherein the industrial communication protocol template includes Modbus, Profinet, or EtherNet / IP; Filling the extracted specific parameter values ​​into the data field of the selected industrial communication protocol template to generate an original control instruction frame; Calculate and encapsulate the checksum of the original control instruction frame to generate standard control instructions that comply with the industrial communication protocol; Send standard control instructions to the programmable logic controller (PLC) on the production line via industrial Ethernet; After the programmable logic controller (PLC) parses and executes the standard control instructions, the control execution module obtains the production line data after execution through the data acquisition module to verify the instruction execution effect and form a closed-loop optimization control system.

Images