Intelligent factory production and LED screen media interaction system based on AR
Through the AR-based smart factory production and LED screen media interaction system, the problems of single production information interaction and poor adaptability to dynamic scenarios in smart factories have been solved, and dynamic optimization of production tasks and resources and efficient closed-loop management of information interaction have been achieved.
Patent Information
- Application Number
- CN202510954351.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-11
- Publication Date
- 2025-10-17
AI Technical Summary
The production information interaction in existing smart factories is single and has poor interactivity, with low efficiency in information acquisition. It is difficult to adapt to dynamic scenarios such as equipment failures and order changes, and there is a lack of dynamic optimization capabilities.
An AR-based smart factory production and LED screen media interaction system is adopted. Through data collection, preprocessing, analysis, decision optimization and execution modules, data quality improvement, dynamic task allocation and resource scheduling are achieved. Combined with the AR interaction module, immersive interaction is provided to drive the execution of production equipment and link the LED screen display.
It realizes the dynamic optimization scheduling of production tasks and resources, improves production efficiency and resource utilization, enhances the intuitiveness and convenience of information interaction, reduces the switching cost of information acquisition and equipment operation, and forms efficient closed-loop management.
Smart Images
Figure CN120806511A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application relates to the technical field of media interaction, and more particularly to an AR-based intelligent factory production and LED screen media interaction system. BACKGROUND
[0002] In the process of intelligent factory construction and development, efficient interaction and visual presentation of production information are crucial to improving production management level. In the prior art, production data is displayed through traditional monitoring interfaces, LED static boards and the like, and there are problems such as single information presentation and poor interaction, and obtaining information requires switching between different devices or interfaces, which is inefficient and prone to errors. In addition, task allocation and resource scheduling rely on static rules, lack adaptability in the face of dynamic scenarios such as equipment failure and order changes, and are difficult to optimize in a timely manner. SUMMARY
[0003] The technical problem solved by the present application is to provide an AR-based intelligent factory production and LED screen media interaction system, which can solve the problems in the background art.
[0004] To solve the above technical problems, according to one aspect of the present application, more specifically, an AR-based intelligent factory production and LED screen media interaction system, comprising: a data acquisition module acquires intelligent factory production element data, and a preprocessing module performs outlier detection, missing value filling, normalization and format unification operations to solve the problem of raw data quality and lay a solid foundation for subsequent analysis; then an analysis module calls machine learning and deep learning techniques to mine data value and understand production rules; a decision optimization module dynamically generates optimal task allocation and resource scheduling strategies based on reinforcement learning and optimization algorithms in combination with analysis results to adapt to dynamic production scenarios in the factory; then an AR interaction module presents decision information to workers in a virtual-real fusion manner to realize immersive interaction; and finally an execution module receives instructions to drive production equipment and auxiliary systems to execute accurately, forming a "collection-processing-analysis-decision-interaction-execution" closed loop.
[0005] Further, the preprocessing module comprises: outlier detection, missing value filling, normalization and format unification. Outlier detection: Z-Score statistical method is used to identify outliers in the data. For the collected production element data, the statistical characteristics of the data distribution are calculated, and the data deviating from the normal distribution range is marked as an outlier to avoid interference of abnormal data in the subsequent analysis process. Missing value filling: linear interpolation method is used to fill the missing data caused by communication packet loss and temporary sensor failure to ensure data integrity. Normalization: eliminate dimensional differences by Min-Max normalization, make data adapt to the input requirements of subsequent machine learning and deep learning models; Format unification: convert the data format output by different devices and different systems into unified structured data.
[0006] Further, the analysis module includes a machine learning module and a deep learning module. Machine learning module: call decision tree and random forest to analyze preprocessed production data. Deep learning module: deploy convolutional neural network and long short-term memory network deep neural network to process high-dimensional and time-series production data.
[0007] Further, the decision optimization module includes a reinforcement learning module and an optimization algorithm module. Reinforcement learning module: learn the optimal decision strategy that adapts to dynamic scenarios through deep Q network and proximal policy optimization, and respond to sudden situations such as device failure and order insertion in real time. Optimization algorithm module: intelligent optimization algorithm integrating genetic algorithm and particle swarm algorithm, assisting reinforcement learning strategy iteration.
[0008] Further, the AR interaction module, based on AR glasses hardware devices, superimposes the strategy output by the decision optimization module on the real production scene in the form of virtual three-dimensional models and dynamic guidance animations, while supporting gesture recognition and voice interaction human-computer interaction methods.
[0009] Further, the execution module receives the decision instructions confirmed by the AR interaction module through industrial programmable logic controllers and distributed control systems, drives production equipment to perform task scheduling actions, and links LED screens for display.
[0010] Further, through MQTT and OPCUA industrial communication protocols, cross-module collaboration of the data acquisition module, preprocessing module, analysis module, decision optimization module, AR interaction module, and execution module is realized.
[0011] The AR-based intelligent factory production and LED screen media interaction system has the beneficial effects that: production element data is acquired through the data acquisition module and the data quality is improved through the preprocessing module, the data value is mined by combining the machine learning and deep learning technologies of the analysis module, and the optimal strategy is dynamically generated by using the reinforcement learning and optimization algorithm of the decision optimization module, the system can effectively solve the problems of poor data quality and static task allocation in the prior art, realize dynamic optimization scheduling of production tasks and resources, improve production efficiency and resource utilization, accurately adapt to dynamic scenes such as equipment failure and order change, and intuitively present the decision information in a virtual-real fusion manner through the AR interaction module to support gesture and voice interaction, and the execution module drives the equipment to execute through the industrial controller and synchronizes the LED screen display, so that the system breaks through the interaction limitation of the traditional monitoring interface and static panel, reduces the switching cost of information acquisition and equipment operation of workers, enhances the intuitiveness and convenience of production information interaction, forms an efficient closed loop of 'acquisition-treatment-analysis-decision-interaction-execution', and improves the intelligence and collaboration of intelligent factory production management. BRIEF DESCRIPTION OF DRAWINGS
[0012] The application will be described in further detail below with reference to the drawings and specific implementation methods.
[0013] Fig. 1 is a system principle schematic diagram; Fig. 2 is a step flow schematic diagram. DETAILED DESCRIPTION
[0014] The application will be described in further detail below with reference to the drawings and specific implementation methods.
[0015] According to one aspect of the application, as shown in Figs. 1-2 An AR-based intelligent factory production and LED screen media interaction system is provided, which comprises: a data acquisition module that uses deployed temperature sensors, pressure sensors, encoders and other equipment to acquire device running parameters (such as speed, current), environmental data (temperature and humidity, dust concentration) and production progress data (order completion quantity, work-in-process quantity) in real time, and transmits the data dispersed in each station to a preprocessing module through a 5G network by using MQTT and OPCUA industrial communication protocols to form a multi-dimensional and real-time production data source for subsequent processing.
[0016] The preprocessing module solves the problem of raw data quality and lays a solid foundation for subsequent analysis. The module comprises: Outlier detection, using Z-Score statistical method to identify outliers in the data, for the production factor data collected, calculate the statistical characteristics of data distribution, mark the data deviating from the normal distribution range as outliers (such as sudden rise and fall of equipment current), avoid abnormal data interference subsequent analysis process, the specific formula is:
[0017] Wherein is the sample data, is the sample mean, is the sample standard deviation; Missing value filling: using linear interpolation method to fill the data missing caused by communication packet loss, sensor temporary failure (such as temperature data missing in a period), to ensure data integrity, Wherein the linear interpolation method is used to fill in the missing data, based on the linear relationship of adjacent known data points to estimate the missing value, which is suitable for continuous missing scenario of time series data, and its function is:
[0018] Wherein and are the time points of adjacent known data, and are the measured values of the corresponding time points, is the time point of missing data; Normalization: eliminate the difference of dimension (such as temperature 20-80℃, pressure 0-10MPa) by Min-Max normalization, and map to [0,1] interval, so that the data adapt to the input requirements of subsequent machine learning, deep learning model, the specific function is:
[0019] Wherein is the original data, and are the minimum and maximum values of the data respectively; Format uniformity: convert the data format (such as JSON, XML and other heterogeneous data) output by different devices and different systems into unified structured (CSV format) data; Through the above pretreatment, the accuracy, consistency and usability of production data can be effectively improved, laying a foundation for the algorithm operation of subsequent analysis module.
[0020] Analysis module, mining data value, insight into the production law. This module includes: Machine learning module, calling decision tree and random forest to analyze the preprocessed production data, Wherein the decision tree builds a tree model by recursively dividing the feature space. Taking equipment failure prediction as an example, based on vibration frequency, temperature, current and other features, information gain or Gini index is used as the splitting criterion. For example, when the motor temperature exceeds the threshold and the vibration frequency is abnormal, the decision tree will output the prediction result of “bearing wear failure”, And the random forest improves the prediction stability by integrating multiple decision trees and using voting mechanism (classification problem) or mean method (regression problem). For example, when analyzing the production efficiency of a production line, the random forest can consider multiple dimensional features such as equipment load, material distribution delay, and operator proficiency. Through the integrated learning of 500 decision trees, the influence weight of each factor on production capacity is output, and the accuracy is improved compared with single decision tree, Both of them work together to help engineers understand the failure causes through the interpretability of decision trees and reduce the overfitting risk of single model through the integrated advantage of random forest, suitable for rapid analysis of structured production data; Deep learning module, deploy convolutional neural network (CNN) and long short-term memory network (LSTM) deep neural network to process high-dimensional, time-series production data, Among them, the convolutional neural network (CNN) automatically extracts the spatial features of vibration spectrum image, device image and other data through convolution layer and pooling layer. For example, after converting the bearing vibration signal into a frequency spectrum image, the convolutional neural network (CNN) can identify subtle fault features (such as high-frequency components generated by early cracks) that traditional methods cannot detect, And the long short-term memory network (LSTM) processes the long-term dependence relationship in time series data through the gating mechanism. For example, when analyzing the production progress data of the past 30 days, the long short-term memory network (LSTM) can capture the influence law of order fluctuation and equipment maintenance cycle on production capacity, and predict the production capacity fluctuation range in the next 7 days, Both of them work together to process device state image and production progress data through the strong extraction ability of convolutional neural network (CNN) on spatial features and the long-term memory ability of long short-term memory network (LSTM) on time series, and realize multi-dimensional analysis of complex production scenarios; Through the collaborative analysis of production data by machine learning module and deep learning module, potential equipment failures can be accurately identified, production bottlenecks can be excavated, and data support can be provided for decision optimization module.
[0021] Decision optimization module, combined with the analysis results, dynamically generates the optimal task allocation and resource scheduling strategy to adapt to the dynamic production scene of the factory. This module includes: Reinforcement learning module: through deep Q network and proximal policy optimization, learn the optimal decision strategy to adapt to the dynamic scene, and respond to sudden situations such as equipment failure and order insertion, Wherein the deep Q network (DQN) encodes the device state (running / failure), task queue length (1-10), and resource margin (0-100%) into a 10x10x3 state tensor, uses the experience replay mechanism to store 100,000 historical decision data, and iteratively updates the task allocation strategy through the Q value network, And the proximal policy optimization (PPO) iteratively updates the strategy through the importance sampling technique, and the core is to use the clipping function (ClipFunction) to avoid the strategy update being too large, and the objective function is:
[0022] Wherein The formula is:
[0023] Wherein is the probability ratio of the new and old strategies, is the advantage function, and ∈ is the clipping parameter (usually 0.2), Both of them cooperate with each other to accelerate the policy learning efficiency through the experience replay mechanism of the deep Q network (DQN) and guarantee the decision stability through the clipping strategy of the proximal policy optimization (PPO), and together realize the 100ms-level task reallocation response when the device fails, and reduce the production downtime compared with the traditional static rules; Optimization algorithm module: intelligent optimization algorithm integrating genetic algorithm and particle swarm algorithm, assisting in policy iteration of reinforcement learning, Wherein the genetic algorithm generates an initial task allocation scheme for reinforcement learning through selection (fitness function is task completion rate), crossover (probability 0.8), and mutation (probability 0.05) operations, such as encoding 100 tasks to be allocated into a chromosome, and obtaining an initial strategy through 50 generations of evolution, so as to improve the convergence speed of the subsequent deep Q network (DQN) training, And the particle swarm algorithm (PSO) iterates through the particle position (resource allocation proportion) and speed (strategy update direction), and in the proximal policy optimization (PPO) training process, the particle swarm algorithm (PSO) optimizes the resource allocation weight of each production line with the goal of balancing the utilization rate of the device, for example, when the injection molding machine and the assembly line resources conflict, the particle swarm algorithm (PSO) adjusts the allocation proportion to improve the overall utilization rate; Through the intelligent optimization algorithm integrating the genetic algorithm (which has strong global search ability to provide high-quality initial strategy for reinforcement learning) and the particle swarm algorithm (which has the characteristics of fast iteration to optimize real-time resource allocation), the global optimality and convergence efficiency of the decision in the dynamic scene are improved.
[0024] AR interaction module, intuitively presents decision information to workers in a virtual-real fusion manner, realizes immersive interaction, specifically, based on AR glasses hardware device, superimposes the strategy output by the decision optimization module on the real production scene in the form of virtual three-dimensional model and dynamic guidance animation (for example, in the device failure scene, the AR glasses will mark the faulty components (such as a red virtual highlight ring) on the real device and play the maintenance step animation, at the same time, sends instructions to the workshop main LED screen through the MQTT protocol, so that the LED screen synchronously displays the historical fault data chart and maintenance resource scheduling scheme of the device, facilitating multi-person collaboration review), and supports human-computer interaction modes such as gesture recognition (such as grabbing a virtual button to adjust parameters) and voice interaction (for example, when saying "display line A progress", the AR glasses interface and the workshop LED screen will update the real-time Gantt chart and device status dashboard of the production line at the same time).
[0025] Execution module, drives production equipment and auxiliary system to execute accurately, specifically, receives the decision instructions confirmed by the AR interaction module through industrial programmable logic controller (PLC, such as Siemens S7-1200) and distributed control system (DCS) (for example, when the AR interaction module confirms the device task redistribution, the PLC will drive the AGV logistics trolley to adjust the path, at the same time, sends data to the workshop LED screen through the OPCUA protocol, so that the LED screen displays the material flow path and device scheduling state in the form of three-dimensional animation).
[0026] Of course, the above description is not a limitation of the present application, and the present application is not limited to the above examples, changes, modifications, additions or replacements made by ordinary skilled in the art within the essential scope of the present application also belong to the protection scope of the present application.
Claims
1. An AR-based smart factory production and LED screen media interaction system, characterized by: include: The data acquisition module acquires production factor data from smart factories. The pre-processing module performs outlier detection, missing value filling, normalization, and format unification operations to address raw data quality issues and lay a solid foundation for subsequent analysis. The analysis module then uses machine learning and deep learning technologies to mine data value and gain insights into production patterns. The decision-making optimization module relies on reinforcement learning and optimization algorithms, and combines analysis results to dynamically generate optimal task allocation and resource scheduling strategies to adapt to the factory's dynamic production scenarios; then, through the AR interaction module, decision information is intuitively presented to workers in a virtual-real fusion manner, achieving immersive interaction; finally, the execution module receives instructions, drives production equipment and auxiliary systems to execute precisely, forming a "collection-processing-analysis-decision-making-interaction-execution" closed loop.
2. The AR-based smart factory production and LED screen media interaction system according to claim 1, characterized in that: The preprocessing module includes: outlier detection, missing value filling, normalization, and format unification; Outlier detection: Use the Z-Score statistical method to identify outliers in the data. For the collected production factor data, calculate the statistical characteristics of the data distribution and mark the data that deviates from the normal distribution range as outliers to prevent abnormal data from interfering with the subsequent analysis process. Missing value filling: Linear interpolation is used to fill in missing data caused by communication packet loss and temporary sensor failure to ensure data integrity; Normalization: Min-Max normalization is used to eliminate dimensional differences and make the data suitable for the input requirements of subsequent machine learning and deep learning models; Format unification: Convert data formats output by different devices and systems into unified structured data.
3. The AR-based smart factory production and LED screen media interaction system according to claim 1, characterized in that: The analysis module includes: a machine learning module and a deep learning module; Machine learning module: calls decision trees and random forests to analyze pre-processed production data; Deep learning module: Deploys convolutional neural networks and long short-term memory networks to process high-dimensional, time-series production data.
4. The AR-based smart factory production and LED screen media interaction system according to claim 1, characterized in that: The decision optimization module includes: a reinforcement learning module and an optimization algorithm module; Reinforcement Learning Module: Through deep Q-network and proximal policy optimization, it learns the optimal decision-making strategy for dynamic scenarios and responds to emergencies such as equipment failures and order insertions in real time. Optimization algorithm module: An intelligent optimization algorithm that integrates genetic algorithm and particle swarm algorithm to assist in reinforcement learning strategy iteration.
5. The AR-based smart factory production and LED screen media interaction system according to claim 1, characterized in that: The AR interaction module, based on AR glasses hardware devices, superimposes the strategies output by the decision optimization module on the real production scene in the form of virtual three-dimensional models and dynamic guidance animations, and also supports human-computer interaction methods such as gesture recognition and voice interaction.
6. The AR-based smart factory production and LED screen media interaction system according to claim 1, characterized in that: The execution module receives the decision instructions confirmed by the AR interaction module through the industrial programmable logic controller and the distributed control system, drives the production equipment to execute the task scheduling action, and links the LED screen for display.
7. The AR-based smart factory production and LED screen media interaction system according to claim 1, characterized in that: The cross-module collaboration of the data acquisition module, preprocessing module, analysis module, decision optimization module, AR interaction module, and execution module is achieved through the MQTT and OPCUA industrial communication protocols.