Display screen assembly system, method and equipment and storage medium

Abstract

The invention provides a display screen assembly system, method and device and a storage medium, the system comprises a data management analysis module, a multi-agent module, an implementation monitoring module and a communication module which are in communication connection with each other, the data management analysis module is configured in a training framework and is responsible for processing and generating a process database related to display screen assembly, and the multi-agent module is configured in a training framework; intelligent process planning and optimization suggestions are provided; the multi-agent module is configured for assembly process management and comprises a plurality of agents, and each agent is responsible for different assembly tasks and carries out coordination and information sharing through the communication module; the implementation monitoring module is configured on a production site, integrates various sensors, and collects key data in the assembly process in real time. And the communication module is configured to manage communication and information transmission among the intelligent agents. According to the method, by introducing the large language model and the multi-agent module, higher-level intelligent process planning and optimization are realized, so that the assembly process is more efficient and flexible, and the adaptability is higher.

Description

Technical Field

[0001] The present invention relates to the field of electronic manufacturing technology, and in particular to a display screen assembly system, method, device and storage medium. Background Art

[0002] In the field of modern electronic manufacturing, the assembly process of display screens usually relies on highly automated production lines, which are composed of robotic arms, conveyor belts and visual inspection systems, and realize the automation of each process through preset programs. For example, some advanced display screen manufacturers use full-process automation technology, including the bonding of glass substrates, the installation of circuit components and the inspection of finished products. These systems significantly improve production efficiency and assembly accuracy through fixed workflows and pre-programmed steps.

[0003] However, the existing automated and intelligent assembly systems still have deficiencies in many aspects. First, the intelligent planning of the process flow is insufficient. The existing systems usually rely on preset rules or a single model, which makes it difficult to fully optimize complex process flows. In addition, most systems have limited capabilities in real-time monitoring and dynamic process optimization, and cannot fully utilize real-time feedback to achieve global optimization. The system's flexibility and scalability are also poor, and it is unable to efficiently respond to diverse and personalized production needs, limiting the flexibility and response speed of the production line.

[0004] In addition, in the face of complex process changes and abnormal situations, the existing system often needs to rely on manual intervention for adjustment and maintenance, and lacks an automated response mechanism. These deficiencies have led to reduced production efficiency and unstable product quality. A new technical solution is urgently needed to improve the intelligent level of automated display assembly to meet increasingly complex production needs. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a display screen assembly system, method, device and storage medium.

[0006] According to the first aspect of the present application, a display screen assembly system is proposed, the system comprising a data management and analysis module, a multi-agent module, an implementation monitoring module and a communication module which are communicatively connected to each other, wherein:

[0007] The data management and analysis module is configured to store, manage and analyze various data generated during the production process, and to process and generate a process database related to display assembly based on a large language model, and provide intelligent process planning and optimization suggestions;

[0008] The multi-agent module is configured for assembly process management and includes multiple agents. Each agent is responsible for a different assembly task and coordinates and shares information through the communication module to achieve efficient execution of the process and real-time feedback.

[0009] Implement monitoring modules, configure them at the production site, integrate various sensors, collect key data in the assembly process in real time, perform abnormality detection and feedback, and ensure the stability of the production process;

[0010] The communication module is configured to manage the communication and information transmission between various intelligent agents, and is responsible for standardizing and protocolizing the information to achieve compatibility and collaboration between various intelligent agents.

[0011] In the above technical solution, this application improves the intelligence level of the assembly process through the collaborative work of the data management and analysis module and the multi-agent module, ensuring the efficient execution and real-time feedback of each assembly link, thereby effectively reducing production costs and improving product quality. At the same time, the real-time data collection and anomaly detection functions of the monitoring module are implemented to ensure the stability and reliability of the production process and further reduce the production failure rate.

[0012] Furthermore, the multi-agent includes management analysis and decision-making agents, communication agents, and production process agents that are interconnected and have a large language model built in. The design of the multi-agent module optimizes resource allocation and decision support in the production process through the interconnection of management analysis and decision-making agents, communication agents, and production process agents. Through efficient information sharing and coordination among agents, the response speed and flexibility of the system are improved, and effective management of complex production environments is achieved.

[0013] Furthermore, the construction of the large language model of the production process agent includes:

[0014] A data processing unit is configured to collect industry-related documents, operation manuals and historical production records to form a data set, and clean and pre-process the data set;

[0015] The model training unit is configured to load the pre-trained model using a deep learning framework, calculate the loss function through forward propagation and optimize and update the model parameters through back propagation to obtain the training model for each process;

[0016] The model updating unit is configured to analyze the characteristics and requirements of each group of processes for different assembly processes, identify key parameters and variables, and use actual production data under specific process conditions to fine-tune and optimize the training model to obtain a dedicated model for each process;

[0017] In the above technical solution, the construction of the large language model can continuously optimize the production process plan through the division of labor and cooperation among the data processing unit, model training unit and model update unit, and enhance the model's ability to adapt to changes in the production environment. Through precise model fine-tuning and optimization, the accuracy of each process under specific conditions can be ensured, thereby improving the product's qualification rate and production efficiency.

[0018] Furthermore, the management analysis and decision-making agent includes a scheduling resource management agent and a quality control agent that are interconnected and have a large language model built in. The scheduling resource management agent is configured to manage and schedule production resources, optimize production plans, coordinate the work progress of each process, and dynamically adjust production plans; the quality control agent communicates with the production process agent and is responsible for monitoring and ensuring product quality. According to the data uploaded by the production process agent, a quality inspection process is formed, and a quality inspection report is generated, which is fed back to the corresponding type of agent of the production process agent. With the introduction of the scheduling resource management agent, the utilization efficiency of production resources and the flexibility of production plans are effectively improved. By dynamically adjusting the production plan, the coordination between the processes is ensured, the production cycle is reduced, and the overall production efficiency and the ability to respond to market changes are improved.

[0019] Furthermore, according to the production process and process, different process agents are defined, and the structures and communication protocols of different process agents are set. The production process agent includes each process agent that communicates with the data management and analysis module, the management analysis and decision-making agent, and the communication agent, specifically including:

[0020] Glass bonding intelligent body, used to accurately bond the glass layer of the display screen to the liquid crystal layer to obtain the display screen assembly, and use visual sensors and pressure sensors to monitor the bonding quality, and adjust the temperature and pressure in real time during the bonding process to avoid bubble generation and improve alignment accuracy;

[0021] The mainboard locking intelligent body fixes the mainboard to the display assembly, monitors the pressure value of the locking process through the force sensor, and provides timely feedback to adjust the locking speed and sequence;

[0022] The circuit installation agent is responsible for the installation and connection of the circuit board. It monitors the temperature of the soldering points, the depth and position of the wire insertion based on sensors, predicts potential circuit failures through electrical test data analysis, and feeds back to the data management and analysis module.

[0023] The device locking agent is responsible for the fixing and installation of other small components. It uses the intelligent gripping system to identify the components and accurately locate and lock them, and uses the visual system to feedback the installation accuracy.

[0024] The image quality inspection agent performs real-time inspection of the output image quality of the display screen, uses image processing algorithms to analyze color, brightness, and contrast indicators, and uploads the inspection results to the quality control agent;

[0025] Appearance inspection agent, which detects the appearance and labeling status of the display screen based on high-resolution cameras and image recognition technology, and uploads the inspection results to the quality control agent;

[0026] The packaging intelligence uses robots to automatically pack and seal the packages, and checks the firmness and sealing of the packages based on sensors.

[0027] Furthermore, the system also includes a maintenance support module, a security risk module and a user interface module:

[0028] The maintenance support module is configured for equipment monitoring and maintenance management. It is responsible for real-time monitoring of the operating status of production equipment, collecting operating data, performing fault prediction and maintenance scheduling, and ensuring the stability and reliability of the equipment.

[0029] Safety risk module, configured to monitor safety risks in the production process, detect potential dangers and abnormal situations, implement risk prevention and response measures, record and analyze safety incidents, and optimize safety strategies and measures;

[0030] The user interface module is configured as the interactive interface between the operator and the system, providing intuitive production status display, parameter adjustment and alarm information, helping operators to monitor and manage the production process in a timely manner. The user interface module provides a friendly operation interface, improves the operator's management efficiency of the production process, makes the entire production process more transparent and controllable, and effectively reduces operation errors and production risks.

[0031] Furthermore, the data processing unit is used to delete irrelevant information, duplicate data and data with irregular formats, perform denoising using regular expressions and / or text processing tools, and classify the data according to requirements.

[0032] Furthermore, the model training unit uses PyTorch and Huggin Face's Transformers library for model training. The models include GPT series models, such as L l ama, Vicuna, etc., and the model update unit uses business question and answer pairs to fine-tune and optimize the business, and uses LoRA and QLoRA algorithms for accelerated training. Using the PyTorch deep learning framework and LoRA and QLoRA algorithms for model training and fine-tuning can significantly improve training efficiency and reduce the time cost of model updates. At the same time, the fine-tuning method based on business question and answer pairs makes the model more targeted, can effectively meet actual production needs, and enhance the system's intelligent decision-making capabilities.

[0033] In a second aspect, the present application proposes a display screen assembly method, which is implemented based on the display screen assembly system described in the first aspect, and includes:

[0034] S1, the data management and analysis module obtains production requirements, generates an initial assembly process plan, and sends the information of each assembly process in the initial assembly process plan to the production process agent and management analysis and decision agent corresponding to the multi-agent module;

[0035] S2, the process unit agents of the production process agent cooperate to execute the assembly process, and the monitoring module feeds back the collected real-time data to the data management and analysis module and the management analysis and decision-making agent;

[0036] S3, the data management and analysis agent adjusts the initial assembly process plan according to the real-time data, and re-issues the revised assembly process information to the corresponding process unit agent, and updates the process database at the same time;

[0037] S4, the process unit agent continues to execute the assembly process based on the revised assembly process information, and the implementation monitoring module feeds back the collected real-time data to the data management and analysis module and the management analysis and decision-making agent, traversing and executing steps S3-S4 until the assembly process task required by production is completed.

[0038] In the above technical solution, the design of the display assembly method, based on the intelligent decision-making and real-time feedback mechanism of the large language model, forms a closed-loop control system, which can continuously optimize the process plan during the production process and improve the intelligence and flexibility of production. Through real-time data feedback and adjustment, the high quality and production efficiency of the final product are ensured.

[0039] In a third aspect, the present application proposes a terminal device, including a processor, a memory, and a computer program stored in the memory, wherein the computer program is executed by the processor to implement a display screen assembly system as described above.

[0040] In a fourth aspect, the present application proposes a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, a display screen assembly system as described above is implemented.

[0041] Compared with the prior art, the beneficial results of the present invention are:

[0042] 1. The present invention realizes a higher level of intelligent process planning and optimization by introducing a large language model and a multi-agent module, making the assembly process more efficient, flexible and adaptable. By collecting and analyzing various business data and fine-tuning the large language model, the model's ability to handle a variety of business problems is improved, allowing the system to flexibly adapt to different production environments and requirements.

[0043] 2. The design of intelligent entities in each process in the present invention uses sensors and visual systems for real-time monitoring and adjustment to ensure the accuracy of each process, thereby improving the product qualification rate and quality control level. This design is a process management system with adaptive adjustment and rapid expansion capabilities, which meets diversified and personalized production needs, supports flexible production models, and adapts to market changes.

[0044] 3. The present invention implements the integration of monitoring modules, collects key data in real time, performs abnormality detection and feedback, significantly improves the stability and reliability of the production process, reduces the production failure rate, and realizes real-time monitoring and dynamic process optimization of the production process by introducing real-time data analysis and feedback mechanisms, ensuring the global optimal operating state of the system under different conditions.

[0045] 4. The present invention reduces the reliance on manual intervention and improves the system's autonomous response capabilities to complex process changes and abnormal situations through intelligent process management and fault diagnosis mechanisms. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and are used together with the description to explain the principles of the present invention. It will be easy to recognize other embodiments and many expected advantages of the embodiments because they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with each other. The same reference numerals refer to corresponding similar parts.

[0047] Figure 1 is a flow chart of a display screen assembly method according to an embodiment of the present invention;

[0048] Figure 2 is a schematic diagram of a framework of a Transformer pre-training model according to an embodiment of the present invention;

[0049] Figure 3 is a schematic diagram of a framework of a model updating unit according to an embodiment of the present invention;

[0050] Figure 4 is a lora structure diagram according to an embodiment of the present invention;

[0051] Figure 5 is a schematic diagram of the overall framework of a display screen assembly system according to an embodiment of the present invention;

[0052] Figure 6a-6b is a diagram of the inspection result of the appearance inspection agent during the processing of the agent at the workstation 12 according to one embodiment of the present invention;

[0053] Figure 7 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0054] In the following detailed description, reference is made to the accompanying drawings, which form a part of the detailed description and are shown by illustrative specific embodiments in which the present invention can be put into practice. To this end, directional terms, such as "top", "bottom", "left", "right", "up", "down", etc., are used with reference to the orientation of the figures described. Because the parts of the embodiments can be positioned in several different orientations, directional terms are used for the purpose of illustration and are by no means limiting. It should be understood that other embodiments may be utilized or logical changes may be made without departing from the scope of the present invention. Therefore, the following detailed description should not be adopted in a limiting sense, and the scope of the present invention is defined by the appended claims.

[0055] The present invention provides a display screen assembly method. Figure 1 A flow chart of a display screen assembly method according to an embodiment of the present invention is shown. As shown in the figure, the method includes:

[0056] S101, the data management and analysis module obtains production requirements, generates an initial assembly process plan, and sends the information of each assembly process in the initial assembly process plan to the production process agent and management analysis and decision-making agent corresponding to the multi-agent module.

[0057] In a specific embodiment, the operator inputs production requirements, such as display screen specifications, production targets, etc. The data management and analysis module obtains the knowledge base, generates an initial assembly process plan according to the production requirements, and assigns each assembly process information in the initial assembly process plan to the corresponding intelligent agent to perform the corresponding process assembly operation.

[0058] S102, the process unit agents of the production process agent collaborate to execute the assembly process, and the implementation monitoring module feeds back the collected real-time data to the data management and analysis module and the management analysis and decision-making agent.

[0059] In a specific embodiment, various sensors are installed on assembly equipment and workstations, such as temperature sensors, pressure sensors, optical detection sensors, etc., and multiple agents collaborate to perform assembly processes. Each sensor collects various data in the production process in real time, and the collected data is input into the large language model module for analysis to identify potential process anomalies and provide timely feedback to relevant agents for adjustment.

[0060] S103, the data management analysis model adjusts the initial assembly process plan according to the real-time data, and re-issues the revised assembly process information to the corresponding process unit intelligent body, and updates the process database at the same time.

[0061] S104, the process unit agent continues to execute the assembly process based on the revised assembly process information, and the implementation monitoring module feeds back the collected real-time data to the data management and analysis module and the management analysis and decision-making agent, traversing and executing steps S103-S104 until the assembly process task required by the production is completed.

[0062] In a specific embodiment, the data management and analysis module analyzes the data uploaded by the implementation monitoring module. If a process deviation or potential fault is detected, the management analysis decision-making agent obtains the knowledge base, deviation or fault data, and uses appropriate prompts to enable the large language model to generate an adjustment plan and instruct related agents to make corrections. This method achieves adaptive optimization of the process flow by continuously accumulating production data, optimizing the knowledge base and assembly plan.

[0063] The following describes the prompt debugging process, which uses different methods to verify the business effect:

[0064] Fault data:

[0065] 1. Fault type A: occurs frequently and mainly affects device X.

[0066] 2. Fault type B: occurs frequently, affects device Y, and is usually accompanied by fault type C.

[0067] 3. Fault type C: Occurrence frequency is low, but repair time is long.

[0068] Knowledge base data:

[0069] 1. For fault type A, the recommended solution is to replace a component of device X.

[0070] 2. Fault types B and C usually need to be repaired at the same time, and the repair of fault C requires additional time and resources.

[0071] 3. Regular inspection of equipment Y can reduce the occurrence of fault type B.

[0072] Here are three common prompt engineering methods:

[0073] a. Direct instruction method: Based on the provided knowledge base data and fault data, provide corrected solutions and better strategies.

[0074] b. COT (Chai n of Thought): Applicable to scenarios that require step-by-step reasoning. Thinking about the problem step by step can help the model understand and generate solutions more clearly. Based on the provided knowledge base data and fault data, think step by step and list the following: "Correction plan for each fault type", "Priority and strategy when considering multiple fault types" and "Recommended long-term preventive measures".

[0075] c. TOT (Thoughts on Thoughts): Applicable to in-depth discussion based on existing thinking. It can guide the model to reflect on the conclusions that have been drawn and propose better solutions. Based on the knowledge base data and fault data provided, please think about the following questions: "For faults A, B, and C, are the current correction solutions effective enough? What room for improvement is there?", "When repairing faults, how to optimize resource allocation to improve efficiency?", "Are there other preventive measures that can more effectively reduce the frequency of faults?" and "Please evaluate the long-term sustainability of the existing strategy and make suggestions for improvement."

[0076] In a specific embodiment, in response to the production requirements of display screens of different specifications, the system can quickly adjust the assembly process parameters through a large language model to achieve flexible switching of the production line. The specific steps include:

[0077] Demand change input: operators or upstream systems input new production requirements.

[0078] Planning of process solutions: The large language model replans the assembly process flow and adjusts the parameter settings of each process according to new requirements.

[0079] Intelligent agent task reallocation, under the MAS architecture, the intelligent agent dynamically adjusts task allocation according to the new process plan to ensure the efficient operation of the production line.

[0080] As an implementation of the above method, in the second aspect, the present application provides a display screen assembly system, which includes a data management and analysis module, a multi-agent module, an implementation monitoring module and a user interface module that are interconnected and communicated with each other. The data management and analysis module is configured in the training framework, responsible for processing and generating a process database related to display screen assembly, and providing intelligent process planning and optimization suggestions; the multi-agent module is configured in the assembly process management, including multiple agents each responsible for different assembly tasks, each agent achieves efficient execution of the process and real-time feedback through coordination and information sharing; the implementation monitoring module is configured in the production site, integrates various sensors, collects key data in the assembly process in real time, performs abnormality detection and feedback, and ensures the stability of the production process. The user interface module is configured in the interactive interface between the operator and the system, providing intuitive production status display, parameter adjustment and alarm information, helping the operator to monitor and manage the production process in a timely manner.

[0081] In a specific embodiment, multiple agents include a management analysis and decision-making agent and a production process agent that are interconnected and have a large language model built in them, wherein the construction of the large language model of the production process agent includes a data processing unit, a model training unit and a model updating unit. The data processing unit collects industry-related documents, operation manuals and historical production records to form a training data set, and cleans and preprocesses the data set. The specific data processing unit is configured to delete irrelevant information, duplicate data and data in an irregular format, use regular expressions and / or text processing tools for denoising, and classify the data according to requirements; the model training unit uses a deep learning framework to load a pre-trained model, and uses a forward propagation to calculate the loss function and a back-propagation optimization to update the model parameters to obtain a training model for each process, which uses TensorFlow, PyTorch or Huggi ng Face's Transformers library is used for model training, and the models include one or more of GPT, BERT or T5; the model update unit analyzes the characteristics and requirements of each group of processes for different assembly processes, identifies key parameters and variables, and uses actual production data under specific process conditions to fine-tune and optimize the training model to obtain a test model for each process.

[0082] Specifically, refer to Figure 2 , Figure 2The schematic diagram of the framework of the Transformer pre-training model according to an embodiment of the present invention is shown. First, as much business data as possible is collected to perform pre-training and fine-tuning training of the large language model, so that the model is more adept at handling businesses related to the electronic manufacturing industry. The pre-training training data is mainly composed of business text data, such as "Order Picking is one of the core links in warehouse management. It refers to the process of selecting goods that meet customer order requirements from inventory and preparing them for packaging and shipment." Such text data is self-supervised learning using the Transformer model structure, that is, given the context of a GPT text, let GPT predict the next context. Through such pre-training, the model can obtain more business knowledge related to the electronic manufacturing industry. The business data of the electronic manufacturing industry belongs to structured knowledge, and the multimodal master data mainly includes picture data taken in the production process and corresponding sensor data.

[0083] Specific references Figure 3 , Figure 3 The schematic diagram of the framework of the model update unit according to an embodiment of the present invention is shown. The fine-tuning training data is mainly composed of business question-answer pairs, such as (help me check the inventory, select * from stock) business data in this form is supervised fine-tuning (SFT), which mainly uses algorithms such as lora and qlora for accelerated training. Through SFT training, the model can be better at processing specific business, such as text to SQL statements, etc. For further reference Figure 4 , Figure 4 The structure diagram of Lora according to an embodiment of the present invention is shown. As shown in the figure, Lora (Low-Rank Adaptation) is a technology for efficiently fine-tuning large pre-trained language models, such as GPT, BERT, etc. Traditional model fine-tuning requires updating a large number of model parameters, which can be very expensive in terms of computing resources and storage. Lora does not directly adjust all the parameters of the pre-trained model, but at certain key positions of the model, usually the weight matrix in the attention mechanism. Introduce low-rank (low-dimensional) trainable adaptation matrices. The parameters of these adaptation matrices are far less than those of the original model, which has the effect of accelerating training. Qlora is an extension of the Lora technology, and the optimization method is similar, but Qlora combines quantization technology to further improve efficiency, especially in resource-constrained environments. On the basis of applying Lora for fine-tuning, the basic model is quantized, usually converting high-precision parameters to low precision, such as 8 bits or lower, to reduce the memory usage of the model and speed up the calculation speed.

[0084] In a specific embodiment, the system also includes a data management and analysis module and a maintenance support module: the data management and analysis module is configured in the central control unit of the system, and is responsible for storing, managing and analyzing various types of data generated during the production process; the maintenance support module is configured in the equipment monitoring and maintenance management, and is responsible for real-time monitoring of the operating status of the production equipment, collecting operating data, performing fault prediction and maintenance scheduling, and ensuring the stability and reliability of the equipment.

[0085] In a specific embodiment, reference Figure 5 , Figure 5 FIG. 1 shows a schematic diagram of the overall framework of a display screen assembly system according to an embodiment of the present invention. Figure 5 As shown, a management analysis and decision module and a display screen assembly module that is connected to the management analysis and decision module through a communication module, wherein the management analysis and decision module includes a user interface agent, an optimization and decision agent, a scheduling and resource management agent, a maintenance and maintenance agent, a safety and risk agent, and a quality control agent, wherein the quality control agent is connected to various agents in communication, is responsible for monitoring and ensuring product quality, forms a quality inspection process based on the data uploaded by various agents, and generates a quality inspection report, which is fed back to the corresponding type of agent; the communication module includes a communication and coordination agent, a monitoring and data acquisition agent; the display screen assembly includes: a glass bonding agent, which is used to accurately bond the glass layer of the display screen to the liquid crystal layer to obtain a display screen assembly, and uses a visual sensor and a pressure sensor to monitor the bonding quality, and adjusts the temperature and pressure in real time during the bonding process to avoid bubble generation and improve alignment accuracy; a motherboard locking agent, which fixes the motherboard to the display screen assembly, through The force sensor monitors the pressure value of the locking process and provides timely feedback to adjust the locking speed and sequence; the circuit installation agent is responsible for the installation and connection of the circuit board. It monitors the temperature of the welding point, the depth and position of the wire insertion based on the sensor, predicts potential circuit failures through electrical test data analysis, and feeds back to the data management and analysis module; the device locking agent is responsible for the fixation and installation of other small components, applies the intelligent grasping system to identify the components and accurately locate and lock them, and uses the visual system to feedback the installation accuracy; the image quality inspection agent is configured to perform real-time detection of the output image quality of the display screen, uses image processing algorithms to analyze color, brightness, and contrast indicators, and uploads the inspection results to the quality control agent; the appearance inspection agent detects the appearance and labeling status of the display screen based on high-resolution cameras and image recognition technology, and uploads the inspection results to the quality control agent; the packaging agent uses robots to automatically pack and package, and checks the firmness and sealing of the packaging based on sensors.

[0086] Specifically, the user interface agent in management analysis and decision-making is used to provide a human-computer interaction interface, allowing operators to view production status, adjust process parameters and receive system prompts, accept instructions and inputs from operators, and pass them to related agents for processing, while providing visual data display and report generation to help decision-making and management. It not only improves the interactive experience between operators and the system, simplifies the operation process, but also provides intuitive information display to assist operators in making effective decisions.

[0087] Specifically, the optimization and decision-making agent analyzes the collected production data, identifies potential optimization opportunities, and proposes adjustment plans for process parameters based on the large language model and optimization algorithm, adjusts production parameters in real time, ensures the optimization of the production process, improves production efficiency and product quality, and realizes dynamic optimization of the process through intelligent decision-making. Adapt to different production needs and changes to maintain the efficient operation of the production system. The optimization and decision-making agent is based on the planning and reasoning ability of the large language model as the brain and the call of external tools to execute and implement decisions. Therefore, when the agent is optimizing, it first uses the world knowledge of the large language model. The large language model is good at planning and reasoning, and will directly give specific optimization ideas and external tool calls based on the current input, such as Bayesian optimization algorithm, cost-optimal linear programming algorithm, search engine, data access interface, etc.

[0088] Specifically, the scheduling and resource management agent manages and schedules production resources (such as raw materials, workstations, equipment, etc.) to ensure efficient use of resources; optimizes production plans, coordinates the progress of each process, avoids resource conflicts and bottlenecks; dynamically adjusts production plans to respond to emergencies and changing needs. Improve resource utilization and reduce production costs. Ensure smooth production processes and improve overall production efficiency. The agent calls the data access interface to obtain information such as raw materials and workstations, uses the planning and reasoning capabilities of the large language model to optimize task scheduling, and then calls related agents to execute strategies.

[0089] Specifically, the maintenance and repair agent monitors the equipment status, analyzes the equipment operation data, predicts potential failures and maintenance needs; arranges and dispatches maintenance and repair work to ensure the normal operation of the equipment; records and tracks maintenance history and optimizes maintenance strategies. Reduce equipment failure rate and downtime, extend equipment service life, and improve the overall reliability and stability of the production line. The safety and risk management agent monitors safety risks in the production process, detects potential dangers and abnormal situations, implements risk prevention and response measures, ensures the safety of the production environment, records and analyzes safety incidents, and optimizes safety strategies and measures; ensures personal and equipment safety in the production process, reduces accidents; improves the safety and reliability of the system, and maintains the production stability of the enterprise. The quality control agent is responsible for monitoring and ensuring product quality, analyzing test data, identifying and correcting defects, and realizing automated quality inspection processes, such as image recognition defects, parameter monitoring, etc., generating quality reports, and feeding back to relevant agents for further processing. Improve product consistency and reliability, reduce defective rates, and ensure that the production process meets standards and specifications through continuous quality monitoring.

[0090] Specifically, the communication and coordination agent in the communication module manages the communication and information transmission between different agents, ensures the coordinated execution of tasks, realizes the standardization and protocolization of information, ensures the compatibility and collaboration between different modules, handles cross-process and cross-module coordination tasks, and solves potential conflicts and problems. Improve the coordination and integrity of the system, ensure the efficient cooperation of each agent, promote the timely sharing and transparency of information, and enhance the responsiveness of the system. The agent collects information from other agents, analyzes data from a global perspective, and assigns tasks to other agents to achieve the role of communication and coordination. Preferably, various agents exchange information and coordinate tasks in the system through predefined communication protocols and collaboration mechanisms. For example, the data collected by the monitoring and data acquisition agent will be used by the quality control agent and the optimization and decision-making agent at the same time. The adjustment plan proposed by the optimization and decision-making agent will be passed to the corresponding process agent and the scheduling and resource management agent to ensure the dynamic adjustment of process parameters and the reasonable allocation of resources. The maintenance and repair agent actively arranges maintenance tasks based on the equipment status data, and collaborates with the scheduling and resource management agent to ensure the efficient execution of maintenance work. The communication and coordination agent is responsible for the information flow between the agents, ensuring the smooth collaboration of tasks and the timely resolution of problems. The monitoring and data acquisition agent monitors the various parameters (such as temperature, pressure, humidity, etc.) of the entire production process in real time, collects and integrates data from various processes and sensors, ensures the accuracy and completeness of the data, performs preliminary processing and filtering of the data, and eliminates noise and abnormal data. Provides comprehensive visualization of the production process to help the system and operators understand the production status in real time. Provides a reliable data foundation for subsequent data analysis and optimization. By introducing various types of collaborative agents, not only can the intelligent execution and monitoring of each process be realized, but also the overall coordination, optimization and adaptability of the system can be enhanced. This multi-agent collaboration mechanism helps to build an efficient, flexible and intelligent display assembly process processing system, and comprehensively improve production efficiency, product quality and system reliability.

[0091] In another specific embodiment, the multi-agent module specifically includes:

[0092] At the initial stage of display assembly, the dedicated agent at station 1 drives the automated equipment to fit the conductive foam and the wire buckle at the designated location. The agent uses a camera to check the fit accuracy and position to ensure compliance with relevant standards. If a deviation is detected, the agent will immediately adjust the equipment to correct it.

[0093] The dedicated agent at station 2 controls the automation equipment to connect the circuit board to the wire. The agent uses sensors to detect the depth and position of the wire to ensure the normal function of the circuit board. After the connection is completed, the agent performs electrical testing to confirm the reliability of the connection.

[0094] The dedicated agent at station 3 guides the automated equipment to fix the tube board to the display frame and secure it with screws. The agent monitors the tightening torque of the screws in real time to ensure the stability of the tube board. If it detects that the tightening torque is insufficient or too high, the agent will sound an alarm and make adjustments.

[0095] The dedicated agent at Workstation 4 uses automated equipment to screw the front and side screws of the motherboard. The agent monitors the tightening process of the screws to ensure their stability and connection reliability. After the tightening is completed, the agent performs electrical testing to confirm the normal working status of the motherboard.

[0096] The dedicated agent at station 5 uses automatic camera detection technology to inspect the aforementioned workstations. The agent uses image recognition technology to confirm whether the screws are leaking and whether the wires are inserted in place. The test results are uploaded to the data analysis agent in real time for further analysis.

[0097] The dedicated agent in station 6 controls the automated equipment to fix the back cover of the display screen in the correct position with screws. The agent monitors the tightening torque and position of the screws to ensure the integrity of the overall structure. If any anomalies are detected, the agent will immediately adjust the equipment to make corrections.

[0098] The dedicated agent at station 7 uses automated equipment to check the tightening of the rear case screws. The agent uses sensors to detect the tightening status of each screw to ensure that all screws are properly tightened. The inspection results are uploaded to the quality control agent in real time for further analysis.

[0099] The dedicated intelligent agent at Workstation 8 controls the automated equipment to fix the display screen to the base and lock it with screws. The intelligent agent monitors the tightening process of the screws to ensure that the display screen can stand firmly. After the locking is completed, the intelligent agent performs a stability test to confirm the standing stability of the display screen.

[0100] The dedicated agent at station 9 uses automatic photo comparison technology to check the image quality of the display screen. The agent uses image processing technology to detect image quality problems such as bright spots, dark spots, and color differences. The inspection results are uploaded to the quality control agent in real time for further analysis.

[0101] The dedicated agent at station 10 guides the automated equipment to attach the B / C logo or related labels to the display screen. The agent detects the pasting position and firmness through sensors to ensure the accuracy and stability of the label pasting. If a deviation is detected, the agent will immediately adjust the equipment to make corrections.

[0102] The dedicated intelligent agent at station 11 controls the automated equipment to wipe the exterior of the display screen. The intelligent agent detects the wiping effect through sensors to ensure that there are no stains or fingerprints. After wiping is completed, the intelligent agent performs an appearance inspection to confirm the cleanliness of the appearance.

[0103] The dedicated agent at station 12 uses automated equipment to check the appearance of the display screen for scratches and dirt. The agent uses image recognition technology to detect appearance problems to ensure the appearance quality of the product. The inspection results are uploaded to the quality control agent in real time for further analysis.

[0104] The dedicated intelligent control equipment at station 13 seals and vacuums the display screen. The intelligent agent uses sensors to detect the sealing and vacuuming of the bag to ensure that dust and other contamination can be prevented. If a poor seal is detected, the intelligent agent will immediately adjust the equipment to make corrections.

[0105] The dedicated intelligent control device at station 14 puts all the accessories required for the display into the package and automatically weighs them to confirm whether the accessories are complete. The intelligent agent detects the position and weight of each accessory through sensors to ensure that nothing is missing. The detection results are uploaded to the data analysis intelligent agent in real time for further analysis.

[0106] The dedicated agent at station 15 guides the automated equipment to perform the final packaging of the display and its accessories. The agent uses sensors to detect the firmness and sealing of the packaging to ensure the safety and stability of the packaging. After the packaging is completed, the agent performs a final inspection to confirm the quality of the packaging.

[0107] Specifically, the display assembly process covers more than a dozen processes. Here we focus on one of the workstations, and the other workstations are similar. Each workstation is similar to a robotic process automation (RPA) equipped with a brain (agent), which upgrades PPA to agentic process automation (APA). Traditional RPA uses scripting language and workflow design to complete automated operations, which is complex and has low accuracy. APA uses agents based on large language models (LLM) to achieve advanced automation, and reduces the burden of manpower by assigning tasks to agents responsible for construction and execution. It can create workflows based on manual instructions and make complex decisions by coordinating professional agents. After using APA, the performance was significantly improved from 22.28% of the baseline to 95.21%. The workstation agent also integrates RPA design with tool calls, making APA more flexible and able to handle more business. For example, some workstations need to train image-related algorithms and then deploy the algorithms as application programming interfaces (APIs) for agents to select and call; some workstations need to use sensors to obtain data for analysis. Through such decoupling, the intelligent agent can focus on logic and decision-making, while the tools can focus on the development of specific applications.

[0108] Continue to refer Figure 6a-6b , Figure 6a-6b 1 and 10 respectively show the detection result diagrams of the appearance detection agent in the processing process of the station 12 agent according to an embodiment of the present invention. The main function of the station 12 agent is to detect whether there are scratches or dirt on the appearance of the display screen, that is, to train an object detection algorithm, first collect data related to the appearance of various display screens, and then annotate the collected data (such as Figure 6a , the red box indicates the target to be detected by the model, Figure 6b After preprocessing (such as rotation, cutting, splicing, etc.), it is used to train the YOLO v10 target detection algorithm. After training a network with good performance, it is deployed as an API for the intelligent agent to call. When the camera of the workstation 12 takes a frontal photo of the display screen and transmits it to the YOLO API service, it can identify whether the display screen is cracked or dirty.

[0109] In a third aspect, the present application proposes a terminal device, including a processor, a memory, and a computer program stored in the memory, wherein the computer program is executed by the processor to implement any of the above display screen assembly methods.

[0110] In a fourth aspect, the present application proposes a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, any of the above display screen assembly methods is implemented.

[0111] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 of a terminal device or server suitable for implementing an embodiment of the present application. Figure 7 The terminal device or server shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0112] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the computer system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0113] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0114] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium of the present application can be a computer-readable signal medium or a computer-readable medium or any combination of the above two. The computer-readable medium can be, for example, - but not limited to - a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus or a device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, an apparatus or a device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0115] Computer program code for performing the operations of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0116] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] Obviously, those skilled in the art can make various modifications and changes to the embodiments of the present invention without departing from the spirit and scope of the present invention. In this way, if these modifications and changes are within the scope of the claims of the present invention and their equivalents, the present invention is also intended to cover these modifications and changes. The word "comprising" does not exclude the presence of other elements or steps not listed in the claims. The simple fact that certain measures are recorded in mutually different dependent claims does not indicate that the combination of these measures cannot be used to profit. Any reference numerals in the claims should not be considered to limit the scope.

Claims

1. A display screen assembly system, characterized in that: The system comprises a data management and analysis module, a multi-agent module, an implementation monitoring module and a communication module which are communicatively connected to each other, wherein: The data management and analysis module is configured to store, manage and analyze various types of data generated during the production process, and to process and generate a process database related to the assembly of the display screen based on a large language model, and to provide intelligent process planning and optimization suggestions; The multi-agent module is configured for assembly process management and includes multiple agents, each of which is responsible for a different assembly task, and coordinates and shares information through the communication module to achieve efficient execution of the process and real-time feedback; The implementation monitoring module is configured at the production site, integrates various sensors, collects key data in the assembly process in real time, performs abnormality detection and feedback, and ensures the stability of the production process; The communication module is configured to manage the communication and information transmission between the various intelligent agents, and is responsible for standardizing and protocolizing the information to achieve compatibility and collaboration between the various intelligent agents.

2. A display screen assembly system according to claim 1, characterized in that: The multi-agents include a management analysis and decision-making agent and a production process agent which are interconnected and have a large language model built in.

3. A display screen assembly system according to claim 2, characterized in that: The construction of the large language model of the production process agent includes: A data processing unit configured to collect industry-related documents, operation manuals, and historical production records to form a data set, and to clean and pre-process the data set; The model training unit is configured to load the pre-trained model using a deep learning framework, calculate the loss function through forward propagation and optimize and update the model parameters through back propagation to obtain the training model for each process; The model updating unit is configured to analyze the characteristics and requirements of each group of processes for different assembly processes, identify key parameters and variables, and use actual production data under specific process conditions to fine-tune and optimize the training model to obtain a test model for each process.

4. A display screen assembly system according to claim 2, characterized in that: The management analysis and decision-making intelligent agent includes a scheduling resource management intelligent agent and a quality control intelligent agent that are interconnected and have a large language model built in. The scheduling resource management intelligent agent is configured to manage and schedule production resources, optimize production plans, coordinate the work progress of each process, and dynamically adjust production plans; the quality control intelligent agent is communicated with the production process intelligent agent, is responsible for monitoring and ensuring product quality, forms a quality inspection process based on the data uploaded by the production process intelligent agent, and generates a quality inspection report, which is fed back to the intelligent agent of the corresponding type of the production process intelligent agent.

5. A display screen assembly system according to claim 4, characterized in that: According to the actual production process and process, different production process agents are defined, and the structures and communication protocols of different process agents are set. The production process agent includes each process unit agent that is communicatively connected with the data management and analysis module, the management analysis and decision agent, and the communication module, specifically including: A glass bonding intelligent body, which is used to accurately bond the glass layer of the display screen to the liquid crystal layer to obtain a display screen assembly, and to monitor the bonding quality using a visual sensor and a pressure sensor, and to adjust the temperature and pressure in real time during the bonding process to avoid bubble generation and improve alignment accuracy; The mainboard locking intelligent body fixes the mainboard to the display The display assembly monitors the pressure value of the locking process through the force sensor, and provides timely feedback to adjust the locking speed and sequence; The circuit installation agent is responsible for the installation and connection of the circuit board. It monitors the temperature of the welding point, the depth and position of the wire insertion based on sensors, predicts potential circuit failures through electrical test data analysis, and feeds back to the data management and analysis module. The device locking agent is responsible for the fixing and installation of other small components. It uses the intelligent gripping system to identify the components and accurately locate and lock them, and uses the visual system to feedback the installation accuracy. An image quality inspection agent performs real-time inspection of the output image quality of the display screen, uses an image processing algorithm to analyze color, brightness, and contrast indicators, and uploads the inspection results to the quality control agent; An appearance inspection agent detects the appearance and labeling status of the display screen based on a high-resolution camera and image recognition technology, and uploads the inspection results to the quality control agent; The packaging intelligence uses robots to automatically pack and seal the packages, and checks the firmness and sealing of the packages based on sensors.

6. A display screen assembly system according to claim 1, characterized in that: The system also includes a maintenance support module, a security risk module and a user interface module: The maintenance support module is configured for equipment monitoring and maintenance management, responsible for real-time monitoring of the operating status of production equipment, collecting operating data, performing fault prediction and maintenance scheduling, and ensuring the stability and reliability of the equipment; The safety risk module is configured to monitor safety risks in the production process, detect potential dangers and abnormal situations, implement risk prevention and response measures, record and analyze safety incidents, and optimize safety strategies and measures; The user interface module is configured as an interactive interface between the operator and the system, providing intuitive production status display, parameter adjustment and alarm information, helping the operator to monitor and manage the production process in a timely manner.

7. A display screen assembly system according to claim 3, characterized in that: The data processing unit is used to delete irrelevant information, duplicate data and data in irregular formats, perform denoising using regular expressions and / or text processing tools, and classify data according to requirements.

8. A display screen assembly system according to claim 3, characterized in that: The model training unit uses the Transformers library of PyTorch or Hugging Face to perform model training, the model includes one or more of GPT, BERT or T5, and the model updating unit uses the form of business question and answer pairs to fine-tune and optimize the business, and uses LoRA and / or QLoRA algorithms for accelerated training.

9. A display screen assembly method, characterized in that: The method is implemented based on the display screen assembly system according to any one of claims 1 to 8, and the method comprises: S1, the data management and analysis module obtains production requirements, generates an initial assembly process plan, and sends the information of each assembly process in the initial assembly process plan to the production process agent and management analysis and decision agent corresponding to the multi-agent module; S2, each process unit agent of the production process agent cooperates to perform the assembly process, and the implementation monitoring module feeds back the collected real-time data to the data management and analysis module and the management analysis and decision agent; S3, the data management and analysis model adjusts the initial assembly process plan according to the real-time data, and re-issues the revised assembly process information to the corresponding process unit intelligent agent, and updates the process database at the same time; S4, the process unit agent continues to execute the assembly process based on the revised assembly process information, and the implementation monitoring module feeds back the collected real-time data to the data management and analysis module and the management analysis and decision-making agent, traversing and executing steps S3-S4 until the assembly process task of the production requirement is completed.

10. A computer-readable storage medium, wherein a computer program is stored in the medium, and when the computer program is executed by a processor, the display screen assembly system according to any one of claims 1 to 8 is implemented.

Images