Display panel repair path optimization method and device based on deep reinforcement learning

Abstract

The invention relates to a display panel repair path optimization method and device based on deep reinforcement learning. The method comprises the following steps: obtaining a target display panel image; performing preprocessing and feature extraction on the target display panel image to obtain related features; inputting the related features into a pre-established deep reinforcement learning model, outputting an optimized repair path by the deep reinforcement learning model, and repairing the display panel; the deep reinforcement learning model is a PPO algorithm model which is constructed based on deep reinforcement learning and takes a strategy network and a value network as intelligent agents. According to the method, the problems of path redundancy and planning time consumption in a multi-defect scene in a traditional method are solved. Through real-time data acquisition, a reward function and a strategy are continuously updated by using a deep reinforcement learning algorithm, so that a repair path and an on-site defect state are matched in real time, and the problem that dynamic defect changes cannot be responded in real time in a traditional method is solved.

Description

Technical Field

[0001] The present invention relates to the technical field related to reinforcement learning and displays, and in particular to a method and device for optimizing a display panel repair path based on deep reinforcement learning. Background Art

[0002] In industrial manufacturing, OLED display panels can develop a variety of defects during the production process, including macro defects such as dots, lines, scratches, and bumps, as well as micro defects such as overspray, underspray, missed spray, overlapping, and scattered dots. To improve production efficiency and product quality, these defects need to be repaired promptly on the production line. Common repair methods include laser repair and electro-hydraulic inkjet printing. The repair heads used in these two methods must follow a specific path to address multiple defect points on the display panel one by one. To achieve efficient and automated panel repair, it is necessary to develop repair path optimization algorithms with multiple repair heads. This allows the repair tool to accurately repair all defect points using the shortest path and in an efficient manner, thereby improving production efficiency and reducing costs. Traditional repair path planning uses fixed sequential scanning or heuristic algorithms (such as genetic algorithms and ant colony algorithms). These problems include lengthy path planning times when using multiple repair heads in an integrated mode, a static path that cannot adapt to the dynamic distribution of defects, and high path redundancy in multi-defect scenarios. Summary of the Invention

[0003] The purpose of the present invention is to address at least one of the deficiencies of the prior art and to provide a method and device for optimizing a display panel repair path based on deep reinforcement learning.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions: Specifically, a display panel repair path optimization method based on deep reinforcement learning is proposed, including the following: Acquire the target display panel image; Preprocessing and feature extraction are performed on the target display panel image to obtain relevant features; Inputting relevant features into a pre-established deep reinforcement learning model, which outputs an optimized repair path and performs display panel repair; The deep reinforcement learning model is a PPO algorithm model built based on deep reinforcement learning with a policy network and a value network as intelligent agents.

[0005] Further, specifically, preprocessing and feature extraction are performed on the target display panel image to obtain relevant features, including: The target display panel image is processed by adaptive bilateral filtering to obtain a preprocessed image, the defect area is separated from the background, and the centroid coordinates, equivalent diameter, and aspect ratio geometric features of the preprocessed image are extracted to obtain relevant features.

[0006] Further, specific, pre-built deep reinforcement learning models include, Based on the actual repair scenario of multiple repair heads, simulation modeling is carried out, and a PPO algorithm model with policy network and value network as intelligent agents is constructed based on deep reinforcement learning; The PPO algorithm model is trained offline for multiple rounds in a simulation environment to obtain the final repair path planning strategy. During the training process, the intelligent agent interacts with the repair process simulation environment, collects interaction trajectory data, and calculates the advantage value and target value; Based on the advantage value and target value, the policy network and value network are updated in small batches to obtain new policy parameters and complete the synchronization of policy parameters.

[0007] Furthermore, specifically, simulation modeling is performed based on the actual repair scenario of multiple repair heads, including: Aiming at the actual repair scenario of multiple repair heads, the path optimization problem of OLED display panel defect repair is modeled. The display panel defect repair process is abstracted into a Markov decision process. For each time t, the system state, action, reward, and next state are expressed as follows: ; in represents the state space, represents the action space, represents the reward function, Represents the new environment after the action is executed; The state space is divided into two parts: panel defect information and repair tool status, using a two-dimensional matrix Indicates panel defect information, that is, displays the defect conditions of each area on the panel. Indicates no defects, Indicates the presence of a defect, i and j are the two-dimensional coordinate indexes of the display panel defect, where i represents the horizontal position coordinate of the panel defect, and j represents the vertical position coordinate of the panel defect. The numerical value represents the severity of the defect. The selection of the repair tool adopts discrete variables. Indicates, where 0 represents the laser repair head and 1 represents the inkjet repair head. The position of the repair tool is represented by two-dimensional coordinates, and the current position is represented by , Indicates the horizontal position coordinates of the current tool. Indicates the longitudinal position coordinate of the current tool, and the inkjet volume and laser power repair parameters of the tool are expressed as The final state space is defined as: ; Where t represents the current time step, and the state space is continuously updated with time step t. M×N represents the dimension of the panel defect information matrix, 2 represents the dimension of the two-dimensional coordinates of the current repair tool position, and P represents the dimension of the tool repair parameter; The selection of a tool is a discrete action, expressed as , indicates whether the dth tool is in the selected state at time step t, where 0 indicates that the tool is not selected and 1 indicates that the tool is selected; the movement of the tool position and the adjustment of the tool parameters are continuous actions, and the offset of the repair head moving on the panel is defined as , represents the action value of the C1 action category at time step t, △X represents the distance the repair tool moves along the horizontal axis at the current time step, △y represents the distance the repair tool moves along the vertical axis at the current time step, and Indicates the change of repair head parameters, at （c2） Represents the action value of the C2 action category at time step t, and the final action space is defined as: ; Reward t The agent performs the action The reward function is set to give feedback from the environment. The reward design is related to the path length, repair quality and energy consumption. The longer the distance moved, the worse the repair quality, and the excessive energy consumption, the negative reward is given. : ; in is the weight of the adjustment parameter, For the ideal repair state, is the moving distance of the repair head, Ideal tool parameters.

[0008] Furthermore, specifically, based on deep reinforcement learning, a PPO algorithm model with a policy network and a value network as intelligent agents is constructed, including: To build a PPO algorithm model, we need to build two neural networks, where the policy network is used to generate hybrid policies and the value network is used to estimate state values. Since the action space is a hybrid, the policy function can be decomposed into discrete and continuous parts. Discrete actions use softmax to output probability distribution: ; in Network status Discrete action score, It represents the policy network parameters of the d-th tool. As a learnable parameter in the neural network, it provides policy decisions for the d-th tool. a' represents the set of all tool selection actions, where a'=0 means that the repair tool is not selected at the current time step, and a'=1 means that the repair tool is selected at the current time step. Continuous actions use multi-dimensional Gaussian distribution, and its probability density function is: ; Among them, =[at （c1） ,at （c2） ], is the set of the repair head offset and the repair head parameter change, represents the policy network parameters of the c-th controller; The mean is output by the policy network: ; The value network is used to estimate the long-term return of a state and is defined as: ; in is the discount factor, Neural network weight parameters representing the value function, It represents the immediate reward at step t+k, and the reward of each step is multiplied by the discount factor.

[0009] Furthermore, the method also includes the objective function and update of the PPO algorithm model, including: By clipping policy updates, we can limit the change between the old and new policies to obtain stable policy improvements and define the probability ratio of the new and old policies: ; in Indicates that under the current policy parameters θ, in state Take action The probability of Indicates that under the old policy, in the state Take action Probability Establish the advantage function and calculate the advantage using the generalized advantage estimate: ; Where T is the end time of a complete trajectory, γ is the reward discount factor, λ is the GAE smoothing factor, which is used to control the trade-off between bias and variance, and Δt+l is the time difference error of the t+lth step; The time difference error is ; The clipping objective function of the policy network is defined as: ; in, Indicates the ratio Restricted to [ ], where ε is the shear threshold The mean squared error loss of the value network is defined as: ; in, Expressed as the target state value, it is composed of the advantage function estimate and the current state value estimate; In order to encourage strategy exploration, the entropy regularization term is introduced: ; in, Indicates that in a given state Next, the probability distribution of action a; The total loss function is: ; in and is a hyperparameter used to balance the contribution of value loss and entropy regularization.

[0010] Furthermore, the method also includes the following training process of the deep reinforcement learning model: Step 1 is the initialization phase, which initializes the policy network parameters. , value network parameters , old policy parameters and a defect repair simulation environment configured for OLED panels; The second step is interaction and data collection. In each training cycle, the current strategy is used Run multiple time steps in the environment and record the state, action, reward and next state at each step, storing the collected data in temporary storage; Step 3 is strategy update and parameter optimization. In order to update the strategy, it is necessary to calculate the advantage estimate for each time step and target value ,Using the collected data and the calculated advantage value, multiple small batch updates are made to the strategy and value network; Step 4 is the synchronization of strategy parameters. After completing the update of several epochs, the updated strategy parameters are synchronized. Assign values ​​to the old strategy parameters to ensure that the latest strategy is used in the next round of data collection; Repeat steps 2 to 4 until the model converges.

[0011] The present invention also proposes a device for optimizing a display panel repair path based on deep reinforcement learning, comprising the following: A base, wherein a motion guide rail is provided on the upper end thereof, a gantry beam is provided on the motion guide rail, the gantry beam is capable of sliding along the motion guide rail, and the gantry beam is arranged perpendicular to the motion guide rail; The laser repair head and the inkjet repair head are respectively fixed on the gantry crossbeam and can slide along the gantry crossbeam; A processing module, electrically connected to the laser repair head and the inkjet repair head, for acquiring an image of a target display panel through an image acquisition module; Preprocessing and feature extraction are performed on the target display panel image to obtain relevant features; Input relevant features into a pre-established deep reinforcement learning model, which outputs an optimized repair path and controls the laser repair head and the inkjet repair head to repair the display panel; The deep reinforcement learning model is a PPO algorithm model built based on deep reinforcement learning with a policy network and a value network as intelligent agents.

[0012] Furthermore, specifically, the image acquisition module is realized by multiple industrial cameras and laser displacement sensors installed on the production line. The industrial cameras include visible light cameras and infrared cameras. After the display panel is in place, a trigger signal is sent through the photoelectric sensor to start multiple industrial cameras and laser displacement sensors at the same time. The visible light camera captures surface microscopic defects, and the infrared camera detects internal defects to obtain the target display panel image. The laser displacement sensor records the three-dimensional coordinates of the optical head in real time, and the IEEE1588 protocol is used to achieve microsecond-level synchronization of multiple devices. The target display panel image is compressed by the acquisition card and transmitted to the processing module through the PCle bus.

[0013] The beneficial effects of the present invention are: This paper proposes a display panel repair path optimization method and device based on deep reinforcement learning. Simulating real-world repair scenarios, the method employs offline training and multi-objective optimization of the deep reinforcement learning PPO algorithm to construct a reward function that incorporates multiple process constraints (including repair quality, path efficiency, and tool parameters). This method implements repair path planning for a multi-head integrated system with both laser and electro-hydraulic inkjet repair heads, addressing the path redundancy and time-consuming planning issues inherent in traditional methods for multiple defect scenarios. By collecting data in real time and continuously updating the reward function and strategy using a deep reinforcement learning algorithm, the repair path is aligned with the on-site defect status in real time, addressing the traditional method's inability to respond to dynamic defect changes in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other features of the present disclosure will become more apparent through a detailed description of the embodiments shown in conjunction with the accompanying drawings. The same reference numerals in the drawings of the present disclosure represent the same or similar elements. Obviously, the drawings described below are only some embodiments of the present disclosure. It is possible for a person skilled in the art to derive other drawings based on these drawings without inventive effort. In the drawings: Figure 1 Shown is a flow chart of a display panel repair path optimization method based on deep reinforcement learning according to the present invention; Figure 2 Shown is a diagram showing the construction principle of the deep reinforcement learning model of the present invention; Figure 3 Shown is a schematic diagram of part of the hardware structure of the repair device of the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other unless there is a conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0016] Example 1, with reference to Figure 1 The present invention proposes a display panel repair path optimization method based on deep reinforcement learning, which includes the following: Acquire the target display panel image; Preprocessing and feature extraction are performed on the target display panel image to obtain relevant features; Inputting relevant features into a pre-established deep reinforcement learning model, which outputs an optimized repair path and performs display panel repair; The deep reinforcement learning model is a PPO algorithm model built based on deep reinforcement learning with a policy network and a value network as intelligent agents.

[0017] In Example 1, simulation modeling was performed for actual repair scenarios. Using offline training and multi-objective optimization using a deep reinforcement learning (DRL) PPO algorithm, a reward function was constructed that incorporated multiple process constraints (including repair quality, path efficiency, and tool parameters). This enabled repair path planning for a multi-head integrated system with both a laser repair head and an electro-hydraulic inkjet repair head, addressing the path redundancy and time-consuming planning issues inherent in traditional approaches for multi-defect scenarios. By collecting data in real time and continuously updating the reward function and strategy using a DRL algorithm, the repair path was aligned with the on-site defect status in real time, addressing the traditional approach's inability to respond to dynamic defect changes in real time.

[0018] As a preferred embodiment of the present invention, specifically, the target display panel image is preprocessed and feature extracted to obtain relevant features, including: The target display panel image is processed by adaptive bilateral filtering to obtain a preprocessed image, the defect area is separated from the background, and the centroid coordinates, equivalent diameter, and aspect ratio geometric features of the preprocessed image are extracted to obtain relevant features.

[0019] In this preferred embodiment, the collected raw data undergoes preprocessing operations such as dynamic noise reduction, defect segmentation, multimodal fusion, and feature extraction. Geometric features such as centroid coordinates and equivalent diameter / aspect ratio are extracted. These features serve as input to the deep reinforcement learning model. Adaptive bilateral filtering is employed to preserve the dynamic noise reduction method of defect edges, improving the accuracy of subsequent processing and reducing false detections and missed detections. Defect segmentation accurately separates defect areas from the background, providing a foundation for subsequent feature extraction and classification, improving detection accuracy, reducing computational resource waste, and lowering energy consumption.

[0020] Reference Figure 2 As a preferred embodiment of the present invention, a specific pre-established deep reinforcement learning model includes: Based on the actual repair scenario of multiple repair heads, simulation modeling is carried out, and a PPO algorithm model with policy network and value network as intelligent agents is constructed based on deep reinforcement learning; The PPO algorithm model is trained offline for multiple rounds in a simulation environment to obtain the final repair path planning strategy. During the training process, the intelligent agent interacts with the repair process simulation environment, collects interaction trajectory data, and calculates the advantage value and target value; Based on the advantage value and target value, the policy network and value network are updated in small batches to obtain new policy parameters and complete the synchronization of policy parameters.

[0021] As a preferred embodiment of the present invention, specifically, simulation modeling is performed based on the actual repair scenario of multiple repair heads, including: Aiming at the actual repair scenario of multiple repair heads, the path optimization problem of OLED display panel defect repair is modeled. The display panel defect repair process is abstracted into a Markov decision process. For each time t, the system state, action, reward, and next state are expressed as follows: ; in represents the state space, represents the action space, represents the reward function, Represents the new environment after the action is executed; The state space is divided into two parts: panel defect information and repair tool status, using a two-dimensional matrix Indicates panel defect information, that is, displays the defect conditions of each area on the panel. Indicates no defects, Indicates the presence of a defect, i and j are the two-dimensional coordinate indexes of the display panel defect, where i represents the horizontal position coordinate of the panel defect, and j represents the vertical position coordinate of the panel defect. The numerical value represents the severity of the defect. The selection of the repair tool adopts discrete variables. Indicates, where 0 represents the laser repair head and 1 represents the inkjet repair head. The position of the repair tool is represented by two-dimensional coordinates, and the current position is represented by , Indicates the horizontal position coordinates of the current tool. Indicates the longitudinal position coordinate of the current tool, and the inkjet volume and laser power repair parameters of the tool are expressed as The final state space is defined as: ; Where t represents the current time step, and the state space is continuously updated with time step t. M×N represents the dimension of the panel defect information matrix, 2 represents the dimension of the two-dimensional coordinates of the current repair tool position, and P represents the dimension of the tool repair parameter; The selection of a tool is a discrete action, expressed as , indicates whether the dth tool is in the selected state at time step t, where 0 indicates that the tool is not selected and 1 indicates that the tool is selected; the movement of the tool position and the adjustment of the tool parameters are continuous actions, and the offset of the repair head moving on the panel is defined as , represents the action value of the C1 action category at time step t, △X represents the distance the repair tool moves along the horizontal axis at the current time step, △y represents the distance the repair tool moves along the vertical axis at the current time step, and Indicates the change of repair head parameters, at （c2） Represents the action value of the C2 action category at time step t, and the final action space is defined as: ; Reward t The agent performs the action The reward function is set to give feedback from the environment. The reward design is related to the path length, repair quality and energy consumption. The longer the distance moved, the worse the repair quality, and the excessive energy consumption, the negative reward is given. : ; in is the weight of the adjustment parameter, For the ideal repair state, is the moving distance of the repair head, Ideal tool parameters.

[0022] As a preferred embodiment of the present invention, specifically, a PPO algorithm model with a policy network and a value network as intelligent agents is constructed based on deep reinforcement learning, including: To build a PPO algorithm model, we need to build two neural networks, where the policy network is used to generate hybrid policies and the value network is used to estimate state values. Since the action space is a hybrid, the policy function can be decomposed into discrete and continuous parts. Discrete actions use softmax to output probability distribution: ; in Network status Discrete action score, It represents the policy network parameters of the d-th tool. As a learnable parameter in the neural network, it provides policy decisions for the d-th tool. a' represents the set of all tool selection actions, where a'=0 means that the repair tool is not selected at the current time step, and a'=1 means that the repair tool is selected at the current time step. Continuous actions use multi-dimensional Gaussian distribution, and its probability density function is: ; Among them, =[at （c1） ,at （c2） ], is the set of the repair head offset and the repair head parameter change, represents the policy network parameters of the c-th controller; The mean is output by the policy network: ; The value network is used to estimate the long-term return of a state and is defined as: ; in is the discount factor, Neural network weight parameters representing the value function, It represents the immediate reward at step t+k, and the reward of each step is multiplied by the discount factor.

[0023] As a preferred embodiment of the present invention, the method further includes: the objective function and update of the PPO algorithm model, including: By clipping policy updates, we can limit the change between the old and new policies to obtain stable policy improvements and define the probability ratio of the new and old policies: ; in Indicates that under the current policy parameters θ, in state Take action The probability of Indicates that under the old policy, in the state Take action Probability Establish the advantage function and calculate the advantage using the generalized advantage estimate: ; Where T is the end time of a complete trajectory, γ is the reward discount factor, λ is the GAE smoothing factor, which is used to control the trade-off between bias and variance, and Δt+l is the time difference error of the t+lth step; The time difference error is ; The clipping objective function of the policy network is defined as: ; in, Indicates the ratio Restricted to [ ], where ε is the shear threshold The mean squared error loss of the value network is defined as: ; in, Expressed as the target state value, it is composed of the advantage function estimate and the current state value estimate; In order to encourage strategy exploration, the entropy regularization term is introduced: ; in, Indicates that in a given state Next, the probability distribution of action a; The total loss function is: ; in and is a hyperparameter used to balance the contribution of value loss and entropy regularization.

[0024] As a preferred embodiment of the present invention, the method further includes the following training process of the deep reinforcement learning model: Step 1 is the initialization phase, which initializes the policy network parameters. , value network parameters , old policy parameters and a defect repair simulation environment configured for OLED panels; The second step is interaction and data collection. In each training cycle, the current strategy is used Run multiple time steps in the environment and record the state, action, reward and next state at each step, storing the collected data in temporary storage; Step 3 is strategy update and parameter optimization. In order to update the strategy, it is necessary to calculate the advantage estimate for each time step and target value ,Using the collected data and the calculated advantage value, multiple small batch updates are made to the strategy and value network; Step 4 is the synchronization of strategy parameters. After completing the update of several epochs, the updated strategy parameters are synchronized. Assign values ​​to the old strategy parameters to ensure that the latest strategy is used in the next round of data collection; Repeat steps 2 to 4 until the model converges.

[0025] In this preferred embodiment, after multiple rounds of offline simulation training, the model can be used to optimize and execute repair paths. The model can select the correct repair tools and parameters and plan the optimal repair path within an integrated repair system equipped with two parallel repair heads. An online update strategy ensures that real-time data collected is fed into the model, ensuring that the repair path matches the on-site defect situation in real time. This achieves automated and efficient display panel defect repair.

[0026] Through repeated training and optimization of the deep reinforcement learning model, the model is able to achieve high-precision and efficient defect repair while ensuring repair quality while shortening operation time. The model can automatically determine which repair tool to use for different defect areas and plan the optimal repair path, reducing the number of manual interventions, improving overall repair efficiency, and enhancing the consistency and production yield of OLED panels, with significant economic benefits and broad industrial application prospects. Because the state space incorporates multidimensional information and the reward mechanism considers multiple factors, the model can adaptively adjust its strategy when faced with different defect types, locations, and severities, showing strong generalization and robustness.

[0027] Combine Figure 3 In Example 2, the present invention further proposes a device for optimizing a display panel repair path based on deep reinforcement learning, comprising the following: A base, wherein a motion guide rail is provided on the upper end thereof, a gantry beam is provided on the motion guide rail, the gantry beam is capable of sliding along the motion guide rail, and the gantry beam is arranged perpendicular to the motion guide rail; The laser repair head and the inkjet repair head are respectively fixed on the gantry crossbeam and can slide along the gantry crossbeam; A processing module, electrically connected to the laser repair head and the inkjet repair head, for acquiring an image of a target display panel through an image acquisition module; Preprocessing and feature extraction are performed on the target display panel image to obtain relevant features; Input relevant features into a pre-established deep reinforcement learning model, which outputs an optimized repair path and controls the laser repair head and the inkjet repair head to repair the display panel; The deep reinforcement learning model is a PPO algorithm model built based on deep reinforcement learning with a policy network and a value network as intelligent agents.

[0028] As a preferred embodiment of the present invention, specifically, the image acquisition module is realized by multiple industrial cameras and laser displacement sensors installed on the production line. The industrial cameras include visible light cameras and infrared cameras. After the display panel is in place, a trigger signal is sent through the photoelectric sensor, and multiple industrial cameras and laser displacement sensors are started at the same time. The visible light camera captures surface microscopic defects, and the infrared camera detects internal defects to obtain the target display panel image. The laser displacement sensor records the three-dimensional coordinates of the optical head in real time, and the IEEE1588 protocol is used to achieve microsecond-level synchronization of multiple devices. The target display panel image is compressed by the acquisition card and transmitted to the processing module through the PCle bus.

[0029] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0030] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system that can carry the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0031] Although the present invention has been described in considerable detail and with particularity with respect to several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but rather should be construed as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively encompassing the intended scope of the invention. In addition, the invention has been described above in terms of embodiments foreseen by the inventors for the purpose of providing a useful description, and those insubstantial modifications of the invention that are not currently foreseen may still represent equivalent modifications of the invention.

[0032] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. As long as the technical effects of the present invention are achieved by the same means, they shall fall within the scope of protection of the present invention. Within the scope of protection of the present invention, various modifications and variations of the technical solutions and / or implementation methods may be made.

Claims

1. A display panel repair path optimization method based on deep reinforcement learning, characterized in that: These include: Acquire the target display panel image; Preprocessing and feature extraction are performed on the target display panel image to obtain relevant features; Inputting relevant features into a pre-established deep reinforcement learning model, which outputs an optimized repair path and performs display panel repair; The deep reinforcement learning model is a PPO algorithm model built based on deep reinforcement learning with a policy network and a value network as intelligent agents.

2. The display panel repair path optimization method based on deep reinforcement learning according to claim 1, characterized in that: Specifically, the target display panel image is preprocessed and feature extracted to obtain relevant features, including: The target display panel image is processed by adaptive bilateral filtering to obtain a preprocessed image, the defect area is separated from the background, and the centroid coordinates, equivalent diameter, and aspect ratio geometric features of the preprocessed image are extracted to obtain relevant features.

3. The display panel repair path optimization method based on deep reinforcement learning according to claim 1, characterized in that: Specific, pre-built deep reinforcement learning models, including, Based on the actual repair scenario of multiple repair heads, simulation modeling is carried out, and a PPO algorithm model with policy network and value network as intelligent agents is constructed based on deep reinforcement learning; The PPO algorithm model is trained offline for multiple rounds in a simulation environment to obtain the final repair path planning strategy. During the training process, the intelligent agent interacts with the repair process simulation environment, collects interaction trajectory data, and calculates the advantage value and target value; Based on the advantage value and target value, the policy network and value network are updated in small batches to obtain new policy parameters and complete the synchronization of policy parameters.

4. The display panel repair path optimization method based on deep reinforcement learning according to claim 3, characterized in that: Specifically, simulation modeling is performed based on the actual repair scenario of multiple repair heads. include, Aiming at the actual repair scenario of multiple repair heads, the path optimization problem of OLED display panel defect repair is modeled. The display panel defect repair process is abstracted into a Markov decision process. For each time t, the system state, action, reward, and next state are expressed as follows: ； in represents the state space, represents the action space, represents the reward function, Represents the new environment after the action is performed; The state space is divided into two parts: panel defect information and repair tool status, using a two-dimensional matrix Indicates panel defect information, that is, displays the defect conditions of each area on the panel. Indicates no defects, Indicates the presence of a defect, i and j are the two-dimensional coordinate indexes of the display panel defect, where i represents the horizontal position coordinate of the panel defect, and j represents the vertical position coordinate of the panel defect. The numerical value represents the severity of the defect. The selection of the repair tool adopts discrete variables. Indicates, where 0 represents the laser repair head and 1 represents the inkjet repair head. The position of the repair tool is represented by two-dimensional coordinates, and the current position is represented by , Indicates the horizontal position coordinates of the current tool. Indicates the longitudinal position coordinate of the current tool, and the inkjet volume and laser power repair parameters of the tool are expressed as The final state space is defined as: ； Where t represents the current time step, the state space is continuously updated with the time step t, M×N represents the dimension of the panel defect information matrix, 2 represents the dimension of the two-dimensional coordinate of the current repair tool position, and P represents the dimension of the tool repair parameter; The selection of a tool is a discrete action, expressed as , indicates whether the dth tool is in the selected state at time step t, where 0 indicates that the tool is not selected and 1 indicates that the tool is selected; the movement of the tool position and the adjustment of the tool parameters are continuous actions, and the offset of the repair head moving on the panel is defined as , represents the action value of the C1 action category at time step t, △X represents the distance the repair tool moves along the horizontal axis at the current time step, △y represents the distance the repair tool moves along the vertical axis at the current time step, and Indicates the change of repair head parameters, at （c2） Represents the action value of the C2 action category at time step t, and the final action space is defined as: ； Reward t The agent performs the action The reward function is set to give feedback from the environment. The reward design is related to the path length, repair quality and energy consumption. The longer the distance moved, the worse the repair quality, and the excessive energy consumption, the negative reward is given. : ； in is the weight of the adjustment parameter, For the ideal repair state, is the moving distance of the repair head, Ideal tool parameters.

5. The display panel repair path optimization method based on deep reinforcement learning according to claim 4, characterized in that: Specifically, based on deep reinforcement learning, a PPO algorithm model with a policy network and a value network as intelligent agents is constructed, including: To build a PPO algorithm model, we need to build two neural networks, where the policy network is used to generate hybrid policies and the value network is used to estimate state values. Since the action space is a hybrid, the policy function can be decomposed into discrete and continuous parts. Discrete actions use softmax to output probability distribution: ； in Network status Discrete action score, It represents the policy network parameters of the d-th tool. As a learnable parameter in the neural network, it provides policy decisions for the d-th tool. a' represents the set of all tool selection actions, where a'=0 means that the repair tool is not selected at the current time step, and a'=1 means that the repair tool is selected at the current time step. Continuous actions use multi-dimensional Gaussian distribution, and its probability density function is: ； Among them, =[at （c1） ,at （c2） ], is the set of the repair head offset and the repair head parameter change, represents the policy network parameters of the c-th controller; The mean is output by the policy network: ; The value network is used to estimate the long-term return of a state and is defined as: ； in is the discount factor, Neural network weight parameters representing the value function, It represents the immediate reward at step t+k, and the reward of each step is multiplied by the discount factor.

6. The display panel repair path optimization method based on deep reinforcement learning according to claim 5, characterized in that: The method also includes the objective function and update of the PPO algorithm model, including: By clipping policy updates, we can limit the change between the old and new policies to obtain stable policy improvements and define the probability ratio of the new and old policies: ； in Indicates that under the current policy parameters θ, in state Take action The probability of Indicates that under the old policy, in the state Take action Probability Establish the advantage function and calculate the advantage using the generalized advantage estimate: ； Where T is the end time of a complete trajectory, γ is the reward discount factor, λ is the GAE smoothing factor, which is used to control the trade-off between bias and variance, and Δt+l is the time difference error of the t+lth step; The time difference error is ; The clipping objective function of the policy network is defined as: ； in, Indicates the ratio Restricted to [ ], where ε is the shear threshold The mean squared error loss of the value network is defined as: ； in, Expressed as the target state value, it is composed of the advantage function estimate and the current state value estimate; In order to encourage strategy exploration, the entropy regularization term is introduced: ； in, Indicates that in a given state Next, the probability distribution of action a; The total loss function is: ； in and is a hyperparameter used to balance the contribution of value loss and entropy regularization.

7. The display panel repair path optimization method based on deep reinforcement learning according to claim 3, characterized in that: The method also includes the following process of training the deep reinforcement learning model: Step 1 is the initialization phase, which initializes the policy network parameters. , value network parameters , old policy parameters and a defect repair simulation environment configured for OLED panels; The second step is interaction and data collection. In each training cycle, the current strategy is used Run multiple time steps in the environment and record the state, action, reward and next state at each step, storing the collected data in temporary storage; Step three is strategy update and parameter optimization. In order to update the strategy, it is necessary to calculate the advantage estimate of each time step. and target value ,Using the collected data and the calculated advantage value, multiple small batch updates are made to the strategy and value network; Step 4 is the synchronization of strategy parameters. After completing the update of several epochs, the updated strategy parameters are synchronized. Assign values ​​to the old strategy parameters to ensure that the latest strategy is used in the next round of data collection; Repeat steps 2 to 4 until the model converges.

8. A device for optimizing display panel repair paths based on deep reinforcement learning, characterized in that: These include: A base, wherein a motion guide rail is provided on the upper end thereof, a gantry beam is provided on the motion guide rail, the gantry beam is capable of sliding along the motion guide rail, and the gantry beam is arranged perpendicular to the motion guide rail; The laser repair head and the inkjet repair head are respectively fixed on the gantry crossbeam and can slide along the gantry crossbeam; A processing module, electrically connected to the laser repair head and the inkjet repair head, for acquiring an image of a target display panel through an image acquisition module; Preprocessing and feature extraction are performed on the target display panel image to obtain relevant features; Input relevant features into a pre-established deep reinforcement learning model, which outputs an optimized repair path and controls the laser repair head and the inkjet repair head to repair the display panel; The deep reinforcement learning model is a PPO algorithm model built based on deep reinforcement learning with a policy network and a value network as intelligent agents.

9. The device for optimizing display panel repair path based on deep reinforcement learning according to claim 8, characterized in that: Specifically, the image acquisition module is realized by multiple industrial cameras and laser displacement sensors installed on the production line. The industrial cameras include visible light cameras and infrared cameras. After the display panel is in place, a trigger signal is sent through the photoelectric sensor to start multiple industrial cameras and laser displacement sensors at the same time. The visible light camera captures surface microscopic defects, and the infrared camera detects internal defects to obtain the target display panel image. The laser displacement sensor records the three-dimensional coordinates of the optical head in real time, and the IEEE1588 protocol is used to achieve microsecond-level synchronization of multiple devices. The target display panel image is compressed by the acquisition card and transmitted to the processing module through the PCle bus.

Images